Skip to main content
FlashGrid.io

How to move VM to another Availability Zone

  • Updated
  • Products: FlashGrid Cluster on Azure

Note: This procedure requires stopping the entire cluster.

1. Get a list of LUNs used by ASM:

$ flashgrid-node

From the Azure Console, you will see the following volumes:

  • *-root - OS disk
  • LUN0 - /u01
  • LUN1 through LUN49  - volumes used by ASM. This LUN range is automatically included and shared by FlashGrid. If you have increased the range in the configuration to have more volumes, please keep that in mind.

If there are any other volumes, these are custom volumes not used by FlashGrid. This article does not cover them, so they must be handled accordingly.

2. Power off the VM that should be moved to a different AZ

3. Locate the Disks section of the target VM, select the root device, and create a snapshot by choosing:

  • Basics tab -> Snapshot type -> choose "Full"
  • Networking tab -> Network access -> choose "Disable public and private access"

If it is a database node, create a snapshot of LUN0 disk using a similar method.

In the case of the quorum node, create snapshots of all 1GB ASM disks. 

4. Locate snapshots and create corresponding disks using the following options:

  • Basics tab -> Availability zone -> choose the target AZ
  • Networking tab -> Network access -> choose "Disable public and private access"

5. Create a new AN-enabled NIC with a specific name, VNet, subnet, NSG in a chosen resource group and statically allocated IP. Accelerated-networking option must be set to True.

 Azure CLI example:

az network nic create \
--resource-group <resource group name> \
--name <assign a name to NIC> \
--vnet-name <VNet name> \
--subnet default \
--accelerated-networking true \
--network-security-group flashgrid-db-nsg \
--private-ip-address <indicate private IP address for a new VM>

6. Create a VM from the recently created OS disk. Indicate resource group, VM name, plan details (retrieve it from other VMs), OS disk name, OS type, size, previously created NIC name and Zone number.

 Azure CLI example:

az vm create \
--resource-group <resource group name> \
--name <new VM name> \
--plan-name cluster-ol8 \
--plan-product flashgrid-skycluster \
--plan-publisher flashgrid-inc \
--attach-os-disk <previously created OS disk name> \
--os-type Linux \
--size <VM type> \
--nics <previously created NIC name> \
--zone <target AZ number>

7. Attach previously created disk(s) in step 4

  • If it is a DB node, attach the Oracle software disk as LUN0.
  • In the case of the quorum node, attach all 1GB ASM disk(s) starting with LUN1. Please match the numbers to their respective source disks.

8. For database node only, under newly created VM settings, go to the "Disks" section, create/attach new disks, starting with LUN 1.

Ensure that this node has the same number, size, and performance of disks per diskgroup as the old node and other database nodes. If you are using Premium SSD v2 disks, ensure that you select the same sector size (e.g. 512). If you plan to add disks using the "Create and attach a new disk" option and need to change the sector size before disk creation, you'll need to click the "edit" button situated at the end of the disk information line, just before the "detach" button.

9. Verify that passwordless SSH connection works from the other nodes for the fg user using the new IP address, e.g.:

$ ssh fg@10.0.0.17 hostname
rac1.mycompany.mydomain

Replace 10.0.0.17 with the IP address of your new virtual machine.

10. Stop all databases using srvctl command.

11. Stop clusterware:

# crsctl stop cluster -all

12. On a new node, comment the line containing HWADDR in /etc/sysconfig/network-scripts/ifcfg-eth0.

sed -i s/HWADDR/#HWADDR/g /etc/sysconfig/network-scripts/ifcfg-eth0

13. Reconfigure FlashGrid services:

Note: When we refer to "All nodes", we imply all database and quorum nodes, including newly created/migrated node.

    • All nodes: Temporarily remove the immutable attribute from /etc/hosts file: 
      $ sudo chattr -i /etc/hosts
    • All nodes: Update the IP address of the migrated VM in the /etc/hosts file.
    • All nodes: Restore the immutable attribute on the /etc/hosts file: 
      $ sudo chattr +i /etc/hosts
    • First DB node: Deploy the CLAN configuration to all nodes: 
      $ sudo flashgrid-clan-cfg deploy-config

14. Stop all nodes (do not start any node until all nodes are stopped)

15. Start all of the virtual machines except for the old (migrated) VM(s). You need to avoid starting an old node again. 

16. Verify that ASM disk groups are mounted on all DB nodes:

$ flashgrid-cluster

Note: After starting all VMs, wait a few minutes for all services to come up. If you see a warning status, check the status again later.

17. Start all databases

18. Modify DNS record and/or client connect strings with the new IP address.

19. In case you've migrated a database node, replace the old IP address with the new one in the /etc/motd file. Repeat the same on all database and quorum nodes. 

# sed -i s/10.0.0.7/10.0.0.17/g /etc/motd

Please replace 10.0.0.7 with an old IP address and 10.0.0.17 with a new IP address in the above command.

20. Wait until there are no offline disks, and the cluster status is Good:

$ flashgrid-cluster

It takes some time to resync the data on the ASM disk groups.