FlashGrid Cluster on Azure. Backup Best Practices – FlashGrid Help Center

This article covers backup procedures that must be used for different types of data including:

OS and software on the cluster nodes
Database files
Files on ACFS, if used

Backing up OS and software on the cluster nodes

It is strongly recommended to back up the OS disk and software disk (LUN 0) of all cluster nodes after the initial cluster configuration and before and after applying any changes, such as patch installation or security settings changes.

Azure VM Backup service is the recommended method for backing up the disks.

Note: if the node is a quorum or storage node, then only the OS disk must be backed up.

To configure Azure VM backups, use either the Backup Center in Azure Portal, or the CLI.

CLI examples are provided below:

Using Azure CLI configure disk backup of the Azure VM:
- Backing up LUN0 and OS disks (on database nodes):
  
  az backup protection enable-for-vm --resource-group {resourcegroup} --vault-name {vaultname} --vm {vmname} --policy-name {policyname} --disk-list-setting include --diskslist 0
  
  The backup configured by the above script includes both LUN0 and OS disks, because the OS disk is by default added to the VM backup and can't be excluded.
- Backing up only OS disk (on quorum or storage nodes):
  
  az backup protection enable-for-vm --resource-group {resourcegroup} --vault-name {vaultname} --vm {vmname} --policy-name {policyname} --exclude-all-data-disks

Verify that the backup configuration contains only OS and LUN0 disks:

Connect to the Azure portal -> Recovery Services Vault -> {vaultname} -> Backup Items (Azure Virtual Machine) -> click {vmname} -> check Included disk(s) section.

Click Backup now from the Azure portal, or use CLI to initiate the backup and wait until the job is successfully completed before applying any changes.

Note: fsFreeze feature of Azure VM Backup is disabled on clusters deployed with FlashGrid Launcher 21.11 or newer because of excessively long freezes potentially triggering node fencing.

Restoring root or software disk of a cluster node

The root volume or the software volume may need to be restored if the volume fails, has file system corruption, or has logical corruption.

Note: Before using this procedure, open a ticket with Flashgrid Support

Restoring OS disk from Azure VM Backup

a) Restore the OS disk for the damaged server from the latest available "good" Azure VM backup. The restored disk should be placed in the resource group and storage account used by the affected cluster.

b) Stop the damaged cluster node

c) Locate the damaged node in the Azure console, open the Disks blade, select "Swap OS Disks", and choose the restored disk, for example rac2-osdisk-20230503-043518

d) Start the node.

e) Check cluster status:

# flashgrid-cluster

Wait until all disks are back online, and resyncing operations are complete on all disk groups before starting the database instances on the node.

f) After all the steps above are complete, upload diags to FlashGrid support.

Restoring software disk from Azure VM Backup

a) Restore the software disk volume (lun 0) for the damaged server from the latest available "good" Azure VM backup. The restored disk should be placed in the resource group and storage account used by the cluster.

b) Stop the damaged cluster node

c) Locate the damaged node in the Azure console, open the Disks blade.

d) Alongside lun0, select the X icon to delete the current lun0 disk, then Save.

e) Click on "Attach existing disks", select LUN 0, and enter the restored lun0 disk name, for example rac2-datadisk-000-20230505-054329. Confirm "Host caching" is set to "Read-only", then Save.

f) Start the node.

g) Check cluster status:

# flashgrid-cluster

Wait until all disks are back online, and resyncing operations are complete on all disk groups before starting the database instances on the node.

h) After all the steps above are complete, upload diags to FlashGrid support.

Backing up database files

Standard RMAN procedures should be used for backing up and restoring database files.

Two main options for backup destination are available:

A disk attached to one of the database node VMs
Azure File Share

Configuring a disk attached to one of the database node VMs for database backup

This option allows higher performance of the backups and does not consume network bandwidth.
The disk must be attached with a LUN number 50 through 64 (LUNs 1 through 49 are automatically shared by Storage Fabric).
The disk must be formatted with a local file system, such as XFS.
Disk snapshots can be used for exporting backup data to external locations (Make sure that RMAN backup is complete before you make a snapshot of the backup disk.)

Configuring Azure File Share for database backup

Create a Standard performance Storage Account and add a file share.
Install cifs-utils package on all database nodes:
```
# yum install cifs-utils -y
```
Make sure port 445/TCP is not blocked:

Azure Files uses SMB protocol, which communicates over TCP port 445. If you're having trouble mounting your Azure file share, make sure your firewall is not blocking TCP port 445.

Create a backup mount point on each database node:

# mkdir /db_backup
# chown oracle:oinstall /db_backup
# chmod 755 /db_backup

Create a persistent mount point for the Azure file share in /etc/fstab on each database node. Example:
```
//mystorageaccount.file.core.windows.net/myfileshare  /db_backup cifs vers=3.0,username=mystorageaccount,password=c3N+JcWQk7bw0yTT/yAMBmDcEFcRa4AkPQp+zpaljj6e...,dir_mode=0755,file_mode=0755,gid=2000,uid=3001
```
Where uid and gid options are Oracle software owner user and group IDs respectively. If you have a customized user, then you need to change these parameters accordingly.
Mount the backup folder:
```
# mount /db_backup
```

Backing up Grid Infrastructure configuration files

Please follow Backup Best Practices: Grid Infrastructure Configuration

Backing up files on ACFS

For backing up and restoring files on ACFS use the same tools and procedures that you would normally use for file-level backup and restore.