FlashGrid Server (non-clustered) on Azure. Deployment Guide – FlashGrid Help Center

rev. 26.05

1 Overview
    1.1 Key Benefits
    1.2 Key Components
    1.3 Infrastructure-as-Code Deployment
2 Prerequisites
3 Deploying FlashGrid Server
4 After Deploying
    4.1 Verifying the Status
    4.2 Verifying Synchronization of Clocks
    4.3 OS User Accounts
    4.4 Finalizing System Configuration
    4.5 Adding a Protection Lock
    4.6 Installing an Additional Database Home
    4.7 Use of Anti-virus and Other Third-party Software
    4.8 Use of Automatic Configuration Tools
    4.9 Security Hardening
    4.10 Convert RHEL PAYG to BYOS License
5 Monitoring FlashGrid Server Health
6 Before Going Live
7 Deleting a System
8 Additional Documentation
9 Technical Support

1 Overview

FlashGrid Server is an engineered cloud system that helps you achieve reliable Oracle Database operation and maximum uptime on Azure cloud VMs.

1.1 Key Benefits

Key benefits of using FlashGrid Server for running Oracle Database on Azure VMs

Automated and repeatable Infrastructure-as-Code deployment with the help of FlashGrid Launcher tool.
Operating system optimized in accordance with proven best practices for running Oracle Database.
Extensive validation for compatibility and reliable operation between the Azure VMs, the OS, and the Oracle Database, including OS updates and Oracle patches.
Monitoring, logging, and diagnostics tools that help prevent and analyze reliability issues.
24x7 support for the entire infrastructure stack provided by engineers with Azure cloud, Linux OS, and Oracle Database expertise.

1.2 Key Components

Key components of FlashGrid Server on Azure

FlashGrid Storage Fabric software
FlashGrid Diagnostics software
FlashGrid Health Checker software
Oracle Database: 19c, or 26ai
Oracle Grid Infrastructure: 19c, or 26ai
Operating Systems:
- Oracle Linux: 8 (UEKR7), or 9 (UEKR7)
- Red Hat Enterprise Linux (RHEL): 8, or 9
Note: Contact FlashGrid support if new deployment with RHEL 7 or Oracle Linux 7 is required
Azure VMs:
- General purpose: Dasv7, Dsv6, Dasv6, Dsv5
- Compute optimized: FXmsv2
- Memory optimized: Easv7, Esv6, Easv6, Ebsv5, Esv5, Mbsv3, Msv3
Disks: Premium SSD or Premium SSD v2

1.3 Infrastructure-as-Code Deployment

FlashGrid Server is delivered as Azure Resource Manager templates that automate the configuration of multiple components required for a database. FlashGrid Launcher is an online tool that simplifies the deployment process by guiding through the system configuration parameters and generating Azure Resource Manager templates.

2 Prerequisites

The following prerequisites are required for automated deployment of a FlashGrid Server:

Azure Storage Blob Container with Oracle installation files that will be downloaded to the system during software initialization. FlashGrid Launcher will show you the list of files that must be placed in the Storage Container. The files in the Storage Container must be accessible from the VM in the target VNet. For configuring access permission options, see the Knowledge Base article Uploading Oracle Installation Files to Azure Storage Container.
Microsoft.Storage service endpoint configured for the VNet. Having the storage service endpoint allows access to the storage container from the VM. If Microsoft.Storage service endpoint is not added, and a public IP is not assigned then system initialization will fail because downloading Oracle files from the VM will not be possible.
Azure subscription with sufficient quotas for creating the required VM type and sufficient number and size of Premium Managed Disks.
SSH key pair that will be used for accessing the VM. Use of passwords instead of the key pair is not supported. To create a new key pair use ssh-keygen in Linux or puttygen in Windows. In the FlashGrid Launcher tool you will need to provide the public key that will be placed on the VM. Example of a valid public key pair format:
```
ssh-rsa <PublicKeyBody>
```
Keep blank if using Azure managed SSH keys for VMs access.
Properly configured Network Security Group (NSG) when deploying in an existing VNet. You have a choice of attaching an NSG to the VM or using the NSG attached to the subnet. The following ports should be open:
- Inbound: TCP port 22 for SSH access to the VM
- Inbound: TCP port 5901 if you choose to use VNC for creating a database using DBCA in GUI mode with direct connection (vs. SSH tunnel)
- Inbound: TCP port 1521 for database client and application server access
- Inbound access to the ports listed above must be allowed only from those security groups or IP ranges that require such access. Do not configure Any or 0.0.0.0/0 as allowed sources.
FlashGrid recommends configuring the NSG rules by using an Application Security Group (ASG).

3 Deploying FlashGrid Server

The FlashGrid Launcher tool simplifies provisioning of Oracle Database system in Azure by automating the following tasks:

Creating cloud infrastructure: VM, storage, and optionally network
Installing and configuring FlashGrid Server software
Installing, configuring, and patching Oracle Grid Infrastructure
Installing and patching Oracle Database software
Creating ASM disk groups

To create a VM

Open FlashGrid Launcher tool:
- Start with one of the standard configurations at https://www.flashgrid.io/products/flashgrid-for-oracle-db-on-azure
- or, if you have a custom configuration file, upload it at https://latest.cloudprov.flashgrid.io/
Configure parameters for the deployment
Click Validate Configuration button
If verification passes then click Launch FlashGrid button, which will take you to Azure Resource Manager
Select Resource group - Create new. By having the system in a separate resource group you can later delete all infrastructure by simply deleting the resource group.
Enter a name for the new resource group that will contain the infrastructure. A name matching the system name is recommended.
Select your target location (region)
If you did not provide an SSH key, select Use existing key stored in Azure in SSH public key source and specify a stored key.
Check 'I agree to the terms and conditions stated above'
Click Purchase
Open list of Notifications (bell icon) and click 'Deployment in progress…'
Wait until the deployment status changes to Succeeded
If the deployment fails:
1. Check for the cause of the failure in the Operation details
2. Correct the cause of the error
3. Delete the failed resource group
4. Repeat the steps for creating a new resource group
SSH to the VM as user az-admin
The welcome message will show the current software initialization status: in progress, failed, or completed.
If software initialization is still in progress then wait for it to complete (this includes Oracle software installation and configuration). You will receive a broadcast message when initialization completes or fails. Software initialization takes approximately 30 minutes.

Note: for deploying FlashGrid Server with SELinux please refer to the Knowledge Base article: How to configure SELinux mode during system deployment.

4 After Deploying

4.1 Verifying the Status

On the VM run flashgrid-health-check command to verify that the status is Good and all checks are passing.

[az-admin@myhostname ~]$ sudo flashgrid-health-check
HealthCheck 20.9.1.51823.test #43fdf490ae61edb4febd0f6f378fb56dfc6a3036
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Check: ASM DiskGroup status
   myhostname: OK
      --------------------------------------------------------------------------------------------------------
      GroupName  Status  Mounted   Type    TotalMiB  FreeMiB  OfflineDisks  LostDisks  Resync  ReadLocal  Vote
      --------------------------------------------------------------------------------------------------------
      DATA       Good    AllNodes  EXTERN  11264     11172    0             0          No      Enabled    N/A
      FRA        Good    AllNodes  EXTERN  12288     12196    0             0          No      Enabled    N/A
      GRID       Good    AllNodes  EXTERN  5120      5020     0             0          No      Disabled   N/A
      --------------------------------------------------------------------------------------------------------

Check: Alerts in Storage Fabric logs in the last 7 days
      myhostname: OK

Check: Available memory
      myhostname: OK : avail mem: 32.8%

Check: Check db memory settings
      myhostname: OK

Check: Check local_listener for each db
      myhostname: OK

Check: Check tnsnames.ora
      myhostname: OK

Check: Flashgrid CLAN check
      myhostname: OK

Check: Free system disk space
      myhostname: OK : /u01: avail 67%, /: avail 95%

Check: Kernel taint check
      myhostname: OK

Check: SF node status
      myhostname: OK

Check: Swap disabled
      myhostname: OK : Swap disabled

Check: System config file modifications
      myhostname: OK

Check: System services
      myhostname: OK

Check: Unexpected or 3rd party RPMs installed
      myhostname: OK

Check: Unexpected or 3rd party services enabled
        myhostname: OK

4.2 Verifying Synchronization of Clocks

Chrony service is used for synchronizing a VM clock with external NTP servers. Without active clock synchronization service the clocks are likely to get out of sync.

To check status of CHRONYD service

sudo chronyc sources

Example:

[az-admin@myhostname ~]$ sudo chronyc sources
210 Number of sources = 4
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===============================================================================
^* time1.google.com              1   6   177    58  -2383us[-4178us] +/-   12ms
^+ time2.google.com              1   6   177    58   -164us[ -164us] +/-   12ms
^+ time3.google.com              1   6   177    58  -1798us[-1798us] +/-   11ms
^+ time4.google.com              1   6   177    57   -165us[ -165us] +/- 9051us

Note that the '*' character shows which NTP server is currently used for synchronization. Normally this should be one of the external NTP servers. If it shows that VM is used for synchronization, then this means that the external NTP servers are not accessible.

Public NTP servers, e.g. timeX.google.com, can be used only if public IPs are enabled on the VM (not recommended in production use for security reasons) or if NAT is configured in the network. If needed, the list of NTP servers can be modified in /etc/chrony.conf after the software is configured.

4.3 OS User Accounts

During software initialization the following OS user accounts are created:

az-admin - the user account used to SSH to the VM with the SSH key that was selected when creating the software configuration. The user has sudo rights.
fg - can be used for running FlashGrid Storage Fabric or FlashGrid Cloud Area Network utilities. The user has sudo rights.
grid - Grid Infrastructure owner. GI environment variables are preconfigured.
oracle - Database home owner. Database environment variables, except ORACLE_SID and ORACLE_UNQNAME, are preconfigured. After creating a database you can configure ORACLE_SID and ORACLE_UNQNAME by editing /home/oracle/.bashrc file on the VM.

Note that no passwords are configured for any users. Also password-based SSH authentication is disabled in /etc/ssh/sshd_config. Key-based authentication is recommended for better security. Creating passwords for any user is not recommended.

Users az-admin and fg have sudo rights and allow switching to any other user without requiring a password (which is not configured by default). Example:

sudo su - grid

4.4 Finalizing System Configuration

See the Knowledge Base article Creating a Database for instructions.

Note: ACFS support on RHEL may require an additional Oracle Clusterware patch. Please refer to Oracle KB129209 for ACFS patch information.

4.5 Adding a Protection Lock

It is strongly recommended to add a lock to the resource group to protect it against accidental deletion or modification.

4.6 Installing an Additional Database Home

In most cases manual installation of database software is not required. However, if you need an additional software then follow Oracle Database documentation for installing the database software.

4.7 Use of Anti-virus and Other Third-party Software

If anti-virus software has to be used then it is recommended to configure it in a way that avoids putting any files in quarantine. Automatic quarantine of files creates risk of downtime in case of a false positive detection on a critical system file on the VM.

Any proprietary kernel modules installed by third-party software create risks to reliable operation of the system. Such proprietary kernel modules are not tested or supported by FlashGrid, Red Hat, or Oracle Linux. Proprietary kernel modules may consume kernel resources and may create instability, especially under high load. Symptoms may include kernel crashes, network disruptions, storage i/o disruptions, and server brown-out. If such reliability issue is encountered and no other root cause can be readily identified, FlashGrid support reserves the right to request removal of all proprietary kernel modules before continuing investigation.

4.8 Use of Automatic Configuration Tools

Automatic configuration tools (e.g. Ansible, Salt, etc.) must be used with extra care. Incorrect modification of a critical system file (e.g. /etc/resolv.conf) may cause system downtime. Note that many critical system configuration files are protected with immutable attribute and have warnings in them. Do not remove the immutable attribute or allow automatic modification of such files unless absolutely necessary.

4.9 Security Hardening

The system is deployed using RHEL or Oracle Linux images that have main security best practices implemented by default. The following steps are recommended, in case additional security hardening is required:

Request FlashGrid support to review the list of required changes
Back up the system
Implement the required changes
Restart the system
Verify health of the system as user fg:
```
flashgrid-health-check
```
In case of errors, roll back the changes or restore the system from backup

4.10 Convert RHEL PAYG to BYOS License

If RHEL is used, an Azure VM is provisioned with a RHEL PAYG license attached. To adopt BYOS, the RHEL license must be converted as per the procedure documented by Azure, following server deployment.

5 Monitoring FlashGrid Server Health

The following methods of monitoring system health are available:

flashgrid-health-check utility checks multiple items including database configuration, storage, OS kernel, config file modifications, errors in the logs, and other items that may affect health of the system or could help with troubleshooting. It is recommended for manual checks only.
FlashGrid Node Monitor service is part of the flashgrid-diags package. It provides monitoring of various system health indicators, including CPU utilization, available memory, and clocks.
Alerts about failures are recorded in system log and can be analyzed by 3rd-party tools.
Email alerts can be configured in FlashGrid Launcher and sent to one or several email addresses. See the Knowledge Base article Configuring email for FlashGrid notifications.
FlashGrid Node Monitor can send alerts and diagnostic uploads via HTTP/HTTPS to a remote endpoint, with support for routing through an HTTP proxy. Node Monitor detects a condition, then sends either an email alert and/or an HTTP/REST request to a monitoring/alerting service.
ASM disk group monitoring and alerting via Oracle Enterprise Manager.

6 Before Going Live

Before switching to live use (run commands as user fg):

Confirm that only minimally required access to the server is allowed in network security settings, and remove any unnecessary access
Verify health of the VM: flashgrid-health-check
Confirm that email alerts are configured and delivered: flashgrid-node test-alerts
Upload diags to FlashGrid support: flashgrid-diags upload-all
Stop the VM and back it up
Start the VM and do a final health check: flashgrid-health-check

7 Deleting a System

To delete a system

Delete any protection lock(s) for the resource group
Delete the resource group corresponding to the system

8 Additional Documentation

FlashGrid Server on Azure Knowledge Base

FlashGrid Server (non-clustered) on Azure Backup Best Practices

FlashGrid Storage Fabric CLI Reference Guide

FlashGrid Cloud Area Network CLI Reference Guide

9 Technical Support

For technical help with FlashGrid Server please open a support request.

To expedite troubleshooting please also collect and upload diagnostic data to the secure storage used by FlashGrid support by running the following command as user fg:

flashgrid-diags upload-all

For reporting emergency type of issues that require immediate attention please also use the 24/7 telephone hotline: +1-650-641-2421 ext 7. Please note that use of the 24/7 hotline is reserved for emergency situations only.

Support Tiers and SLA details document: FlashGrid Technical Support Services.

Table of Contents