Oracle Business Continuity and Disaster Recovery on VMware Hybrid Multi-Clouds

Customers have successfully run their business-critical Oracle workloads with high-performance demands on VMware vSphere for many years.

Common concerns that often delay virtualization of business-critical database workloads include:

• Rapid database growth and the need to reduce backup windows to meet performance and business SLAs
• The size of modern databases makes it harder to regularly clone and refresh data from production to QA and other environments
• Choosing the optimum business continuity plan to ensure rapid recovery from significant disruption to the operations
• Making the Correct choice of disaster recovery technology to ensure business needs of RTO and RPO are met

Solution

This paper describes the configuration and implementation of various business continuity and disaster recovery options (Native Oracle , SRM, vSphere Replication, Array based using VMFS/vVOLS, VSR , VCDR) across the application, VMware platform, and storage levels of Oracle single instance and Real Application Cluster (RAC) workloads on the VMware vSphere hybrid multi-cloud platform. This includes on-premises and VMware clouds, with an emphasis on VMware Cloud™ on AWS.

This solution validates the business continuity and disaster recovery functionality of Oracle Single-Instance and Oracle RAC deployments using Pure x50 Storage (VMFS & vVOL) at all three below levels at on-premises and/or VMware clouds:

Business Continuity

• Application level
• vSphere level
• Storage level

Disaster Recovery

• Application level
• vSphere level
• Storage level

The choice of the business continuity or disaster recovery solution to adopt depends on application needs, SLAs, RTO, RPO and various other factors.

The above business continuity and disaster recovery methods can be summarized in the illustration below:

Audience

This reference architecture is intended for Oracle database administrators (DBAs) as well as virtualization and storage architects involved in planning, architecting, and administering business continuity and disaster recovery processes for business-critical Oracle environments on the VMware SDDC platform.

The paper can be found at Oracle Business Continuity and Disaster Recovery on VMware Hybrid Multi-Clouds REFERENCE ARCHITECTURE

Acknowledgements

Thanks to the following for their technical contributions and help with Lab setup:

• Cato Grace – Senior Technical Marketing Architect, CPBU
• Michael McLaughlin – Senior Technical Marketing Architect, DRaaS Engineering

Thanks to the following for their reviews:

• Jason Massae – Storage Technical Marketing Architect, VMware Core Storage (Storage Product Marketing) –

Moving away from DBA-as-a-Service to Database as a Service (DBaaS) for Oracle Workloads on VMware Hybrid Platform

Deploying and managing an Oracle DB environment is not a trivial undertaking. Oracle infrastructure tends to have stringent performance, business continuity, and backup and recovery requirements. To support these requirements, IT departments often are tasked with maintaining sprawling infrastructure for production, disaster recovery, backup, quality assurance, test, training, development, and sandbox.

Enterprise customers have identified Database-as-a-Service (DBaaS) as a key initiative that will enable higher levels of agility via rapid application development. The idea is to allow database consumers such as application developers, testers, and architects to provision databases easily using an on-demand, self-service platform.

DBaaS will greatly simplify the deployment and management of a robust Oracle environment, while delivering higher levels of efficiency and flexibility for IT departments throughout the application lifecycle—implementation, migrations, consolidations, upgrades, and ongoing maintenance , thus allowing DBA’s to focus on more important pressing issues – Moving away from DBA-as-a-Service to Database as a Service (DBaaS) model.

This blog focuses on VMware Virtual Volumes (vVols) using Pure Storage and how vVols can greatly enhance DBaaS to provide higher levels of agility via rapid application development for both on-premises and the cloud for Oracle workloads on VMware Hybrid Platform.

What is Database-as-a-Service (DBaaS)

DBaaS can be defined as the ability to manage, monitor, and provision database resources on demand through self-service catalogs with minimal requirements for installation and configuration of hardware or software up front.

Core Benefits of DBaaS solution

A good DBaaS platform needs to provide developers, DevOps engineers, and database administrators (DBAs) an ability to automate daily tasks, optimize their infrastructure, and refocus their efforts on new innovations for their applications with speed and agility. In addition, the platform should support different types of databases and their capabilities.

1. Reduce Application Time to Market: DBaaS gives you the ability to rapidly develop, test, and deploy new applications at a fraction of the time it used to take with legacy systems, so faster time to market new apps
2. Reduce costs by optimizing infrastructure use: DBaaS increases the efficiency of both DB/application and infrastructure teams. The storage administrator will no longer need to spend time and effort on provisioning storage resources for databases. DBAs will no longer have to wait for DB resources to be allocated and provisioned. This frees up time for both the teams
3. Automate Database Lifecycle Management: Database sprawl is a major problem in many companies. IT departments have difficulty keeping track of all the DB instances that are deployed. This has serious implications from licensing, data governance, and security standpoints. With DBaaS you can ensure that you have complete visibility into your database environment and enforce governance through policy-based management. Save thousands of hours through automation of database monitoring, management and maintenance operations so you can focus on product.
4. Availability: Improve reliability of applications by making them highly available with failover protection and minimal downtime.
5. Scalability: Scale to any size on demand by dynamically scaling databases as and when needed without underutilization or overpaying for a larger scale.

Compelling Use cases for DBaaS

One can define DBaaS as the ability to manage, monitor, and provision database resources on demand through self-service catalogs with minimal requirements for installation and configuration of hardware or software up front. The idea is to allow database consumers such as application developers, testers, and architects and owners to provision databases easily and quickly using an on-demand, self-service platform.

Some of the use cases of Database as a Service

• Defining templates of database VMs to be available in self-service catalog
• Commissioning / Decommissioning Single Instance / Clustered databases from self-service catalog
• Cloning of Single Instance / Clustered Databases with data masking for Test / Dev / QA
• Database Refresh
• Database Backup & Restore
• Metering / Reporting

Critical factors for DBaaS

Time is critical for an as a service solution and it is also critical for DBaaS. Quite often the storage performance can be a limiting factor for DBaaS solutions. Traditional disk-based storage cannot meet the performance needs for DBaaS and is inherently too complicated to manage.

Some of the key DBaaS considerations include

• High-performing storage infrastructure to ensure rapid provisioning of DBs
• Efficient data reduction to ensure storage capacity is not exhausted
• A high degree of resiliency to ensure minimal-to-no downtime
• Ease of use and management
• Different Databases have different levels of criticality and each DB needs different storage performance characteristics and capabilities
• There is a challenge trying to meet stringent SLAs for performance with slow traditional storage
• With the rapid growth of databases, the need to reduce the backup windows to meet performance & business SLAs grows as well
• With the growth databases, day 2 operations like Clone, Database Refresh  from production to QA becomes a big challenge
• Using Storage-based replication, unnecessary data gets copied over even if the speed of storage-based replication is superior compared to Application based replication

VMware vVols ideal technology to meet DBaaS requirements

VMware Virtual Volumes or vVols enables your existing storage array capabilities to be managed via storage policy-based management (SPBM) and applied at a VM or VM disk level, all while existing in a single vVols datastore that correlates to a storage container on the array. The Guest OS file system is the native FS on the vVol itself. It is not a VMDK sitting on top of another FS such as VMFS or NFS. Each vVol is its own first-class citizen, meaning it is independent of other vVols, LUNs, or volumes. As a result, vVols match very well with First Class Disks (FCD), which are disks not requiring an associated VM. vVols also allows for a much larger scale than traditional LUNs, up to 64k per host! Plus, you don’t have to manage LUNs or volumes, which would be a nightmare at scale.

Array is unaware of use of blocks.  The array and vSphere unaware of each other leading to rigidity to associate unique capability at a LUN or volume level. LUNs or volumes are rigid and fixed in their capabilities. They also require a file system such as VMFS or NFS.

Traditional Storage arrays are a black box in a vSphere environment

With VVols, there is no file system, it’s inside each VVol that a file system is created and that file depends on the OS or the VVol function. IE config, swap, etc. With regards to the capabilities, generally a LUN or volume is provisioned with a specific set of capabilities and that isn’t easily changed without create a new LUN or volume and migrating the data.

Each VM has its associated vVols at the array level

With VVols, changing the capability is simple, with the simple change of an SPBM policy, the capability is changed, no storage vMotion or data migration needed. Depending on the array, you can even change from spinning media to all-flash with a policy change. The array then handles the performance and data migration, no admin action required. Another key aspect of VVols over traditional storage is the array and vSphere has complete insight into the data and I/O requirements. Subsequently, the array can then manage the I/O and data accordingly.

Snapshots and cloning are critical for DBaaS

With traditional storage, snapshots are managed by vSphere. Although vSphere is efficient at snapshots, typically, arrays manage snapshots in a different and more efficient manner. With typical vSphere snapshots, a secondary SESparse disk is created, and all new I/O goes to that disks. If this disk is not removed it can continue to grow even larger than the original disk.  Additionally, if more snapshots are created, then the chain of VM can slow I/O as any read may end up growing though each disk looking for the data. Consequently, performance of the VM can be impacted. When the vSphere snapshot is deleted, depending on how large the snapshot is and how busy the VM is, it can take quite a bit of time to delete. This is the norm, but if a snapshot is left for a long time and grows to 10s or even 100s of GB, it could take hours to delete and impact the VM’s performance while being deleted.

Traditional snapshots with VMFS versus storage level snapshots with vVols

With vVols, snapshots happen on the array and array based snapshots are point in time “pictures” of the data and take very little space. When a vVols snapshot exists, the I/O is NOT redirected and instead continues to go to the original disk or vVol. Leaving a vVols snapshot alone will not continue to grow and can be copied to another array and even used as a copy or clone of the original VM. When a vVol snapshot is deleted, the snapshot is quickly removed with no impact to the VM. Regardless of how many snapshots, up to a maximum of 32, there is no performance impact on the VM. Creation and deletion of snapshots is faster and more efficient on vVol storage due to array offloading of operations

The video VMware vVols with Pure Storage a game changer for DBaaS (Presentation by Sudhir Balasubramanian and Mohan Potheri, VMware)

• explains how one can achieve Scalability , Reliability and Performance with DBasS using VMware Virtual Volumes on Pure Storage for Oracle Workloads.
• highlights primary use cases for VMware Virtual Volumes (VVols) and Storage Policy Based Management (SPBM) by leveraging Pure Storage VVols technology including database backup and restore, refreshing, cloning, patching etc
• shows how DBAs can be empowered with the tools they need to secure, manage, and scale your deployment on-premises

All Oracle on vSphere white papers including Oracle licensing on vSphere/vSAN, Oracle best practices, RAC deployment guides, workload characterization guide can be found in the url below

Oracle on VMware Collateral – One Stop Shop

Hot Remove CPU and Memory for Oracle production workloads

The previous blog On Demand Scaling up resources for Oracle production workloads – Hot Add CPU and Hot Add Memory focused on how to stop hoarding much needed infrastructure resources and live wisely ever after by scaling up as needed effectively.

This blog focuses on the Hot Remove CPU and Memory aspect of Oracle Workloads and the current limitations associated with these technologies.

Current caveats for CPU “Hot-Remove” support

• Currently vSphere does not have the CPU Hot Remove capability on the Web Client.
• Red Hat Enterprise Linux version 7.1 (RHEL7.1) and later supports “Hot-Remove” of physical CPU and Memory as long as the underlying hardware supports this function
• The CPU Hot Remove capability is an In-Guest capability starting RHEL 7.1 and onwards – RedHat KB CPU/Memory “Hot-Add” and “Hot-Remove” Support in Red Hat Enterprise Linux version 7.
• The CPU Hot Remove capability In-Guest capability does not result in a reduction of the vCPU’s allocated to the VM. This capability is just an OS Hot Online/Offline operation

Current caveats for Memory “Hot-Remove” support

• Currently vSphere does not have the Memory Hot Remove capability on the Web Client.
• The Memory Hot Remove Capability In-Guest capability is currently not supported on VMware – RedHat KB How to hot-add and hot-remove memory in RHEL7 ?  in Red Hat Enterprise Linux version 7

Test Setup

A VM ‘SB-OL76-ORA19C’ has 8 vCPUs with 32  GB RAM running OEL 7.6 with Oracle Database 19c.

[root@sb_ol76_ora19c ~]# cat /etc/oracle-release
Oracle Linux Server release 7.6
[root@sb_ol76_ora19c ~]#

[root@sb_ol76_ora19c ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.6 (Maipo)
[root@sb_ol76_ora19c ~]#

[root@sb_ol76_ora19c ~]# uname -a
Linux sb_ol76_ora19c.corp.localdomain 4.1.12-112.14.15.el7uek.x86_64 #2 SMP Thu Feb 8 09:58:19 PST 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@sb_ol76_ora19c ~]#

Oracle instance ‘ora19c’ sga_max_size is set to 16 GB and cpu_count shows 8

oracle@sb_ol76_ora19c:ora19c:/home/oracle> sqlplus / as sysdba
SQL*Plus: Release 19.0.0.0.0 – Production on Wed Sep 15 12:28:07 2021
Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 – Production
Version 19.3.0.0.0

SQL> select name, value from v$parameter where name = ‘sga_max_size’;
NAME VALUE
————————- —————-
sga_max_size 17179869184
SQL>

SQL> select name, value from v$parameter where name = ‘cpu_count’;
NAME VALUE
————————- —————-
cpu_count 8
SQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 – Production
Version 19.3.0.0.0
oracle@sb_ol76_ora19c:ora19c:/home/oracle>

Test Case – “Hot Remove” CPU

Red Hat Enterprise Linux version 7.1 (RHEL7.1) and later supports “Hot-Remove” of physical CPU and Memory as long as the underlying hardware supports this function. Learn more at CPU/Memory “Hot-Add” and “Hot-Remove” Support in Red Hat Enterprise Linux version 7

Currently vSphere does not have the CPU Hot Remove capability on the Web Client. The CPU Hot Remove capability is an In-Guest capability starting RHEL 7 onwards.

The VM ‘SB-OL76-ORA19C’ has 8 vCPUs.

Below commands lists all present vCPUs/CPUs of the system:

[root@sb_ol76_ora19c ~]# ls -l /sys/devices/system/cpu | grep cpu
drwxr-xr-x 6 root root 0 Sep 12 11:13 cpu0
drwxr-xr-x 6 root root 0 Sep 12 11:13 cpu1
drwxr-xr-x 6 root root 0 Sep 12 11:13 cpu2
drwxr-xr-x 6 root root 0 Sep 12 11:13 cpu3
drwxr-xr-x 6 root root 0 Sep 12 11:13 cpu4
drwxr-xr-x 6 root root 0 Sep 12 11:13 cpu5
drwxr-xr-x 6 root root 0 Sep 12 11:13 cpu6
drwxr-xr-x 6 root root 0 Sep 12 11:13 cpu7
drwxr-xr-x 2 root root 0 Sep 12 19:53 cpuidle
[root@sb_ol76_ora19c ~]#

Below command lists all working vCPUs/CPUs of the system:

[root@sb_ol76_ora19c ~]# cat /proc/cpuinfo |grep processor
processor : 0
processor : 1
processor : 2
processor : 3
processor : 4
processor : 5
processor : 6
processor : 7
[root@sb_ol76_ora19c ~]#

Below command will Hot-remove CPU0 from the system:

[root@sb_ol76_ora19c ~]# echo 0 > /sys/devices/system/cpu/cpu0/online

The OS /var/log/messages file shows below message:

Sep 12 20:04:39 sb_ol76_ora19c kernel: smpboot: CPU 0 is now offline

Check the online/offline vCPUs via the commands below:

[root@sb_ol76_ora19c ~]# cat /sys/devices/system/cpu/online
1-7
[root@sb_ol76_ora19c ~]#

[root@sb_ol76_ora19c ~]# cat /sys/devices/system/cpu/offline
0
[root@sb_ol76_ora19c ~]#

The Oracle database alert.log file has the following messages which reflects the Hot Removal of CPU0:
…
2021-09-12T19:55:55.961847-07:00
Detected change in CPU count to 7
* Load Monitor used for high load check
* New Low – High Load Threshold Range = [0 – 0]
PDB$SEED(2):Adjusting the altered value of parameter parallel_max_servers
PDB$SEED(2):from 140 to 100 due to the ROOT Container Value Setting
PDB1(3):Adjusting the altered value of parameter parallel_max_servers
PDB1(3):from 140 to 100 due to the ROOT Container Value Setting

Below SQL statement will reflect the reduction in the number of CPU’s to 7:

SQL> select name, value from v$parameter where name = ‘cpu_count’;
NAME
——————————————————————————–
VALUE
——————————————————————————–
cpu_count
7

Below command will online CPU0 back to the system:

[root@sb_ol76_ora19c ~]# echo 1 > /sys/devices/system/cpu/cpu0/online

The OS /var/log/messages file shows below message:
…
Sep 12 20:07:56 sb_ol76_ora19c kernel: smpboot: Booting Node 0 Processor 0 APIC 0x0

Check the online vCPUs via the commands below:

[root@sb_ol76_ora19c ~]# cat /proc/cpuinfo |grep processor
processor : 0
processor : 1
processor : 2
processor : 3
processor : 4
processor : 5
processor : 6
processor : 7
[root@sb_ol76_ora19c ~]#

[root@sb_ol76_ora19c ~]# cat /sys/devices/system/cpu/online
0-7
[root@sb_ol76_ora19c ~]#

The Oracle database alert.log file has the following messages which reflects the Hot Add of CPU0:
….
2021-09-12T19:56:55.995881-07:00
Detected change in CPU count to 8
* Load Monitor used for high load check
* New Low – High Load Threshold Range = [0 – 0]
PDB$SEED(2):Adjusting the altered value of parameter parallel_max_servers
PDB$SEED(2):from 160 to 100 due to the ROOT Container Value Setting
PDB1(3):Adjusting the altered value of parameter parallel_max_servers
PDB1(3):from 160 to 100 due to the ROOT Container Value Setting
2021-09-12T20:04:56.220823-07:00

Below SQL statement will reflect the increase in the number of CPU’s back to 8:

SQL> select name, value from v$parameter where name = ‘cpu_count’;

NAME
——————————————————————————–
VALUE
——————————————————————————–
cpu_count
8

We were able to successfully Hot Offline / Online CPU’s on RHEL 7.6 OS . Currently vSphere does not have the CPU Hot Remove capability on the Web Client. The CPU Hot Remove capability is an In-Guest capability starting RHEL 7 onwards.

The CPU Hot Remove capability In-Guest capability does not result in a reduction of the vCPU’s allocated to the VM. This capability is just an OS Hot Online/Offline operation

Test Case – “Hot Remove” Memory

RHEL 7.0 onwards supports “Hot-Remove” of physical Memory as long as the underlying hardware supports this function.

As per RedHat KB How to hot-add and hot-remove memory in RHEL7? : – The hardware or virtualization provider must support hot plug functionality. Historically VMware has supported hot-add memory, but not hot-remove memory. Verify functionality with your hardware or virtualization provider before proceeding.

Currently vSphere does not have the Memory Hot Remove capability on the Web Client. The Memory Hot Remove Capability In-Guest capability is currently not supported on VMware.

The VM ‘SB-OL76-ORA19C’ has 32 GB vRAM.

Check current memory status via command. We can remove the memory for which the /sys/devices/system/memory/memoryX/removable is 1

[root@sb_ol76_ora19c memory1]# cat /sys/devices/system/memory/memory1/state
online
[root@sb_ol76_ora19c memory1]# cat /sys/devices/system/memory/memory1/removable
1
[root@sb_ol76_ora19c memory1]#

The offline operation is not successful, and the Memory is still online.
[root@sb_ol76_ora19c memory1]# echo offline > /sys/devices/system/memory/memory1/state
[root@sb_ol76_ora19c memory1]# cat /sys/devices/system/memory/memory1/state
online
[root@sb_ol76_ora19c memory1]# cat /sys/devices/system/memory/memory1/removable
1
[root@sb_ol76_ora19c memory1]#

The OS /var/log/messages has these messages:
Sep 12 21:13:22 sb_ol76_ora19c kernel: Offlined Pages 32768
Sep 12 21:13:22 sb_ol76_ora19c systemd: Stopping Crash recovery kernel arming…
Sep 12 21:13:22 sb_ol76_ora19c kdumpctl: kexec: unloaded kdump kernel
Sep 12 21:13:22 sb_ol76_ora19c kdumpctl: Stopping kdump: [OK]
Sep 12 21:13:22 sb_ol76_ora19c systemd: Stopped Crash recovery kernel arming.
Sep 12 21:13:22 sb_ol76_ora19c systemd: Starting Crash recovery kernel arming…
Sep 12 21:13:23 sb_ol76_ora19c kdumpctl: kexec: loaded kdump kernel
Sep 12 21:13:23 sb_ol76_ora19c kdumpctl: Starting kdump: [OK]
Sep 12 21:13:23 sb_ol76_ora19c systemd: Started Crash recovery kernel arming.
Sep 12 21:13:23 sb_ol76_ora19c systemd: Stopping Crash recovery kernel arming…
Sep 12 21:13:23 sb_ol76_ora19c kdumpctl: kexec: unloaded kdump kernel
Sep 12 21:13:23 sb_ol76_ora19c kdumpctl: Stopping kdump: [OK]
Sep 12 21:13:23 sb_ol76_ora19c systemd: Stopped Crash recovery kernel arming.
Sep 12 21:13:23 sb_ol76_ora19c systemd: Starting Crash recovery kernel arming…
Sep 12 21:13:25 sb_ol76_ora19c kdumpctl: kexec: loaded kdump kernel
Sep 12 21:13:25 sb_ol76_ora19c kdumpctl: Starting kdump: [OK]
Sep 12 21:13:25 sb_ol76_ora19c systemd: Started Crash recovery kernel arming.

The Oracle database is still online and not affected by above operation.

We were not able to Hot Remove / Offline memory on RHEL 7.6 OS on VMware.

Currently vSphere does not have the Memory Hot Remove capability on the Web Client. The Memory Hot Remove Capability In-Guest capability is currently not supported on VMware

Summary

Current caveats for CPU “Hot-Remove” support

• Currently vSphere does not have the CPU Hot Remove capability on the Web Client.
• Red Hat Enterprise Linux version 7.1 (RHEL7.1) and later supports “Hot-Remove” of physical CPU and Memory as long as the underlying hardware supports this function
• The CPU Hot Remove capability is an In-Guest capability starting RHEL 7.1 and onwards – RedHat KB CPU/Memory “Hot-Add” and “Hot-Remove” Support in Red Hat Enterprise Linux version 7.
• The CPU Hot Remove capability In-Guest capability does not result in a reduction of the vCPU’s allocated to the VM. This capability is just an OS Hot Online/Offline operation

Current caveats for CPU “Hot-Remove” support

• Currently vSphere does not have the Memory Hot Remove capability on the Web Client.
• The Memory Hot Remove Capability In-Guest capability is currently not supported on VMware – RedHat KB How to hot-add and hot-remove memory in RHEL7 ?  in Red Hat Enterprise Linux version 7

All Oracle on vSphere white papers including Oracle licensing on vSphere/vSAN, Oracle best practices, RAC deployment guides, workload characterization guide can be found in the url below

Oracle on VMware Collateral – One Stop Shop
https://blogs.vmware.com/apps/2017/01/oracle-vmware-collateral-one-stop-shop.html

Deploying Oracle Workloads on vSAN HCI Mesh Compute Cluster – Disaggregating Compute and Storage

The previous blog No downtime Storage vMotion of Oracle RAC Cluster using shared vmdk’s with multi-writer attribute from one vSAN to another vSAN Cluster using VMware HCI Mesh demonstrated how we can achieve Storage vMotion of Oracle RAC Cluster using shared vmdk’s with multi-writer attribute ONLINE , with NO downtime, from one vSAN to another vSAN Cluster using VMware HCI Mesh.

The previous blog also touched on briefly a very important use case for Oracle workloads on any architecture, Oracle Licensing.

With vSAN HCI Mesh introduced in vSAN 7 Update 1 capability, one could seamlessly and transparently

• move Oracle compute into smaller density (Physical sockets / cores) servers in a different vSAN cluster, thereby significantly reducing their Oracle Licensing costs
• move the Storage for Oracle workloads into a different vSAN cluster, if needed, where excess storage capacity is available

With vSAN 7 Update 2, VMware introduces a significant new capability to vSAN HCI Mesh where traditional vSphere clusters can now mount a remote vSAN datastore.

Yes, we have a winner

This blog demonstrates how we can provision separate Compute and Storage resources for Oracle workloads on vSAN 7 Update 2 Cluster, disaggregating Compute and Storage, with Compute on traditional vSphere Clusters / vSAN Clusters and Storage on vSAN Storage , with a focus on reducing Oracle licensing using HCI Mesh Compute Clusters.

VMware vSAN HCI Mesh Compute Clusters

VMware introduces a significant new capability to vSAN HCI Mesh. In vSAN 7 Update 2, traditional vSphere clusters can mount a remote vSAN datastore. HCI Mesh compute clusters can consume storage resources provided by a remote vSAN cluster, in the same way that multiple vSphere clusters can connect to a traditional storage array.

HCI Mesh compute clusters use native vSAN protocols for maximum efficiency and affords the customer the ability to easily meet a broad variety of use cases. Most importantly, HCI Mesh compute clusters do not need any vSAN licensing

More information on HCI Mesh Compute Clusters can be found in the VMware vSAN HCI Mesh Tech Note.

More information on vSAN HCI Mesh Considerations can be found in the blog vSAN HCI Mesh Considerations.

HCI Mesh Compute Cluster Setup

We have 2 vSphere Clusters attached to the same Virtual Center.

• Source vSAN Cluster – A vSAN Cluster ‘BCA-SiteC’ VMware ESXi, 7.0.2, 17867351, with vSAN datastore ‘BCA-SiteC-vSAN’
• Target traditional vSphere Cluster – vSphere Cluster ‘GPU1’ VMware ESXi, 7.0.2, 17867351

Source vSAN Cluster – ‘BCA-SiteC’ – the vSAN Cluster has 4 hosts , each host has 2 sockets, 24 cores per socket with 384GB RAM.

Source vSAN datastore ‘BCA-SiteC-vSAN’

Target Traditional vSphere Cluster – ‘GPU1’ – the vSphere Cluster has 4 hosts , each host has 2 sockets, 10 cores per socket with 1TB RAM.

The vSAN HCI Mesh Considerations found in the blog vSAN HCI Mesh Considerations. lists out the various requirements that’s is needed to mount a vSAN datastore on a HCI Mesh Compute Cluster

The steps to setup a vSAN HCI Mesh Compute Cluster is shown in the blog post What’s New with vSAN 7 Update 2.

The screenshot below summarizes the setup of vSAN HCI Mesh Compute Cluster as well.

The next step is to mount the remote vSAN datastore ‘BCA-SiteC-vSAN’ on the vSphere Cluster ‘GPU1’ which is now setup as a vSAN HCI Mesh Compute Cluster.

The screenshot below summarizes the steps needed to mount the remote vSAN datastore ‘BCA-SiteC-vSAN’ on the vSphere Cluster ‘GPU1’ which is now setup as a vSAN HCI Mesh Compute Cluster.

The remote vSAN datastore ‘BCA-SiteC-vSAN’ is mounted on the target vSphere cluster ‘GPU1’ which is setup as a vSAN HCI Mesh Compute Cluster.

Test Case

The purpose of the blog is to demonstrate how we can provision separate Compute and Storage resources for Oracle workloads on vSAN 7 Update 2 Cluster, disaggregating Compute and Storage, with Compute on traditional vSphere Clusters / vSAN Clusters and Storage on vSAN Storage, with a focus on reducing Oracle licensing using HCI Mesh Compute Clusters.

The HCI Mesh Compute Cluster Setup demonstrated how we can disaggregating Compute and Storage, with Compute on traditional vSphere Clusters in this case (also vSAN Clusters)  and Storage on vSAN Storage.

We can now vMotion the Oracle workloads compute from source vSAN Cluster ‘BCA-SiteC’ to target traditional vSphere Cluster ‘GPU1’ , leaving the Oracle workloads storage still on source vSAN datastore ‘BCA-SiteC-vSAN’.

We will consider 2 use cases here on HCI Mesh Compute Cluster

• Deploying / Migrating Single Instance Oracle VM
• Deploying / Migrating Oracle RAC VM’s

Deploying / Migrating Single Instance Oracle VM using HCI Mesh Compute Cluster

A VM ‘Oracle19c-BM’ currently in source vSAN Cluster ‘BCA-SiteC’ with storage on vSAN datastore ‘BCA-SiteC-vSAN’ is chosen.

Currently both Compute and Storage is on the vSAN Cluster ‘BCA-SiteC’.

Compute vMotion the VM ‘Oracle19c-BM’ from source vSAN Cluster ‘BCA-SiteC’ to target traditional vSphere Cluster ‘GPU1’  – the VM ‘Oracle19c-BM’ Compute will now relocate to target traditional vSphere Cluster ‘GPU1’ with storage still on source vSAN datastore ‘BCA-SiteC-vSAN’.

After Compute vMotion, the VM ‘Oracle19c-BM’ , VM Compute is now on target traditional vSphere Cluster ‘GPU1’ with Storage still on source vSAN datastore ‘BCA-SiteC-vSAN’

We can perform similar vMotion operations to move the remaining Single Instance Oracle workloads off from the vSAN cluster to the traditional vSphere Cluster ‘GPU1’. This results in Oracle workloads using far less compute.

Deploying / Migrating Oracle RAC VM’s using HCI Mesh Compute Cluster

Oracle RAC VM’s ‘rac19c1’ and ‘rac19c1’ are currently in source vSAN Cluster ‘BCA-SiteC’ with storage on vSAN datastore ‘BCA-SiteC-vSAN’.

Compute vMotion both RAC VM’s ‘rac19c1’ and ‘rac19c2’ from source vSAN Cluster ‘BCA-SiteC’ to target traditional vSphere Cluster ‘GPU1’.

RAC VM’s ‘rac19c1’ and ‘rac19c2’ Compute are now relocated to target traditional vSphere Cluster ‘GPU1’ with storage still on source vSAN datastore ‘BCA-SiteC-vSAN’.

Oracle Licensing Perspective

Assuming that an entire host is used for Oracle workloads –

• Source vSAN Cluster ‘BCA-SiteC’ has 4 hosts, each host has 2 sockets, 24 cores per socket with 384GB RAM
• Each vSAN host has 48 absolute cores which is equivalent to 24 Effective Oracle Enterprise Edition licenses
• Target traditional vSphere Cluster ‘GPU1’ has 4 hosts, each host 2 sockets, 10 cores per socket with 1TB RAM
• Each target vSphere host has 20 absolute cores which is equivalent to 10 Effective Oracle Enterprise Edition licenses
• Moving the Oracle VM Compute off the vSAN Cluster to the traditional vSphere Cluster results in a drastic reduction of 28 absolute cores per host which is equivalent to a drastic reduction of 14 Effective Oracle Enterprise Edition licenses (reduction in $$$$$)

Remember Oracle licensing is for Current usage of Compute (Physical Sockets (SE2)  / Physical Cores (EE) ) or NUP , not proposed futuristic usage !!! Oracle licensing DOES NOT change , whether you run Oracle workloads on a classic vSphere environment or Hyper-Converged Infrastructure or any of the VMware Hybrid Clouds.

Refer to the blog post for more information Oracle on VMware vSphere , vSAN, VxRAIL and VMware Cloud on AWS – Dispelling the Licensing myths

Summary

With vSAN HCI Mesh introduced in vSAN 7 Update 1 capability, one could seamlessly and transparently

• move Oracle compute into smaller density (Physical sockets / cores) servers in a different vSAN cluster, thereby significantly reducing their Oracle Licensing costs
• move the Storage for Oracle workloads into a different vSAN cluster, if needed, where excess storage capacity is available

With vSAN 7 Update 2, VMware introduces a significant new capability to vSAN HCI Mesh where traditional vSphere clusters can now mount a remote vSAN datastore.

All Oracle on vSphere white papers including Oracle on VMware vSphere / VMware vSAN / VMware Cloud on AWS , Best practices, Deployment guides, Workload characterization guide can be found in the url below

Oracle on VMware Collateral – One Stop Shop

http://www.vmw.re/oracle-one-stop-shop

On Demand hot extend clustered vmdk’s online without downtime – Hot Extend RAC clustered disks

 

 

 

This blog focuses on the complete list of steps to be taken to increase the size of the clustered vmdk(s) using Multi-writer flag of an Oracle RAC Cluster. This blog describes the various options using both Native Oracle tools and vSphere tools to attempt to increase the size of the Oracle RAC clustered vmdk(s).

Current restriction of shared vmdk’s using the multi-writer attribute is that Hot extend virtual disk is disallowed as per KB 1034165 & KB 2121181

 

 

 

Reclaiming dead space from Oracle databases on VMware Hybrid Platform

 

 

Dead space reclamation has always been a challenge especially with thin-provisioned volumes. Thin provisioning dynamically allocated storage on-demand to the workloads in a flexible and efficient manner.

Any database day to day 2 operations would include dropping tablespaces, dropping or resizing datafiles, truncating tables, deleting archive logs from ASM FRA diskgroup etc. .

One would expect the free space to go back to the storage free pool to be used for further allocation purposes, but that ability depends on

• How the Application reports the discarded space to the GOS (Guest Operating System)
• How the GOS reports the discarded space to underlying VMware layer
• And eventually how VMware vSphere reports the discarded space to the Storage array

 

This blog focuses on the Oracle ASM Filter Driver (ASMFD) capability to report discarded or free space of thin-provisioned vmdk’s to the GOS which eventually gets reported to the VMware datastores and the underlying Storage array.

 

 

Oracle Java on VMware Hybrid Cloud Platform – Dispelling the Oracle Java Licensing myths

This is just another blog (JAB) about Dispelling the “Galaxy Licensing” Oracle Java on VMware FUD  perpetuated by overzealous licensing and sales teams, which is in complete contrast to the reality of the actual Contractually Impactful documents.

  This blog intends to clear up the reality of how to effectively license Oracle Java workloads on VMware Hybrid Cloud ( vSphere, vSAN , VxRAIL , VMware Cloud on AWS and other VMware Cloud Platforms) and to make it a cost effective one as well.

 

 

Java Licensing Models

 

Prior to 2019, there were three Java SE products known as

• Java SE Advanced
• Java SE Advanced Desktop
• Java SE Suite

  These models required commercial users to purchase upfront licenses and pay annual support.

  Starting 2019, the above three models was replaced by two new subscription-based models

• Java SE Subscription
• Java SE Desktop Subscription

  As per this model, one no longer needs to purchase licenses upfront and pay annual support for Java SE.  Instead, one would have to pay a monthly subscription fee under one to three-year terms for server or desktop licensing and support.

 

 

No downtime Storage vMotion of Oracle RAC Cluster using shared vmdk’s with multi-writer attribute from one vSAN to another vSAN Cluster using VMware HCI Mesh

The previous blog post “Around the “Storage World” in no time – Storage vMotion for Oracle Workloads (RAC & Non-RAC) within same vSphere Cluster” addressed the following use cases of Storage vMotion of an Oracle RAC Cluster with shared vmdk(s) using multi-writer attribute .

 

The following uses cases were addressed by the previous blog post for Oracle RAC clusters within the same vSphere Cluster –

• Migrate Oracle database storage from one Tier to another Tier within a storage array
• Migrate Oracle database storage from one array to another array (within and between data centers) for the same datastore type [ VMFS . NFS, iSCSI, vVOL, vSAN]
• Migrate Oracle database storage from one array to another array (within and between data centers) across different datastore types [ VMFS , NFS, iSCSI, vVOL, vSAN]

 

 

Backing up Oracle Workloads (RAC & Non-RAC) with VMware Snapshot Technology

In computer systems, a snapshot is the state of a system at a particular point in time. To avoid downtime, high-availability systems may instead perform the backup on a snapshot—a read-only copy of the data set frozen at a point in time—and allow applications to continue writing to their data. [ Wikipedia ]

This blog addresses how one can take VM level snapshot of Oracle Single Instance / RAC workloads using VMware snapshot technology, keep certain caveats in mind.

 

Oracle Real Application Cluster (RAC) Support on VMware Clouds

 

As of December 9th, 2020, VMware recommends that anyone currently running OR planning to run, an Oracle Real Application Cluster (RAC) in a VMware Cloud environment, reach out to Oracle Corporation for any Oracle support related matters with RAC on our cloud platform.

 

Please reach out to your VMware representative for any clarifications on the below guidance’s.