Providing Storage savings and Comparable performance using VMware Cloud Flex Storage for Oracle Non Production workloads

Posted on June 2, 2023 by Sudhir Balasubramanian

Non-production workloads e.g. Development, Test, QA, Staging, Functional, Pre-Production etc are equally important as the Production workloads , as they are key entities and instrumental to any Production workload success.

Ensuring that Pre-Production databases are comparable in performance with the Production databases is key in ensuring that any performance related issues are identified well ahead in the process before they make it to the Production environment. In addition, ensuring that there is adequate testing and validation ensures due diligence is done with any software release.

Key points to take away from this blog

This blog is the third blog in the series of evaluating VMware Cloud Flex Storage as a viable & comparable Storage option for Oracle non-production workloads.

The first blog Providing Performance and Storage savings using VMware Cloud Flex Storage for Oracle Archivelogs  was an exercise to evaluate the feasibility and showcase the advantages of using VMware Cloud Flex Storage for business-critical Oracle workloads on VMware Cloud on AWS as a viable & comparable Storage option for Oracle Archivelog files destination.

The second blog Providing Storage savings and Comparable performance using VMware Cloud Flex Storage for Oracle Backups was an exercise to evaluate VMware Cloud Flex Storage as a viable & comparable Storage option for Oracle backups destination

This blog is not meant to be a performance benchmarking-oriented blog in any way.  Remember, any performance data is a result of the combination of hardware configuration, software configuration, test methodology, test tool, and workload profile used in the testing, so the performance improvement I got with my workload in my lab is in no way representative of any real production workload which means the performance improvements for real world workloads will be better.

VMware Cloud Flex Storage

VMware Cloud Flex Storage is a Storage-as-a-Service offering for VMware Cloud on AWS that provides the ability to provision and scale storage independently of their SDDC hosts.

VMware Cloud Flex Storage is fully VMware-managed and natively integrated, so you can cost-effectively supplement your capacity needs rather than purchasing additional SDDC hosts.

VMware Cloud Flex Storage provides disaggregated cloud storage for non-mission critical applications, such as file servers, analytics, and lower-tier databases. Disaggregated storage provides the performance of local storage with the flexibility of storage area networks.

More information on VMware Cloud Flex Storage can be found here and here.

VMware vSAN

VMware vSAN is a software-defined, enterprise storage solution that supports hyper-converged infrastructure (HCI) systems. vSAN is fully integrated with VMware vSphere, as a distributed layer of software within the ESXi hypervisor.

vSAN aggregates local or direct-attached data storage devices to create a single storage pool shared across all hosts in a vSAN cluster. A hybrid vSAN cluster uses flash devices for cache and magnetic drives for capacity. All-flash vSAN clusters use flash devices for both the cache and capacity. vSAN Express Storage Architecture uses NVMe-based TLC flash devices and high performance network interfaces. These architectures provide a flash-optimized, resilient shared datastore designed for the software-defined data center (SDDC).

vSAN eliminates the need for external shared storage, and simplifies storage configuration through Storage Policy-Based Management (SPBM). Using virtual machine (VM) storage policies, you can define storage requirements and capabilities.

More information on VMware vSAN can be found here.

Test Bed

The Test bed is a 4 Node VMware Cloud on AWS cluster with the setup shown as below –

  • vCenter version was 8.0.0 build 20432146
  • 4 Amazon EC2 i3.metal ESXi servers
  • VMware ESXi, 8.0.0, 20430035
  • VMware Cloud Flex Storage (NFS v3)
  • Oracle 21.5 with Grid Infrastructure, ASM Storage and ASMLIB
  • OEL UEK 8.7

VMware Cloud Flex Storage datastore details are shown as below.

‘vSAN Default Storage Policy’ details are shown as below.

Oracle VM ‘Oracle21C-OL8-Customer-DB-FlexStorage’ details are shown as below.

The VM has 18 vCPU’s, 128GB RAM, the single instance database ‘ORA21C’ was created with created with multi-tenant option & provisioned with Oracle Grid Infrastructure (ASM) and Database version 21.5 on O/S OEL UEK 8.7.

Oracle ASM was the storage platform with Oracle ASMLIB for device persistence. Oracle SGA & PGA set to 64G and 10G respectively.

All Oracle on VMware platform best practices were followed.

The vmdk’s for the Oracle VM ‘Oracle21C-OL8-Customer’ are shown as below –

  • Hard Disk 1 (SCSI 0:0) – 80G for OS (/)
  • Hard Disk 2 (SCSI 0:1) – 80G for Oracle Grid Infrastructure & RDBMS binaries
  • Hard Disk 3 (SCSI 1:0) – 100G – ASM Disk Group GRID_DG (Grid Infrastructure)
  • Hard Disk 4 (SCSI 1:1) – 200G – ASM Disk Group DATA_DG (Oracle database)
  • Hard Disk 5 (SCSI 3:0) – 100G – ASM Disk Group REDO_DG (Redo Log Files)
  • Hard Disk 6 (SCSI 2:0) – 250G – ASM Disk Group SLOB_DG on vSAN (SLOB load generator) using default VSAN storage policy – ‘vSAN Default Storage Policy’
  • Hard Disk 7 (SCSI 1:3) – 130G – ASM Disk Group FRA_FLEX_DG for Oracle Archivelog files using VMware Cloud Flex Storage
  • Hard Disk 8 (SCSI 2:1) – 260G – ASM Disk Group SLOB_FLEX_DG on VMware Cloud Flex Storage (SLOB load generator)

The Oracle Archivelog files were placed on VMware Cloud Flex Storage (ASM Disk Group FRA_FLEX_DG).

Oracle VM ‘Oracle21C-OL8-Customer’ vmdk’s provisioning and storage policy details

  • vmdk (Hard Disk 7) for ASM Disk group FRA_FLEX_DG & vmdk (Hard Disk 8) for ASM Disk Group SLOB_FLEX_DG is provisioned on VMware Cloud Flex Storage
  • Rest of the vmdk’s are provisioned on vSAN WorkloadDatastore using default VSAN storage policy – ‘vSAN Default Storage Policy’

Details of ASM Disk Group SLOB_DG on vSAN and ASM Disk Group SLOB_FLEX_DG on VMware Cloud Flex Storage are shown as below.

Oracle ASM Disk Group details are as below:

grid@oracle21c-ol8:+ASM:/home/grid> asmcmd lsdg
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 512 4096 1048576 204799 0 0 0 0 N DATA_DG/
MOUNTED EXTERN N 512 512 4096 4194304 133116 57452 0 57452 0 N FRA_FLEX_DG/
MOUNTED EXTERN N 512 512 4096 4194304 102396 102292 0 102292 0 N GRID_DG/
MOUNTED EXTERN N 512 512 4096 1048576 102399 23755 0 23755 0 N REDO_DG/
MOUNTED EXTERN N 512 512 4096 4194304 255996 255900 0 255900 0 N SLOB_DG/
MOUNTED EXTERN N 512 512 4096 4194304 266236 17860 0 17860 0 N SLOB_FLEX_DG/
grid@oracle21c-ol8:+ASM:/home/grid>

Oracle ASM disks details are as below:

grid@oracle21c-ol8:+ASM:/home/grid> asmcmd lsdsk -k
Total_MB Free_MB OS_MB Name Failgroup Site_Name Site_GUID Site_Status Failgroup_Type Library Label Failgroup_Label Site_Label UDID Product Redund Path
204799 0 204799 DATA_DISK01 DATA_DISK01 00000000000000000000000000000000 REGULAR ASM Library – Generic Linux, version 2.0.17 (KABI_V2) DATA_DISK01 UNKNOWN ORCL:DATA_DISK01
133116 57452 133119 FRA_FLEX_DISK01 FRA_FLEX_DISK01 00000000000000000000000000000000 REGULAR ASM Library – Generic Linux, version 2.0.17 (KABI_V2) FRA_FLEX_DISK01 UNKNOWN ORCL:FRA_FLEX_DISK01
102396 102292 102399 GRID_DISK01 GRID_DISK01 00000000000000000000000000000000 REGULAR ASM Library – Generic Linux, version 2.0.17 (KABI_V2) GRID_DISK01 UNKNOWN ORCL:GRID_DISK01
102399 23755 102399 REDO_DISK01 REDO_DISK01 00000000000000000000000000000000 REGULAR ASM Library – Generic Linux, version 2.0.17 (KABI_V2) REDO_DISK01 UNKNOWN ORCL:REDO_DISK01
255996 255900 255999 SLOB_DISK01 SLOB_DISK01 00000000000000000000000000000000 REGULAR ASM Library – Generic Linux, version 2.0.17 (KABI_V2) SLOB_DISK01 UNKNOWN ORCL:SLOB_DISK01
266236 17860 266239 SLOB_FLEX_DISK01 SLOB_FLEX_DISK01 00000000000000000000000000000000 REGULAR ASM Library – Generic Linux, version 2.0.17 (KABI_V2) SLOB_FLEX_DISK01 UNKNOWN ORCL:SLOB_FLEX_DISK01
grid@oracle21c-ol8:+ASM:/home/grid>

Test Use case

This blog vetted out the Oracle workload performance, using vSAN WorkloadDatastore first and then using VMware Cloud Flex Storage as the archivelogs destination.

Our test was to run SLOB workload generator against ASM disk backed vmdk to study Oracle workload performance with the below storage locations and study the workload test results

  • vSAN WorkloadDatastore
  • VMware Cloud Flex Storage

SLOB 2.5.4.0 was chosen as the load generator for this exercise with following SLOB parameters set as below:

  • UPDATE_PCT=100
  • RUN_TIME=300
  • SCALE=40G
  • WORK_UNIT=3

We deliberately chose the minimum Work Unit size to drive the most amount of IO without stressing REDO to study the performance metrics differences between the 2 scenarios, Oracle workload  performance with both vSAN WorkloadDatastore and VMware Cloud Flex Storage. The number of threads was set to 50.

We ran multiple SLOB runs for the 2 above use cases below and compared the Oracle & GOS metrics;

With the Oracle workload location on vSAN WorkloadDatastore v/s VMware Cloud Flex Storage, we achieved comparable performance

  • Executes (SQL)/sec – Difference was 0.3% and run results were predictable and consistent
  • Transactions/sec – Difference was 0.3% and run results were predictable and consistent

In conclusion, we were able to get comparable results with the Oracle workloads on VMware Cloud Flex Storage as compared to vSAN WorkloadDatastore.

Looking at GOS Load statistics, we can see that the %user, %system, %idle & %IO wait time statistics are comparable across both vSAN WorkloadDatastore and VMware Cloud Flex Storage vmdk’s.

Looking at GOS disk statistics, we can see that the disk statistics are comparable across both vSAN WorkloadDatastore and VMware Cloud Flex Storage vmdk’s.

This blog is not meant to be a performance benchmarking-oriented blog in any way.  Remember, any performance data is a result of the combination of hardware configuration, software configuration, test methodology, test tool, and workload profile used in the testing, so the performance improvement I got with my workload in my lab is in no way representative of any real production workload which means the performance improvements for real world workloads will be better.

Summary

This blog is the third blog in the series of evaluating VMware Cloud Flex Storage as a viable & comparable Storage option for Oracle non-production workloads.

The first blog Providing Performance and Storage savings using VMware Cloud Flex Storage for Oracle Archivelogs  was an exercise to evaluate the feasibility and showcase the advantages of using VMware Cloud Flex Storage for business-critical Oracle workloads on VMware Cloud on AWS as a viable & comparable Storage option for Oracle Archivelog files destination.

The second blog Providing Storage savings and Comparable performance using VMware Cloud Flex Storage for Oracle Backups was an exercise to evaluate VMware Cloud Flex Storage as a viable & comparable Storage option for Oracle backups destination

With the Oracle workload location on vSAN WorkloadDatastore v/s VMware Cloud Flex Storage, we achieved comparable performance

  • Executes (SQL)/sec – Difference was 0.3% and run results were predictable and consistent
  • Transactions/sec – Difference was 0.3% and run results were predictable and consistent

Looking at GOS Load statistics, we can see that the %user, %system, %idle & %IO wait time statistics are comparable across both vSAN WorkloadDatastore and VMware Cloud Flex Storage vmdk’s.

Looking at GOS disk statistics, we can see that the disk statistics are comparable across both vSAN WorkloadDatastore and VMware Cloud Flex Storage vmdk’s.

This blog is not meant to be a performance benchmarking-oriented blog in any way.  Remember, any performance data is a result of the combination of hardware configuration, software configuration, test methodology, test tool, and workload profile used in the testing, so the performance improvement I got with my workload in my lab is in no way representative of any real production workload which means the performance improvements for real world workloads will be better.

Acknowledgements

This blog was authored by Sudhir Balasubramanian, Senior Staff Solution Architect & Global Oracle Lead – VMware.

Conclusion

Non-production workloads e.g. Development, Test, QA, Staging, Functional, Pre-Production etc. are equally important as the Production workloads , as they are key entities and instrumental to any Production workload success.

Ensuring that Pre-Production databases are comparable in performance with the Production databases is key in ensuring that any performance related issues are identified well ahead in the process before they make it to the Production environment. In addition, ensuring that there is adequate testing and validation ensures due diligence is done with any software release.

All Oracle on VMware vSphere collaterals can be found in the url below.

Oracle on VMware Collateral – One Stop Shop
https://blogs.vmware.com/apps/2017/01/oracle-vmware-collateral-one-stop-shop.html

About Sudhir Balasubramanian

27 + years Oracle hands on experience - Principal Oracle DBA / Architect, Oracle RAC/Data Guard Expert, Experienced in EMC SAN Technologies Currently – VMware (Broadcom) [2012-] Senior Staff Solution Architect & Global Oracle Practice Lead Principal Oracle DBA/Oracle Architect (1995 – 2011) [ Tata Consultancy Services (TCS), Sony Electronics, Newgen Results (Aspen) ,Teletech Corp, SAIC, Active Network, Sempra Energy Holdings] VMware VCA – Cloud ,VMware vBCA Specialist VMware vExpert & vExpert Application Modernization - https://vexpert.vmware.com/directory/1038 Member of the Office of the Chief Technical Ambassador VMware (Alumni) Oracle ACE - https://apex.oracle.com/pls/apex/f?p=19297:3 Leading Author of “Virtualizing Oracle Business Critical Databases on VMware SDDC” Recognized Speaker@ Oracle Open World, IOUG, VMworld, VMware Partner Exchange, EMC World, EMC Oracle Summit and Webinars Industry recognized expert in Oracle Virtualization technologies Blogs http://vracdba.com/ | https://community.oracle.com/blogs/sudhirb https://blogs.vmware.com/apps/author/sudhirbalasubramanian/ Twitter : @vracdba [ https://twitter.com/vRacDba ] LinkedIn : https://www.linkedin.com/in/sudhirbalasubramanian/
This entry was posted in Oracle. Bookmark the permalink.