Brief intro
Normally I tend to go back and try to tune my articles so we are all updated with the latest developments with Oracle RAC, VMware and other tools. Most of these articles are triggered by the reader’s feedback, so, in this article we will revisit the shared storage option when using VMware’s DataCenter Virtualization product we all know as ESX 3.0 or VI3, something which was not covered in the past. We will be covering the shared storage considerations on all versions of VMware. This will be updated to the latest version of VMware Server 1.0.3 and Workstation 6 with ACE option products. In addition, we will cover shared storage on ESX 3.0.1 server in our next article. There are several options, such as creating shared storage on the single ESX host, creating shared storage on multiple hosts, using SAN, etc.
Storage considerations: VI3 ESX Server vs VMware Server, VMware Workstation
In previous articles, we have seen how you have to edit your VMX files to create shared storage. This is the version of the shared storage that we create typically on a VMware Workstation or VMware Server. This updated configuration file includes the features in the award winning version of VMware Workstation 6 with ACE (Assured Computing Environment) features. After having installed your OS into this VM, all you have to do is copy it and restart it. You will be prompted to create a new UUID or use the current one. Go for creating a new one and voila! You have your second node ready.
####################################################################
BEGIN VMWARE SERVER OR WORKSTATION(5,6) CONFIG FILE
####################################################################
config.version = “8”
virtualHW.version = “6”
numvcpus = “2”
scsi0.present = “TRUE”
scsi0.virtualDev = “lsilogic”
memsize = “1024”
ide0:0.present = “TRUE”
ide0:0.fileName = “rac01.vmdk”
ide1:0.present = “TRUE”
ide1:0.fileName = “D:”
ide1:0.deviceType = “cdrom-raw”
floppy0.autodetect = “TRUE”
ethernet0.present = “TRUE”
ethernet0.wakeOnPcktRcv = “FALSE”
usb.present = “TRUE”
ehci.present = “TRUE”
sound.present = “TRUE”
sound.fileName = “-1”
sound.autodetect = “TRUE”
pciBridge0.present = “TRUE”
mks.keyboardFilter = “allow”
displayName = “rac01”
guestOS = “other26xlinux”
nvram = “Other Linux 2.6.x kernel.nvram”
deploymentPlatform = “windows”
virtualHW.productCompatibility = “hosted”
tools.upgrade.policy = “useGlobal”
ide0:1.present = “TRUE”
ide0:1.fileName = “rac01-swap.vmdk”
ide1:0.autodetect = “FALSE”
floppy0.present = “FALSE”
ethernet1.present = “TRUE”
ethernet1.connectionType = “hostonly”
ethernet1.wakeOnPcktRcv = “FALSE”
isolation.tools.hgfs.disable = “TRUE”
ide1:0.startConnected = “TRUE”
ethernet0.addressType = “generated”
ethernet1.addressType = “generated”
uuid.location = “56 4d 72 4b c0 bc 43 8b-0d 85 ad 60 b4 cc 6a 84”
uuid.bios = “56 4d 72 4b c0 bc 43 8b-0d 85 ad 60 b4 cc 6a 84”
ide0:0.redo = “”
ide0:1.redo = “”
pciBridge0.pciSlotNumber = “17”
scsi0.pciSlotNumber = “16”
ethernet0.pciSlotNumber = “32”
ethernet1.pciSlotNumber = “33”
sound.pciSlotNumber = “34”
ehci.pciSlotNumber = “35”
ethernet0.generatedAddress = “00:0c:29:cc:6a:84”
ethernet0.generatedAddressOffset = “0”
ethernet1.generatedAddress = “00:0c:29:cc:6a:8e”
ethernet1.generatedAddressOffset = “10”
tools.syncTime = “TRUE”
#
# —————————————————————-
# BEGIN SHARED STORAGE
# —————————————————————-
# – Here we NEW SCSI BUS. Set it to “Virtual” and disk.locking = “false”
#
# —————————————————————-
diskLib.dataCacheMaxSize = “0”
diskLib.dataCacheMaxReadAheadSize = “0”
diskLib.dataCacheMinReadAheadSize = “0”
diskLib.dataCachePageSize = “4096”
diskLib.maxUnsyncedWrites = “0”
disk.locking = “false”
# —————————————————————-
# Create HBA (just as if you had a SAN storage), LSILOGIC or BUSLOGIC
# —————————————————————-
scsi1.present = “TRUE”
scsi1.sharedBus = “virtual”
scsi1.virtualDev = “lsilogic”
# —————————————————————-
# Creating virtual SCSI disks on that HBA
# —————————————————————-
scsi1:0.present = “TRUE”
scsi1:0.fileName = “G:\VM\shared\ocr.vmdk”
scsi1:0.redo = “”
scsi1:0.mode = “independent-persistent”
scsi1:0.deviceType = “disk”
scsi1:1.present = “TRUE”
scsi1:1.fileName = “G:\VM\shared\votingdisk.vmdk”
scsi1:1.redo = “”
scsi1:1.mode = “independent-persistent”
scsi1:1.deviceType = “disk”
scsi1:2.present = “TRUE”
scsi1:2.fileName = “G:\VM\shared\spfileasm.vmdk”
scsi1:2.redo = “”
scsi1:2.mode = “independent-persistent”
scsi1:2.deviceType = “disk”
scsi1:3.present = “TRUE”
scsi1:3.fileName = “G:\VM\shared\ASM1.vmdk”
scsi1:3.redo = “”
scsi1:3.mode = “independent-persistent”
scsi1:3.deviceType = “disk”
scsi1:4.present = “TRUE”
scsi1:4.fileName = “G:\VM\shared\ASM2.vmdk”
scsi1:4.redo = “”
scsi1:4.mode = “independent-persistent”
scsi1:4.deviceType = “disk”
scsi1:5.present = “TRUE”
scsi1:5.fileName = “G:\VM\shared\ASM3.vmdk”
scsi1:5.redo = “”
scsi1:5.mode = “independent-persistent”
scsi1:5.deviceType = “disk”
#
# —————————————————————-
# END SHARED STORAGE
# —————————————————————-
#
scsi1.pciSlotNumber = “36”
ethernet0.connectionType = “bridged”
ethernet0.vnet = “VMnet0”
#
# —————————————————————-
# VMWARE WORKSTATION 6 ACE FEATURES: USB SUPPORT,
# APPLIANCE VIEW, BATTERY INFO IN THE HOST ETC
# —————————————————————-
#
chipset.useAcpiBattery = “TRUE”
chipset.useApmBattery = “TRUE”
RemoteDisplay.vnc.enabled = “FALSE”
applianceView.enabled = “FALSE”
applianceView.showAtPowerOn = “TRUE”
applianceView.coverPage.logo = “”
applianceView.coverPage.name = “RAC01”
applianceView.coverPage.author = “Tarry Singh”
applianceView.coverPage.version = “1.0.0.0”
applianceView.coverPage.text = “RAC test bed”
RemoteDisplay.vnc.key = “Kj0lMTwWFTg6JBQiFT47JB4+DDsVMg0QDi4qAA82CTkOHSMGBycyEwc9My0vIgg
GLzkcFAsrBCYdHTQSOysVCRU7EDIPLSA2HxMeMi4tPAg3Nwg1KQ0LCTkyKQg5LSI5PDYjKjg/Bgc8Lh0MOBUYBig2
usb.generic.autoconnect = “FALSE”
##########################################################################
END VMWARE SERVER OR WORKSTATION(5,6) CONFIG FILE
##########################################################################
GLzkcFAsrBCYdHTQSOysVCRU7EDIPLSA2HxMeMi4tPAg3Nwg1KQ0LCTkyKQg5LSI5PDYjKjg/Bgc8Lh0MOBUYBig2Notes on the SCSI clustering
The option of scsi1.sharedBus = “virtual” allows the whole bus to be shared instead of sharing individual disks. However, if you want to do it selectively then you can individually (not recommended though) do the following:
scsi1:1.shared = “true”
This setting will be ignored if you have already shared the whole bus with the previous option. Besides this, the other important issue is about concurrent shared storage access. This is achieved by disk.Locking = “false”. This prevents the locking of that disk and permits multiple VMs to access the disk concurrently.
In addition to enabling SCSI reservation on the bus, you need to allow virtual machines to access the shared disk concurrently. Add the following line to the virtual machine’s configuration file:
disk.locking = “false”
This prevents the disk from locking and allows for concurrent access from multiple Virtual Machines. This setting applies to all the disks. You will notice that when you start your Virtual Machines, there is often a vmdk.RESLCK; this is needed to preserve the shared state of that disk group.
Some Do’s and Don’t when considering shared storage on VMware Server
- Shared disks with VMs that are collocated, meaning running on the same VMware Server or Workstation. You can eventually attach the shared storage via a Firewire (400/800) for better performance and data protection.
- Do not share a disk on SCSI bus 0 as it is used to boot the disk. Use a different bus like SCSI 1.
- If you want to resize your disk (Think of a situation where your ASM experiences a “disk space exhausted” problem and you need to expand the disk), you will want to disable the SCSI reservation and make sure that the disk is not used by other instances.
- Reserve the whole Bus and share it across all collocated VMs. This will enhance performance and also prevent data corruption.
Conclusion
In the next part of this article, we will explore the Shared Storage considerations on the datacenter product called the ESX 3.0.1 Server. Setting up shared storage is not that hard but you need carefully chose the options in order to have a fully functional RAC. You can copy the above-mentioned script on to a notepad, rename it to *.vmx and you have a Virtual machine ready to be installed. I will also attempt to do a test where we will try to clone or even create a typical RAC template (via Virtual Center 2.0.1) and deploy Oracle RAC nodes on the fly. In fact we will also time it, purely to demonstrate that you can have a test-, development- and staging platform for Oracle RAC in a couple of minutes!
