Final Checks and OCFS2 Setup

Friday May 5th 2006 by Tarry Singh

Part 11 of "RACing ahead with Oracle on VMware" continues with the final checks and the installation and configuration of OCFS2.

A Brief Pep talk: Finally there

In part 10, we covered an important topic called security. I think with the world (IT and likewise) gearing up for Utility Computing that virtualization will play greater role in helping utility computing become reality. There is a lot of talk of hosting applications from distant servers. The concept of AJAX will be used to eventually develop a fully functional OS on the web (webOS) and applications (there are already so many of them) that will make almost everything virtual--a kind of IBM mainframe concept, where you get your space on a big fat server somewhere. OS, Network, almost everything will be virtual. Certainly, security and performance will be the biggest challenges to organizations worldwide. Oracle recently launched the Database Vault, which I think is a very smart move. "Protect yourself from yourself". That will be the motto when we all go out on the web.

Ok now let's get on to testing our configurations and on our path to install and configure OCFS2 for our RHEL 4.2. (There is already a RHEL 4.3/Centos 4.3 available for download). So what will we be doing?

  • Final Checks
  • OCFS2

Final Checks

Checking the /etc/modprobe.conf file, the following lines must be present.

options hangcheck-timer hangcheck_tick=30 hangcheck_margin=180

Checking the /etc/sysctl.conf file, the following lines must be present.

kernel.sem=250 32000 100 128

Checking the /etc/hosts.conf file, all your PRIV, VIP and Public addresses must be present (all machines).

Checking the /etc/hosts.equiv file, remember the file we needed for logging in for copy operations? The following lines must be present:

+node1 oracle
+node2 oracle
+node1-priv oracle
+node2-priv oracle

Checking the required RPMs for your linux distribution.

Now this can be pretty tricky. I have had pretty nasty experiences when installing Oracle (and also RAC) on distributions which were similar to RHEL but not exactly the same. As a rule of thumb, for development purposes, I go ahead and do a complete install. If you follow the installation procedure for RHEL 4.2/Centos 4.2 on my installation article then all should go fine.

The Oracle manual does say (a bit outdated since U3 is already out there) :

Red Hat Enterprise Linux 4 (Update 1) Packages

The following packages (or later versions) must be installed:



  • openmotif21-2.1.30-11.RHEL4.2 is required to install Oracle demos.
  • GNU Compiler Collection (GCC) 2.96 is not supported on Red Hat Enterprise Linux 4.0.

Installing and Configuring OCFS2

Did I mention OCFS2? Oracle develops a lot of its applications on linux and OCFS2 is one of the many projects. I did blog about it briefly, during Oracle's speculation of acquiring its own Linux. Anyways what is it? In OCFS team's words :

OCFS2 is the next generation of the Oracle Cluster File System for Linux. It is an extent based, POSIX compliant file system. Unlike the previous release (OCFS), OCFS2 is a general-purpose file system that can be used for shared Oracle home installations making management of Oracle Real Application Cluster (RAC) installations even easier. Among the new features and benefits are:

  • Node and architecture local files using Context Dependent Symbolic Links (CDSL)
  • Network based pluggable DLM
  • Improved journaling / node recovery using the Linux Kernel "JBD" subsystem
  • Improved performance of meta-data operations (space allocation, locking, etc).
  • Improved data caching / locking (for files such as oracle binaries, libraries, etc)

Follow this project closely.

What do I need for my OCFS2? For a listing of software and where to get it, see Part 6 of this series. I also recommend that you go ahead and install the OCFS2Console as it comes in handy as we move ahead to install and configure OCFS2..

Installing OCFS2

Installing OCFS2 is pretty simple. I put all the binaries in the /tmp folder and did the following:

Configuring OCFS2

Choose cluster/Configure nodes:

You get this message:

Configuring nodes for OCFS2

Click on add.

Port 7777 is automatically chosen.

Add both (or more nodes) and it will look like this:

Click apply to activate them.

Close the window, quit the application and go to the /etc/ocfs2/cluster.conf file.

I unload and load the cluster as you can see here (not necessasy as you can see in the help command, you can query the OCFS2 status):

Starting the cluster manually:

The O2CB should be able to start on boot but with my installation (where I also had to downgrade my kernel to 2.6.11.x) I had to do this in order to get my O2CB driver to load on system startup (With the newer version it might not be needed).

After fixing this, run (remember we will do this on all nodes):

Then do the following (You do this to re-set the boot properties so the O2CB loads on system startup):

Formatting the files with OCFS2

Note: Here we will only format the files where we will keep our OCR, CRS and ASMfile shared files. We will use ASMlibs for all other data files (which we will cover in the next article).

Note2: Unlike other operations, do this ONLY on ONE NODE, it propogates the changes to all nodes.

Choose format:

Here you get to pick the files which we partitioned with fdisk:

Click OK:

Yes and formatting:


After formatting , you will need to mount the disks on ALL nodes!

I did eventually end up adding the third (for my spasmfile), I just don't know why I didn't see it before, during formatting.

Configure OCFS to automatically mount on startup

Edit your /etc/fstab file like this:

Check it by typing "mount":

Note: Note that _netdev mount option. It is essential for OCFS2. It basically means that the volume must be mounted after the network is started and dismounted before the network is shutdown.

Also do chkconfig --list o2cb to see the startup options.

How about permissions?

Do the following (that chkconfig command is illustrated for the above mentioned instruction:


Here we took a good look at OCFS2 and how it works. We formatted our raw devices with OCFS2 and mounted them on all nodes. We made adjustments to cope with the startup during system startup. In our next article, we will pick up the trail and go ahead with ASMlib installation and setup.

» See All Articles by Columnist Tarry Singh

Mobile Site | Full Site