Configuring RHEL/Centos 4.2 Server for Oracle RAC 10g R2 Readiness

Thursday Apr 6th 2006 by Tarry Singh

Part 9 of this series continues the setup of the RHEL/Centos 4.2 server, covering setting up the Swap Space, Shared Memory, Semaphores, File Handles and Hangcheck Timer.

A Brief Pep talk: Continuing the preparation

In part 9 of this series, we will continue setting up our Redhat/Centos 4.2. There is already a 4.3 release downloadable at their respective sites (Redhat and Centos). We will go ahead and setup Swap space, Memory, Semaphores, File Handles and Hangcheck-timer.

On another note on market developments and RACs path to commoditization, Oracle is constantly working on its pricing model and CRM giants like Salesforce.com and NetSuite are making others seriously think about charging clients monthly instead annually. The SaaS is getting a lot of attention and soon a lot of companies are expected to use this model against ASP for solid reasons like automated software updates. Don’t forget that the client is smart enough to calculate, so no matter which model you pick, you better come up with an acceptable pricing model.

Ok we will continue on our path to configure our Linux server. Check out part 7 if you have forgotten where you were last time. We will go ahead and pick the following:

  • Swap Space
  • Memory
  • Semaphores
  • File handles
  • Hangcheck-timer
  • Remote access checks (both rsh – not recommended and scp – recommended)
  • Final check on changes
  • Verify required RPMs

Setting up Swap Space

You should have a minimum of 512 MB RAM, I would personally advise to go ahead with a simple PC with 2G as you will be setting up your Virtual Machines, and 1G memory for the virtual machines is a good starting point. When you do the cluvy check it does complain if the machine’s memory is under the acceptable level, however, you can go ahead and install.

How much memory do I have? Simply do,

# cat /proc/meminfo | grep MemTotal

How much swap is allocated? Again type

# cat /proc/meminfo | grep SwapTotal

Click for larger image

Shared Memory Setup

As a DBA, you know that your SGA uses shared memory to avoid unnecessary I/O like data copying (unless, of course, it does not exist in the memory). This is the fastest way to retrieve data and thus it is crucial that you setup your shared memory on your Redhat/Centos Server. Moreover, appropriately sizing your Oracle database SGA is key to optimal performance. Here in Oracle’s online documentation, you can see for yourself how several memory structures are divided in SGA, such as database buffer cache, log buffers, shared SQL, and access paths to name a few. See more on Oracle’s memory architecture here.

Check out your shared memory limits.

1.  The SHMMAX parameter: Set the maximum (SGA must fit in it or else you will get an error) size of the memory segment. Default value is 32MB and the idea is to set it to 2G. Therefore, we will do the same.

What is my SHMMAX?

# cat /proc/sys/kernel/shmmax

Increase it to 2G

# sysctl -w kernel.shmmax=2147483648

Then edit your /etc/sysctl.conf to make it permanent.

2.  The SHMMNI parameter: Minimum of 4096 is ok.

# cat /proc/sys/kernel/shmmni     4096

3.  The SHMALL parameter: This contains the total memory.

    # cat /proc/sys/kernel/shmall    2097152

This is the default and should be fine.

You can check it all like this.


What is it anyways?

Semaphores are like counters, used to control access to shared resources by multiple processes. They are most often used as a locking mechanism to prevent processes from accessing a particular resource while another process is performing operations on it (thus allowing the other resource to wait). Semaphores are often dubbed the most difficult to grasp of the three types of System V IPC objects. We will quickly brush through the details here.

What are my semaphore limits and how do I set them?

File Handles

This means the maximum allowable open files and descriptors. Even when running Apache, you have a huge load of descriptors opening (there is a theoretical limit of one million). We will stick to the Oracle recommended of 64K or 65536 (64*1024).

How many handles do I have open?

# cat /proc/sys/fs/file-max

How do I change it permanently? To avoid any problems when increasing that value, I would suggest running the ulimit command. Also, note that since kernel 2.4 and up, you do not need to run the # cat /proc/sys/fs/inode-max command.

Finally, restart the network.

/# /etc/rc.d/init.d/network restart

Hangcheck Timer

Until Oracle, we used the watchdog daemon; from onwards you have a hangcheck-timer module loaded and available for you in the Linux kernel itself. Check out the Project Hangcheck-timer for more details.

Since it is already shipped with kernel 2.5 and up, you do not need to install it. Do the following to check, configure and write the changes to the modprobe.conf file.

Let's look at those parameters. The hangcheck_tick checks periodically for system health. It has a default value of 60 but Oracle recommends 30. On a typical Vmware RAC we will play with this setting and Oracle specifically recommends for Vmware from their Cookbooks:

"VMWARE NOTE: For VMWARE use HIGH values, otherwise machine will reboot if brought back from suspend mode, these are good values:hangcheck_tick=9000000 hangcheck_margin=36000000"

I personally have not had many problems with those settings but like I said, play with the settings and see how your VMs react to it.

Similarly, hangcheck_margin is the margin before your hanfcheck-timer resets your node. Therefore, it must be a reasonable figure. Oracle recommends setting it to 180 seconds.


We are getting close to completing configuration. In our next article, we will continue setting up remote access and we will check both rcp and the 10g Release 1 compatible SSH support (or better said scp). We will walk through both the configurations and do a final check on all our settings. Then we will continue with OCFS2 installation.

» See All Articles by Columnist Tarry Singh

Mobile Site | Full Site