Oracle
10g x86 on SUSE LINUX
Enterprise Server 9 AMD64
This mini-HOWTO will cover the installation of oracle on SLES9 (SUSE
LINUX Enterprise Server 9) for x86 and AMD64 architectures.
If you find any error imprecision or simply you have suggestions (or
you need help) feel free to contact me at fabrizio.magni _at_ europe.com
Some notes before starting:
- 10g for linux is only available (at this moment) as 10.1.0.2 for
x86 (a patchset was released but withdrew due to a dangerous bug on
CLOB);
- on this website you can even find the 9.2.0.4 installation for
x86-64 on AMD64 mini-howto.
- the x86-64 architecture allow 64 and 32 bit applications to live
together even if you have to follow some more step for installing and
using the 32 bit part.
- SLES9 for AMD64 can be downloaded both from novell and SuSE
portals.
- Oracle10g for x86 is available from OTN.
As always I'd like to thank the people in the suse-oracle mailing list
and people on comp.databases.oracle.server newsgroup.
I don't wish to cover the operating system installation in this paper
even because the installation is pretty easy.
I suggest you to chose the basic installation and then to add some
packages later using yast.
Oracle, starting with 10g, suggest you don't use neither a filesystem
nor rawdevices for your DB but ASM (I'll try to cover it later) but it
is not mandatory so you can decide to switch back to the old cooked
device or raw devices. The type of filesystem is up to you, your
experience and tastes. There are several papers which covers
performance for every fs flavour. Scalzo's one are the most popular.
Have a look before deciding.
Personally I prefer ext3 for datafiles.
I preferred to recompile my own kernel due to problems with hugetlb,
opteron architecture and numa support. You can skip this part.
Hugetlb are not an issue for 9i but they procure a headache on 10g. I
haven't solved my issues on this topic yet but later I provide a good
link with several solutions.
After installing the basic system make sure you have the libX, libaio,
compat, libaio-devel and openmotif (even the 32 bit version).
glibc-devel-2.3.3-98.28
glibc-2.3.3-98.28
glibc-32bit-9-200407011233
glibc-devel-32bit-9-200407011229
compat-2004.7.1-1.2
compat-32bit-9-200407011229
XFree86-libs-4.3.99.902-43.22
XFree86-libs-32bit-9-200407011229
libaio-devel-0.3.98-18.4
libaio-32bit-9-200407011229
libaio-0.3.98-18.4
libaio-devel-32bit-9-200407011229
openmotif-libs-2.2.2-519.1
openmotif-2.2.2-519.1
Installing orarun make your installation easier so I reccomand it.
However read the notes below before installing.
Last version is orarun-1.8-109.5 and can be downloaded from Novell's
ftp.
Note on gcc:
gcc_old-2.95.3-11 is not necessary (as describe on some website). On
the contrary the linking phase needs gcc 3.x!!!
So be warned: if you are going to install older gcc for any purpose
make sure oracle looks for the 3.x version during the relink.
Note on orarun:
Orarun is a useful package which can simplify the preinstallation part.
linux: # rpm -Uvh
orarun-1.8-109.5.i586.rpm
Create the directory tree for the oracle installation (look at the
standard OFA): the default is /opt/oracle/product/10g/db_1.
I prefer /u01/app/oracle/product/10g/db_1
linux: # mkdir -p
/u01/app/oracle/product/10g/db_1
Make sure to chenge the ownership of the tree with chown (the owner
should be the oracle user while the group oinstall).
linux: # chown -r oracle:oinstall
/u01/app/oracle
Now you can modify some files in /etc:
- /etc/passwd: change the shell for the oracle user created by
orarun (default is /bin/false);
- /etc/group: oracle user should belong to dba and oinstall;
- /etc/sysconfig/oracle for ORACLE_BASE, ORACLE_HOME, ORACLE_SID
and several kernel parameters plus the starting parameter for the
oracle script in /etc/init.d (useful during machin boot).
- /etc/profile.d/oracle.sh (or oracle.csh depending on the shell
you chose above). Make sure to set LD_ASSUME_KERNEL='2.4.21' (other
value could be used: read Ulrich Drepper paper at
http://people.redhat.com/drepper/assumekernel.html).
- modify /etc/sysctl.conf (the explanation in the hugetlb session,
below) and add these two lines:
vm/disable_cap_mlock=1
vm/nr_hugepages=2048
the nr_hugepages represents the memory
you wish to allocate for oracle with the hugetblfs mechanism. Pagesize
default is 2MB so you have to decide the right number for you.
The disable_cap_mlock is a workaround which permits to every user to
use hugetlb.
Enable manually your changes with
linux: # sysctl -p
Remember to add boot.sysctl amongst your booting scripts (if not done
yet):
linux: # chkconfig boot.sysctl on
or if you prefer:
linux: # ln -s /etc/init.d/boot.sysctl /etc/init.d/boot.d/S11boot.sysctl
(from SLES9 it seems impossible to do it via yast).
- comment the following line in /etc/init.d/oracle:
#test -d /lib/i686 && export LD_ASSUME_KERNEL=2.2.5
or change it in
test -f /lib64/libc.so.6 && export LD_ASSUME_KERNEL=2.4.21
- Add a new line under LD_ASSUME_KERNEL:
test -f /usr/lib/libInternalSymbols.so && export
LD_PRELOAD_32=/usr/lib/libInternalSymbols.so
Reconnect to the machine with the oracle user and check if everything
is ok.
Before installing you have to be aware that linking the 32 bit version
of oracle on a 64 bit architecture need some
further steps.
gcc and ld commands will generate 64 bit binaries and libraries giving
you this kind of error during a "relink all":
| /usr/lib64/gcc-lib/x86_64-suse-linux/3.3.3/../../../../x86_64-suse-linux/bin/ld:
skipping incompatible /app/oracle/product/10g/db_01/lib/libsql10.a when
searching for -lsql10 |
The simplest way to solve the problem is to rename gcc and ld:
linux: # mv /usr/bin/gcc
/usr/bin/gcc.orig
linux: # mv /usr/bin/ld /usr/bin/ld.orig
And create two wrappers:
cat >gcc <<"EOF"
#!/bin/bash
exec /usr/bin/gcc.orig -m32 $@
EOF
cat >ld <<"EOF"
#!/bin/bash
exec /usr/bin/ld.orig -melf_i386 $@
EOF
linux: # chmod a+x /usr/bin/gcc
linux: # chmod a+x /usr/bin/ld
After the installation you can revert what you have done (it is up to
you) and have your original gcc and ld back.
But what if you want to issue a "relink all"?
You can re-execute the above steps or you can do the following:
create a new directory as root:
linux: # mkdir /usr/bin32
linux: # cd /usr/bin32
Then define two scripts:
cat >gcc <<"EOF"
#!/bin/bash
exec /usr/bin/gcc -m32 $@
EOF
cat >ld <<"EOF"
#!/bin/bash
exec /usr/bin/ld -melf_i386 $@
EOF
linux: # chmod a+x gcc
linux: # chmod a+x ld
Comment line 58 and 59 in /app/oracle/product/10g/db_01/bin/genclntsh,
40, 41 in /app/oracle/product/10g/db_01/bin/genclntst,
88 in /app/oracle/product/10g/db_01/bin/genagtsh
(the one referring to the explicit declaretion of PATH).
Then change the path in your oracle user env (/etc/profile.d/oracle.sh)
to be:
PATH=/usr/bin32:$PATH:$ORACLE_HOME/bin
in this way the gcc and ld will be called in /usr/bin32 first and will
generate 32 bit code.
Swith in 32 bit mode and then relink:
oracle@linux > linux32 bash
oracle@linux > relink all
This operation requires more work but I prefer to give the chance to
the other users on the system to generate 64 bit code.
The choise, as always, is up to you.
If issuing a relink all you you get:
/app/oracle/product/10g/db_01/bin/relink: line 59: [: too many arguments
Then comment the line from 59 to 63 in
/app/oracle/product/10g/db_01/bin/relink.
You are now ready to install the oracle engine. Insert the CD, mount it
and run the installer:
- mount /dev/cdrom (if you didn't insert oracle in the cdrom group
then the command ahs to be issued by root).
- login as oracle user and start a bash in 32 bit mode:
oracle@linux > linux32 bash
NOTE: Every time you wish to use a
32 bit version of an oracle executable (even to start the instance) use
linux32!!!
- if you are in a remote machine make sure your X server is running
and export the DISPLAY: export DISPLAY=<your local IP>:0.0;
- /media/cdrom/runInstaller -ignoresysprereqs
Known issues:
People referred a problem with glibc
lanching oraInstaller:
| Unable to load native library:
/tmp/OraInstall2004-02-24_10-40-59AM/jre/lib/i386/libjava.so: symbol
__libc_wait, version GLIBC_2.0 not defined in file libc.so.6 with link
time reference. |
Setting LD_ASSUME_KERNEL='2.4.21' and LD_PRELOAD_32 should solve the
problem in either installation: 9i and 10g.
Another way to solve the problem is to install patch #3006854 from
metalink which create a new library containing the definition for
__libc_wait.
If you can't access metalink then you can create your own patch as
suggested by Jakub Jelinek (I found the trick on suse-oracle mailing
list):
------- From Jakub Jelinek on 2003-11-21 16:49 -------
Try using some less buggy JVM.
Latest Sun JDK should work just fine for example.
Or, as a workaround for the buggy JVM, you can try:
gcc -O2 -shared -o ~/libcwait.so -fpic -xc - <<\EOF
#include <errno.h>
#include <sys/syscall.h>
#include <sys/types.h>
#include <sys/wait.h>
pid_t
__libc_wait (int *status)
{
int res;
asm volatile ("pushl %%ebx\n\t"
"movl %2, %%ebx\n\t"
"movl %1, %%eax\n\t"
"int $0x80\n\t"
"popl %%ebx"
: "=a" (res)
: "i" (__NR_wait4), "0" (WAIT_ANY), "c" (status), "d" (0),
"S" (0));
return res;
}
EOF
--------------------------------------------------------
export LD_PRELOAD_32=~/libcwait.so
now you can install.
Using HUGETABLE in oracle and
"ORA-27125: unable to create shared
memory segment"
On SLES9 hugetlb is enable by default and it could cause you problems.
A good resource to resolve this kind of issue can be found here:
http://linux.inet.hr/oracle10g_on_debian.html
However on oracle9i if you wish to use this feature (without the
DISABLE_HUGETLBFS=1 trick) you will need patch #3386122 from metalink
which is available for x86 only. I asked a backport for x86-64 but I'm
still waiting.
On 10g you don't need any patch.
The common way to avoid ORA-27125 on 10g is to disable hugetlb with:
linux: # cd $ORACLE_HOME/bin
linux: # mv oracle oracle.bin
cat >oracle <<"EOF"
#!/bin/bash
export DISABLE_HUGETLBFS=1
exec $ORACLE_HOME/bin/oracle.bin $@
EOF
linux: # chmod +x oracle
This is necessary because the env variables aren't passed to the oracle
command by java applications such as dbca (otherwise to define
DISABLE_HUGETLBFS=1 in your environment should be enough).
If you are experiencing:
ORA-12547: TNS:lost contact
this is likely due to a wrong wrapper generation. Make sure you didn't
create the file above with "vi". Use "cat" instead.
(Another cause could be a wrong relinking phase. Check if "relink all"
works correctly).
But why not to use this
new
kernel feature?
To allocate huge pages do:
echo 2048 > /proc/sys/vm/nr_hugepages
A page, by dafeult, is 2MB so with 2048 you are allocating 4GB of your
RAM for hugepages.
Check it in /proc/meminfo.
There is even the chance to mount a hugetlb file system.
Create a mount point:
mkdir /dev/hugetlbfs
and mount it:
mount -t hugetlbfs -o uid=55,gid=59,mode=0777 hugetlbfs /dev/hugetlbfs
This last bit is not mandatory but can be useful for debugging and
advanced management.
Now you don't need anymore DISABLE_HUGETLBFS=1 or a wrapper for the
"oracle" command.
To make the changes permanent insert these lines into /etc/sysctl.conf
vm/nr_hugepages=2048
vm/disable_cap_mlock=1
And this into /etc/fstab (only if you wish to mount a hugetlbfs).
hugetlbfs
/dev/hugetlbfs hugetlbfs
mode=0777,uid=59,gid=55 0 0
This will allow your oracle user to access the memory in a way similar
to the shm mechanism.
If you don't want to reboot your machine you can issue these commands:
sysctl -p
mount -a
which set all the variables in sysctl.conf in your proc and mount all
the file systems listed in fstab.
To check if your instance is using hugetlbfs use
cat /proc/meminfo |grep Huge
HugePages_Total: 1024
HugePages_Free: 940
Hugepagesize: 2048 kB
The value of Total and Free should be different.
If you are using a kernel without the patch for disable_cap_mlock then
there is another solution (maybe next SLES9 kernels will include it).
From kernel 2.6.7 place in you sysctl.conf the following lines:
vm/nr_hugepages=2048
vm/hugetlb_shm_group=55
Where hugetlb_shm_group contains the gid of the oracle group (usually
dba).
It is a more secure way to access hugetlb than with disable_cap_mlock.
If you encounter:
ORA-00385: cannot enable Very Large Memory with new buffer cache
parameters
Then you have use_indirect_data_buffers=true in your spfile. Set
it to false (I still haven't found a way to use HUGETLB and use_indirect_data_buffers=true).
Asynch I/O on SLES9:
Even on some documentation it is stated that aync I/O is enable by
default: you should need only to relink the binaries with the command:
linux: # su - oracle
oracle@linux > cd $ORACLE_HOME/rdbms/lib
oracle@linux > make PL_ORALIBS=-laio -f ins_rdbms.mk async_on
oracle@linux > make PL_ORALIBS=-laio -f ins_rdbms.mk ioracle
and set
filesystemio_options=setall in you spfile
which enable direct I/O plus async I/O.
Let's say that SLES9 at this stage seems to break async I/O
compatibility with oracle (at least on 9i) so I'm pasting here the same
advices I have written for the 9i mini-HOWTO.
A common error you could meet is:
ORA-27083: skgfrliopo: waiting for async I/Os failed
My guess is that the libc syscall(2) mechanism introduced by libaio
0.3.97 and above is not supported by oracle.
So my solution is quite simple. Download the sources of libaio 0.3.96-3
here, compile them with 'make' and
then copy
libaio.a
libaio.so.0.1
in $ORACLE_HOME/rdbms/lib.
linux: # cp libaio.a
$ORACLE_HOME/rdbms/lib
linux: # cp libaio.so.0.1 $ORACLE_HOME/rdbms/lib
linux: # ln -s $ORACLE_HOME/rdbms/lib/libaio.so.0.1
$ORACLE_HOME/rdbms/lib/libaio.so.0
in this way you still have the libaio-0.99 shipped with SLES9 on your
system but oracle uses the 0.96 you provided.
If you don't want to insert anything into your oracle installation tree
you can create a directory wherever you wish and modify you
LD_LIBRARY_PATH.
example:
linux: # mkdir -p /usr/oracle/lib
linux: # cp libaio.a /usr/oracle/lib
linux: # cp libaio.so.0.1 /usr/oracle/lib
linux: # ln -s /usr/oracle/lib/libaio.so.0.1
/usr/oracle/lib/libaio.so.0
A good reference for the problem is Simon Grabinar's webpage:
http://www.grabinar.com/simon/aio.html
Contact information:
fabrizio.magni _at_ gmail.com