RE: SHMBASE 0x44000000

Hi, SHMBASE does not need to be the same for all instances, at least there's no technical need for this. However, if it's the same for all, then things may be easier to administrate ... Apart from this, to learn more about SHMBASE and its implications, you can read the rather detailed and longish information below (newest at top) ... Regards, Martin -- Martin Fuerderer IBM Informix Development Munich, Germany Information Management ---------------------------------------------------------------------------------- From: "MAEVE DEVINE" maevedevine@yahoo.com Sent by: forum.subscriber@iiug.org Date: 28.06.2005 09:26 To: ids@iiug.org Subject: Re: Creating a temp dbspace failing on RedHat ES 4.0 [5284] Thanks for all the replies, and huge thanks to Sandor Szabo who got this working. 1. Set an enviroment variable INFORMIXSHMBASE to -1 2. In the ONCONFIG, set SHMBASE to 0x44000000 After that it all works fine. Details from Sandor: RHEL 3 Update 3 added a new feature "Exec Shield Randomize" and turned it on by default. IBM Informix application with this release will unpredictably fail to connect to a local IBM Informix database instance because shmat() will fail to attach shared memory at the configured location. If you use SHM communication then I would recommend setting environment variable INFORMIXSHMBASE to -1 because of the exec-shield-randomize feature.... for example: export INFORMIXSHMBASE=-1 This will use the SHMBASE of 0x44000000 which should be set in $ONCONFIG. ---------------------------------------------------------------------------------- How to Change the Location of Shared Libs for a Process Beginning with kernel version 2.4.19, Linux provides a way to dynamically change the default start address for shared libraries on a per-process basis. This feature is available, if the file /proc/$$/mapped_base exists. To change the start address for shared libraries of the oninit processes, the new start address needs to be specified by user root in the shell from where oninit is started. Example: $ echo $$ 29712 $ su root Password: # # the following sets the start address of shared libraries to 0xB0000000: # echo 0xB0000000 > /proc/29712/mapped_base # exit $ oninit Assuming the $ONCONFIG parameter SHMBASE is 0x10000000, this gives 2.5 GB of contiguous address space available for the database server. ---------------------------------------------------------------------------------- Excerpt from Machine Specific Notes on Linux 9.30.UC1 - The default start address for shared libraries on Linux is 0x40000000. Therefore, the maximum available space for shared memory is 768 MB when using 0x10000000L as the SHMBASE value. SHMBASE can also be set to start above the shared library addresses. When doing so, ensure that dynamically loaded shared libraries do not collide with the shared memory segments. The address space layout can be checked by the following command: $ cat /proc/<pid of oninit process>/maps ----------------------------------------------------------------------------------Memory utilization on Linux (Intel) : ------------------------------------- The problem here is, that on Linux a process has 3 GB virtual address space. Part of it is used for the executable (text and data segments), the heap, shared libraries and the stack. On Linux, the executable sits at the bottom (0x0 and up), next comes some heap, then shared libraries start at 0x40000000, the stack grows top down, from 0xc0000000 downwards. With SHMBASE (onconfig) we place the SHM in the heap area, i.e. it can only grow up to 0x40000000 (where the shared libs start). This can be optimized by placing the SHM above the shared libs. Once IDS is up, the stack doesn't grow a lot and the shared libs should be stable as well. You can check this by looking at /proc/<PID>/maps (PID - process id of first CPU-VP). From there you determine where the last shared library was mapped, round up that value and use it as SHMBASE. That way, with 9.20.UC1 we managed to get SHM size of 1,25 GB. The problem with shared libs is, that relocating them to some lower address is not easy. It involves modifying and re-building the kernel and is not really an option. The problem with placing SHM above shared libs as described above is, that it basically assumes no new shared libs will be loaded during runtime. However this can happen in IDS 9.x through the use of Datablades. Datablades in fact are shared libraries that are not known (to be loaded) during startup time of the server. And there are some Datablades that may be used for internal purposes (apart from "external" Datablades, i.e. User Defined Routines for User Defined Types). Therefore the best bet is to examine a systems memory map during normal runtime and with normal userload. From there determine a relatively safe setting for SHMBASE above the shared libs. Example : % cat maps # executable 08048000-08698000 r-xp 00000000 16:42 84921 /work1/9.21_32/sqldist.N182/bin/oninit 08698000-087b0000 rw-p 0064f000 16:42 84921 /work1/9.21_32/sqldist.N182/bin/oninit # heap + SHM 087b0000-087e0000 rwxp 00000000 00:00 0 # heap # compare these with the output from onstat -g seg : 10000000-11377000 rwxs 00000000 00:00 0 # SHM segment 1 11377000-11b77000 rwxs 00000000 00:00 0 # SHM segment 2 11b77000-11c19000 rwxs 00000000 00:00 0 # SHM segment 3 11c19000-12019000 rwxs 00000000 00:00 0 # SHM segment 4 # shared libs 40000000-40012000 r-xp 00000000 03:02 2019 /lib/ld-2.1.2.so ... 40184000-4018c000 r-xp 00000000 03:02 2061 /lib/libnss_nis-2.1.2.so 4018c000-4018d000 rw-p 00007000 03:02 2061 /lib/libnss_nis-2.1.2.so # stack (growing backwards from 0xc0000000) bffe8000-c0000000 rwxp fffe9000 00:00 0 So to place the SHM segments after the shared libs, SHMBASE could be set e.g. to 0x40300000 or 0x41000000 (it has to be a sensible border). Apart from the above, keep in mind, that the kernel parameter for maximum shared memory also may need tuning. This can be done by placing the desired value into the pseudo file /proc/sys/kernel/shmmax. -- 09/04/2000, martinfu ---------------------------------------------------------------------------------- owner-informix-list@iiug.org wrote on 06.09.2005 10:33:27: > SHMBASE should be set to he same value for all instances, it is platform > specific as detailed in the Machine Release Notes. The SERVERNUM in each > onconfig file differentiates the memory for each server. > I'm not familiar with Linux kernel adjustment, but doubling the value might > be a good starting point. > > Keith > > > -> -----Original Message--