IDS 11 - Slowness with DirectIO
Mark found that enabling DIRECT_IO when moving from IDS 9.4 to 11.x on Red Hat Linux made a dbimport take roughly twice as long, with nothing but checkpoint messages in the log. Replies pointed him to the IBM docs on DIRECT_IO and performance tuning, and suggested the slowdown was likely a configuration issue or a platform/filesystem that doesn't really support direct I/O. The discussion then moved to chunk layout advice: IDS can use raw, cooked or filesystem chunks, cooked chunks with O_DIRECT are only ~5% slower than raw, symbolic links should be used for chunk paths, plus scripted touch/onspaces and SQL Admin API examples for creating spaces. No definitive cause or fix for the slowness is recorded.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
We are currently using IDS 9.4 in production and want to upgrade to 11.x
because he heard there are some speed improvments using DirectIO. I have
upgraded a test server with a copy of our live data. I did a dbimport of out
data with and without the DirectIO option turned on and noticed that with it
on, the import took almost twice as long. There were no message in the log
file other than checkpoint messages. Is there anything else I need to change
with my config when I turn on DirectIO?
Thanks,
Mark
http://publib.boulder.ibm.com/infocenter/idshelp/v111/index.jsp?topic=/com.ibm.a dref.doc/adref69.htm http://publib.boulder.ibm.com/infocenter/idshelp/v111/index.jsp?topic=/com.ibm.p erf.doc/perf118.htm for the second link you need to look at the table of contents on the left and chose the 'direct i/o' link... MM
Platform? Does your platform and filesystem support DIRECTIO? Some
filesystems even on systems that support DIRECTIO do not allow DIRECTIO.
Most likely the speed problem is a configuration problem.
Art
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors (art@iiug.org)
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do
those opinions reflect those of other individuals affiliated with any entity
with which I am affiliated nor those of the entities themselves.
On Tue, Jun 2, 2009 at 2:42 PM, MARK NEWNAM
<mnewnam@carolinahandling.com>wrote:
> We are currently using IDS 9.4 in production and want to upgrade to 11.x
> because he heard there are some speed improvments using DirectIO. I have
> upgraded a test server with a copy of our live data. I did a dbimport of
> out
> data with and without the DirectIO option turned on and noticed that with
> it
> on, the import took almost twice as long. There were no message in the log
> file other than checkpoint messages. Is there anything else I need to
> change
> with my config when I turn on DirectIO?
>
> Thanks,
> Mark
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--001636c5a7e84cf787046b644f72
We are running on RedHat Linux. I was also looking into using a raw device but I see that raw devices are no longer supported in Linux. I have read that IDS will allow you to use an unmounted disk for a space. I was going to try a test this way to see how performance was. Do I use an entire disk (/dev/sdb) or just a partition (/dev/sdb1)? What is the onspace command to create a space this way? Thanks, Mark
IDS can use unmounted RAW (or character) devices, or COOKED (or block)
devices, or files in filesystems for chunks, yes. The command to create a
dbspace from a COOKED device or to add a COOKED device to an existing
dbspace is the same as for RAW or filesystem chunks. That is the onspaces
command. Just use the path to the device in the -p option.
HOWEVER, it is STRONGLY recommended to use symbolic links for chunk paths
rather then the actual device path. In case of a device crash and the need
to restore to a different device, it is tedius in the latest releases (and
impossible in the earlier releases) to have to perform the restore including
remapping every chunk that has to be changed. Far easier to just change all
of the chunk links to point to the new devices and poof!
Oh, Linux still supports RAW devices, but Linus seriously wants to get rid
of them. It's an ongoing battle IB. Linux COOKED chunks with O_DIRECT
which is supported in IDS 10.00 and later, is only about 5% slower than RAW
anyway, so it's not a big deal.
Art
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors (art@iiug.org)
Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do
those opinions reflect those of other individuals affiliated with any entity
with which I am affiliated nor those of the entities themselves.
On Wed, Jun 3, 2009 at 9:08 AM, MARK NEWNAM
<mnewnam@carolinahandling.com>wrote:
> We are running on RedHat Linux.
>
> I was also looking into using a raw device but I see that raw devices are
> no
> longer supported in Linux. I have read that IDS will allow you to use an
> unmounted disk for a space. I was going to try a test this way to see how
> performance was. Do I use an entire disk (/dev/sdb) or just a partition
> (/dev/sdb1)? What is the onspace command to create a space this way?
>
> Thanks,
> Mark
>
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>
>
--00504502b19493cdda046b71d050
Unmounted space would be considered raw - which, since you are on Linux is not
what you want. You want to use a mounted file system to create your dbspaces
and chunks. If for example you had a file system called /dev/sdb and it was
100 gig you could use the 'touch' command to create a bunch of files that
would be used for chunks - and optionally create links to the files.
So let's say you have mounted space and called it /dev/sdb, and the sdb
directory has 100 gig. You can create, using the touch command 50 files that
will all end up being your chunks. Using symbolic links in this example, get
ya a little script that creates the 50 files, changes their permissions
accordingly, and links them to symbolic links - this is a ksh example:
#!/usr/bin/ksh
#script to create 50 files and links to be used
#for 2 gig Informix chunks.
i=1
while ((i <=50));
do
touch /dev/sdb/c$i;
chown informix /dev/sdb/c$i
chgrp informix dev/sdb/c$i
chmod 660 /dev/sdb/c$i
ln -s /dev/sdb/c$i /informix/dev/chunk$i
((i=i+1));
done
There's an easier way to do the permissions thing but this works. Once you
have your 50 files you can use onspaces to create your chunks. Some folks
script this also - and you may feel comfie doing that. The onspaces cmd to
create a dbspace is:
onspaces -c -d dbspace_name -p /informix/dev/chunk# -o 0 -s size
Look closely at the documentation for onspaces before your run anything. Look
closely at the doc anyways.
MM
For those on version 11 who desire to use cooked chunks you can use
the sql admin api and you do not have to create the files a head of time.
Here are a few examples, for more examples see
$INFORMIXDIR/demo/sysadmin/system_setup.sql
===================EXAMPLE #1==========================
dbaccess sysadmin -
execute function task( "create dbspace", "dbspace1","/work/CHUNKS/dbspace1", 0, "2 GB");
Another way to create a set of dbspaces and chunks are the following:
===================EXAMPLE #2==========================
database sysadmin;
{ **** Create a table of dbspaces which are to be created ****}
create table dbspaces
(
type varchar(255),
dbspace varchar(255),
path varchar(255),
offset varchar(255),
size varchar(255),
);
insert into dbspaces values
("sbspace", "sbspace", "$INFORMIXDIR/CHUNKS/sblob1", 0 , "50 MB" );
insert into dbspaces values
("dbspace", "dbspace1", "$INFORMIXDIR/CHUNKS/dbspace1", 0 , "50 MB" );
insert into dbspaces values
("dbspace", "dbspace2", "$INFORMIXDIR/CHUNKS/dbspace2", 0 , "50 MB" );
insert into dbspaces values
("dbspace", "physdbs", "$INFORMIXDIR/CHUNKS/physdbs", 0 , "50 MB" ););
insert into dbspaces values
("dbspace", "logdbs", "$INFORMIXDIR/CHUNKS/logdbs", 0 , "50 MB" );););
insert into dbspaces values
("tempdbspace", "tempdbs", "$INFORMIXDIR/CHUNKS/tempdbs", 0 , "10 MB" );
insert into dbspaces values
("blobspace", "bspace1", "$INFORMIXDIR/CHUNKS/blobdbs", 0 , "50 MB" ););
{ **** Create a table of chunks which are to be created **** }
create table chunks
(
dbspace varchar(255),
path varchar(255),
offset varchar(255),
size varchar(255),
);
insert into chunks values
("dbspace1", "$INFORMIXDIR/CHUNKS/chunk",0 , "10 MB" );
insert into chunks values
("dbspace1", "$INFORMIXDIR/CHUNKS/chunk2",0 , "10 MB" );
{**** Create all the dbspaces ****}
SELECT task( "create "|| type , dbspace, path, size, offset)
FROM dbspaces;
{**** Add the chunks to the dbspaces ****}
select task("add chunk", dbspace, path, size, offset) from chunks;
John F. Miller III
STSM, Support Architect
miller3@us.ibm.com
503-578-5645
IBM Informix Dynamic Server (IDS)
ids-bounces@iiug.org wrote on 06/03/2009 10:34:14 AM:
> [image removed]
>
> Re: IDS 11 - Slowness with DirectIO [15928]
>
> MIKE MAGIE
>
> to:
>
> ids
>
> 06/03/2009 10:35 AM
>
> Sent by:
>
> ids-bounces@iiug.org
>
> Please respond to ids
>
> Unmounted space would be considered raw - which, since you are on
> Linux is not
> what you want. You want to use a mounted file system to create your
dbspaces
> and chunks. If for example you had a file system called /dev/sdb and it
was
> 100 gig you could use the 'touch' command to create a bunch of files that
> would be used for chunks - and optionally create links to the files.
>
> So let's say you have mounted space and called it /dev/sdb, and the sdb
> directory has 100 gig. You can create, using the touch command 50 files
that
> will all end up being your chunks. Using symbolic links in this example,
get
> ya a little script that creates the 50 files, changes their permissions
> accordingly, and links them to symbolic links - this is a ksh example:
>
> #!/usr/bin/ksh
>
> #script to create 50 files and links to be used
> #for 2 gig Informix chunks.
>
> i=1
>
> while ((i <=50));
> do
>
> touch /dev/sdb/c$i;
>
> chown informix /dev/sdb/c$i
>
> chgrp informix dev/sdb/c$i
>
> chmod 660 /dev/sdb/c$i
>
> ln -s /dev/sdb/c$i /informix/dev/chunk$i
>
> ((i=i+1));
> done
>
> There's an easier way to do the permissions thing but this works. Once
you
> have your 50 files you can use onspaces to create your chunks. Some folks
> script this also - and you may feel comfie doing that. The onspaces cmd
to
> create a dbspace is:
>
> onspaces -c -d dbspace_name -p /informix/dev/chunk# -o 0 -s size>
> Look closely at the documentation for onspaces before your run anything.
Look
> closely at the doc anyways.
>
> MM
>
>
>
*******************************************************************************
> Forum Note: Use "Reply" to post a response in the discussion forum.
>