ONPLADM , IPLOAD & ONPLOAD scripts
Does anyone already have scripts to make up the files to make up the files
to input to onpladm that would create the backup jobs/devices/etc.... for
onpladm?
I wish to backup with onpload all of my 126 tables with a single command. I
can make up the jobs with the GUI and then run them at the time of
conversion but there is a fair amount of typing.
Thanks for your help. Onpload really goes FAST! Amazing fast!
Tim Ertl
413-442-9000 x6211
Tim,
I have such a script. It is not intended to cover all situations, just those
our group has run into. It serves our needs. You might be able to use it as is
or you might be able to start with it and then modify going forward. The
script is named hpl_create.ksh and it uses a text file named hpl_create.list.
The hpl_create.list file has one line per table. Each line contains the name
of the table, the number of devices (unload files), the letter e or d
indicating express or deluxe mode, and the root name of the job and device(s).
The job name will be the root name with "_job" tacked on the end and the
device name will be the root name with "_dvc" tacked on the end.
For example,
mytab 12 e mytab
Will create both a load and an unload job named mytab_job with 12 unload files
defined by device name mytab_dvc. The jobs will use express mode.
There are restrictions on using express mode (row size, extended data types,
etc.) and the script performs a few rudimentary checks and may change an
express specification to deluxe if needed.
The script expects you to pass the instance name, the database name, and the
path for the location of your unload files (device path). With all of our
scripts, we source a script that sets our environment for a particular
instance. Also, what we call an instance is essentially the extent name of the
onconfig file and we use it for many things to identify an instance. It is
different (usually shorter) than the INFORMIXSERVER value.
The first time a job is created for a table with this script, you will see
several errors related to the "delete job" lines in the script. As I say, this
is a somewhat crude script that meets our needs and it has never been worth
our time to make it squeaky clean.
This script was not written to meet anyone's needs but our own; however, you
may find some useful bits in it. (caveat lector)
Here is the text of the script:
--------------------------------------------------------------------------------
---------
#!/usr/bin/ksh
#-------------------------------------------------------------------------------
# hpl_create.ksh
#
# Notes on job creation options:
# -flu means create both an unload job and a load job
# adding an "a" means treat data source as a device array
# adding a "c" means deluxe mode
# adding an "N" means it is a deluxe mode without replication
# adding an "n" means no-conversion (this is fastest, must be express)
# Without the "c" option, the mode will default to express
#-------------------------------------------------------------------------------
if [ "$#" -lt 3 ]
then
echo "USAGE: `basename $0` instance database device_path"
exit 1
else
export INST=$1
export DB=$2
export DEVICE_PATH=$3
fi
. ~informix/infx_env $INST
#-------------------------------------------------------------------------------
# Create the jobs
#-------------------------------------------------------------------------------
cat hpl_create.list | tr '{A-Z}' '{a-z}' \\\\
| while read TAB NUM_DEVICES MODE JOBNAME_ROOT
do
echo " unload to hpl_create.tabinfo delimiter ' '
select rowsize
from systables
where tabname = '${TAB}'" \\\\
| dbaccess $DB > /dev/null 2>&1
if [ `cat hpl_create.tabinfo | wc -l` -eq 0 ]
then
echo "Table $TAB does not exist in $DB"
else
export ROWSIZE=`cat hpl_create.tabinfo`
export PAGESIZE=`onstat -b|grep "buffer size"|awk '{print ($(NF-2))}'`
if [ "$ROWSIZE" -ge "$PAGESIZE" -a "$MODE" = "e" ]
then
echo "Changing the mode to Deluxe for $TAB because row size exceeds page size."
export MODE="d"
fi
export JOBNAME=${JOBNAME_ROOT}_job
export DVCNAME=${JOBNAME_ROOT}_dvc
onpladm delete device $DVCNAME
onpladm delete job $JOBNAME -fl -R
onpladm delete job $JOBNAME -fu -R
echo "BEGIN OBJECT DEVICEARRAY ${DVCNAME}" > ${TAB}_device_specfile
while [ "$NUM_DEVICES" -gt 0 ]
do
echo "BEGIN SEQUENCE" >> ${TAB}_device_specfile
echo "TYPE FILE" >> ${TAB}_device_specfile
echo "FILE $DEVICE_PATH/${TAB}.unl${NUM_DEVICES}" >> ${TAB}_device_specfile
echo "TAPEBLOCKSIZE 0" >> ${TAB}_device_specfile
echo "TAPEDEVICESIZE 0" >> ${TAB}_device_specfile
echo "PIPECOMMAND" >> ${TAB}_device_specfile
echo "END SEQUENCE" >> ${TAB}_device_specfile
let NUM_DEVICES=NUM_DEVICES-1
done
echo "END OBJECT" >> ${TAB}_device_specfile
onpladm create object -F ${TAB}_device_specfile
if [ "$MODE" = "d" ]
then
onpladm create job $JOBNAME -d ${DVCNAME} -fluaN -D $DB -t $TAB
else
onpladm create job $JOBNAME -d ${DVCNAME} -flua -n -D $DB -t $TAB
fi
fi
done
#-------------------------------------------------------------------------------
# cleanup the files
#-------------------------------------------------------------------------------
rm -f hpl_create.tabinfo
rm -r *_device_specfile
--------------------------------------------------------------------------------
---------
Rob Schmitz
Embarq Data Management
rob.b.schmitz@embarq.com
www.embarq.com
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Tim Ertl
Sent: Thursday, July 24, 2008 4:39 PM
To: ids@iiug.org
Subject: ONPLADM , IPLOAD & ONPLOAD scripts [12892]
Does anyone already have scripts to make up the files to make up the files
to input to onpladm that would create the backup jobs/devices/etc.... for
onpladm?
I wish to backup with onpload all of my 126 tables with a single command. I
can make up the jobs with the GUI and then run them at the time of
conversion but there is a fair amount of typing.
Thanks for your help. Onpload really goes FAST! Amazing fast!
Tim Ertl
413-442-9000 x6211
*******************************************************************************
Forum Note: Use "Reply" to post a response in the discussion forum.
Thanks Rob, I like this idea. I will give it a try!
,,,Tim
Tim Ertl
413-442-9000 x6211
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
Schmitz, Rob B [EQ]
Sent: Friday, July 25, 2008 12:20 PM
To: ids@iiug.org
Subject: RE: ONPLADM , IPLOAD & ONPLOAD scripts [12904]
Tim,
I have such a script. It is not intended to cover all situations, just those
our group has run into. It serves our needs. You might be able to use it as
is
or you might be able to start with it and then modify going forward. The
script is named hpl_create.ksh and it uses a text file named
hpl_create.list.
The hpl_create.list file has one line per table. Each line contains the name
of the table, the number of devices (unload files), the letter e or d
indicating express or deluxe mode, and the root name of the job and
device(s).
The job name will be the root name with "_job" tacked on the end and the
device name will be the root name with "_dvc" tacked on the end.
For example,
mytab 12 e mytab
Will create both a load and an unload job named mytab_job with 12 unload
files
defined by device name mytab_dvc. The jobs will use express mode.
There are restrictions on using express mode (row size, extended data types,
etc.) and the script performs a few rudimentary checks and may change an
express specification to deluxe if needed.
The script expects you to pass the instance name, the database name, and the
path for the location of your unload files (device path). With all of our
scripts, we source a script that sets our environment for a particular
instance. Also, what we call an instance is essentially the extent name of
the
onconfig file and we use it for many things to identify an instance. It is
different (usually shorter) than the INFORMIXSERVER value.
The first time a job is created for a table with this script, you will see
several errors related to the "delete job" lines in the script. As I say,
this
is a somewhat crude script that meets our needs and it has never been worth
our time to make it squeaky clean.
This script was not written to meet anyone's needs but our own; however, you
may find some useful bits in it. (caveat lector)
Here is the text of the script:
----------------------------------------------------------------------------
-------------
#!/usr/bin/ksh
#---------------------------------------------------------------------------
----
# hpl_create.ksh
#
# Notes on job creation options:
# -flu means create both an unload job and a load job
# adding an "a" means treat data source as a device array
# adding a "c" means deluxe mode
# adding an "N" means it is a deluxe mode without replication
# adding an "n" means no-conversion (this is fastest, must be express)
# Without the "c" option, the mode will default to express
#---------------------------------------------------------------------------
----
if [ "$#" -lt 3 ]
then
echo "USAGE: `basename $0` instance database device_path"
exit 1
else
export INST=$1
export DB=$2
export DEVICE_PATH=$3
fi
.. ~informix/infx_env $INST
#---------------------------------------------------------------------------
----
# Create the jobs
#---------------------------------------------------------------------------
----
cat hpl_create.list | tr '{A-Z}' '{a-z}' \\\\
| while read TAB NUM_DEVICES MODE JOBNAME_ROOT
do
echo " unload to hpl_create.tabinfo delimiter ' '
select rowsize
from systables
where tabname = '${TAB}'" \\\\
| dbaccess $DB > /dev/null 2>&1
if [ `cat hpl_create.tabinfo | wc -l` -eq 0 ]
then
echo "Table $TAB does not exist in $DB"
else
export ROWSIZE=`cat hpl_create.tabinfo`
export PAGESIZE=`onstat -b|grep "buffer size"|awk '{print ($(NF-2))}'`
if [ "$ROWSIZE" -ge "$PAGESIZE" -a "$MODE" = "e" ]
then
echo "Changing the mode to Deluxe for $TAB because row size exceeds page
size."
export MODE="d"
fi
export JOBNAME=${JOBNAME_ROOT}_job
export DVCNAME=${JOBNAME_ROOT}_dvc
onpladm delete device $DVCNAME
onpladm delete job $JOBNAME -fl -R
onpladm delete job $JOBNAME -fu -R
echo "BEGIN OBJECT DEVICEARRAY ${DVCNAME}" > ${TAB}_device_specfile
while [ "$NUM_DEVICES" -gt 0 ]
do
echo "BEGIN SEQUENCE" >> ${TAB}_device_specfile
echo "TYPE FILE" >> ${TAB}_device_specfile
echo "FILE $DEVICE_PATH/${TAB}.unl${NUM_DEVICES}" >> ${TAB}_device_specfile
echo "TAPEBLOCKSIZE 0" >> ${TAB}_device_specfile
echo "TAPEDEVICESIZE 0" >> ${TAB}_device_specfile
echo "PIPECOMMAND" >> ${TAB}_device_specfile
echo "END SEQUENCE" >> ${TAB}_device_specfile
let NUM_DEVICES=NUM_DEVICES-1
done
echo "END OBJECT" >> ${TAB}_device_specfile
onpladm create object -F ${TAB}_device_specfile
if [ "$MODE" = "d" ]
then
onpladm create job $JOBNAME -d ${DVCNAME} -fluaN -D $DB -t $TAB
else
onpladm create job $JOBNAME -d ${DVCNAME} -flua -n -D $DB -t $TAB
fi
fi
done
#---------------------------------------------------------------------------
----
# cleanup the files
#---------------------------------------------------------------------------
----
rm -f hpl_create.tabinfo
rm -r *_device_specfile
----------------------------------------------------------------------------
-------------
Rob Schmitz
Embarq Data Management
rob.b.schmitz@embarq.com
www.embarq.com
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Tim
Ertl
Sent: Thursday, July 24, 2008 4:39 PM
To: ids@iiug.org
Subject: ONPLADM , IPLOAD & ONPLOAD scripts [12892]
Does anyone already have scripts to make up the files to make up the files
to input to onpladm that would create the backup jobs/devices/etc.... for
onpladm?
I wish to backup with onpload all of my 126 tables with a single command. I
can make up the jobs with the GUI and then run them at the time of
conversion but there is a fair amount of typing.
Thanks for your help. Onpload really goes FAST! Amazing fast!
Tim Ertl
413-442-9000 x6211
****************************************************************************
***
Forum Note: Use "Reply" to post a response in the discussion forum.
****************************************************************************
***
Forum Note: Use "Reply" to post a response in the discussion forum.
Rob, That is fantastic! Exactly what I wanted!!!! Wow! You made it look so
simple. The only thing that was a problem is that on IDS11 the onstat -b
puts out two buffer sizes (in my case). I just overrode it for the smaller
of the two rather than use the onstat -b. In MY case it works great!
Again WOW! Thanks and I will owe you a beer if we ever meet up!
Tim Ertl
413-442-9000 x6211
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
Schmitz, Rob B [EQ]
Sent: Friday, July 25, 2008 12:20 PM
To: ids@iiug.org
Subject: RE: ONPLADM , IPLOAD & ONPLOAD scripts [12904]
Tim,
I have such a script. It is not intended to cover all situations, just those
our group has run into. It serves our needs. You might be able to use it as
is
or you might be able to start with it and then modify going forward. The
script is named hpl_create.ksh and it uses a text file named
hpl_create.list.
The hpl_create.list file has one line per table. Each line contains the name
of the table, the number of devices (unload files), the letter e or d
indicating express or deluxe mode, and the root name of the job and
device(s).
The job name will be the root name with "_job" tacked on the end and the
device name will be the root name with "_dvc" tacked on the end.
For example,
mytab 12 e mytab
Will create both a load and an unload job named mytab_job with 12 unload
files
defined by device name mytab_dvc. The jobs will use express mode.
There are restrictions on using express mode (row size, extended data types,
etc.) and the script performs a few rudimentary checks and may change an
express specification to deluxe if needed.
The script expects you to pass the instance name, the database name, and the
path for the location of your unload files (device path). With all of our
scripts, we source a script that sets our environment for a particular
instance. Also, what we call an instance is essentially the extent name of
the
onconfig file and we use it for many things to identify an instance. It is
different (usually shorter) than the INFORMIXSERVER value.
The first time a job is created for a table with this script, you will see
several errors related to the "delete job" lines in the script. As I say,
this
is a somewhat crude script that meets our needs and it has never been worth
our time to make it squeaky clean.
This script was not written to meet anyone's needs but our own; however, you
may find some useful bits in it. (caveat lector)
Here is the text of the script:
----------------------------------------------------------------------------
-------------
#!/usr/bin/ksh
#---------------------------------------------------------------------------
----
# hpl_create.ksh
#
# Notes on job creation options:
# -flu means create both an unload job and a load job
# adding an "a" means treat data source as a device array
# adding a "c" means deluxe mode
# adding an "N" means it is a deluxe mode without replication
# adding an "n" means no-conversion (this is fastest, must be express)
# Without the "c" option, the mode will default to express
#---------------------------------------------------------------------------
----
if [ "$#" -lt 3 ]
then
echo "USAGE: `basename $0` instance database device_path"
exit 1
else
export INST=$1
export DB=$2
export DEVICE_PATH=$3
fi
.. ~informix/infx_env $INST
#---------------------------------------------------------------------------
----
# Create the jobs
#---------------------------------------------------------------------------
----
cat hpl_create.list | tr '{A-Z}' '{a-z}' \\\\
| while read TAB NUM_DEVICES MODE JOBNAME_ROOT
do
echo " unload to hpl_create.tabinfo delimiter ' '
select rowsize
from systables
where tabname = '${TAB}'" \\\\
| dbaccess $DB > /dev/null 2>&1
if [ `cat hpl_create.tabinfo | wc -l` -eq 0 ]
then
echo "Table $TAB does not exist in $DB"
else
export ROWSIZE=`cat hpl_create.tabinfo`
export PAGESIZE=`onstat -b|grep "buffer size"|awk '{print ($(NF-2))}'`
if [ "$ROWSIZE" -ge "$PAGESIZE" -a "$MODE" = "e" ]
then
echo "Changing the mode to Deluxe for $TAB because row size exceeds page
size."
export MODE="d"
fi
export JOBNAME=${JOBNAME_ROOT}_job
export DVCNAME=${JOBNAME_ROOT}_dvc
onpladm delete device $DVCNAME
onpladm delete job $JOBNAME -fl -R
onpladm delete job $JOBNAME -fu -R
echo "BEGIN OBJECT DEVICEARRAY ${DVCNAME}" > ${TAB}_device_specfile
while [ "$NUM_DEVICES" -gt 0 ]
do
echo "BEGIN SEQUENCE" >> ${TAB}_device_specfile
echo "TYPE FILE" >> ${TAB}_device_specfile
echo "FILE $DEVICE_PATH/${TAB}.unl${NUM_DEVICES}" >> ${TAB}_device_specfile
echo "TAPEBLOCKSIZE 0" >> ${TAB}_device_specfile
echo "TAPEDEVICESIZE 0" >> ${TAB}_device_specfile
echo "PIPECOMMAND" >> ${TAB}_device_specfile
echo "END SEQUENCE" >> ${TAB}_device_specfile
let NUM_DEVICES=NUM_DEVICES-1
done
echo "END OBJECT" >> ${TAB}_device_specfile
onpladm create object -F ${TAB}_device_specfile
if [ "$MODE" = "d" ]
then
onpladm create job $JOBNAME -d ${DVCNAME} -fluaN -D $DB -t $TAB
else
onpladm create job $JOBNAME -d ${DVCNAME} -flua -n -D $DB -t $TAB
fi
fi
done
#---------------------------------------------------------------------------
----
# cleanup the files
#---------------------------------------------------------------------------
----
rm -f hpl_create.tabinfo
rm -r *_device_specfile
----------------------------------------------------------------------------
-------------
Rob Schmitz
Embarq Data Management
rob.b.schmitz@embarq.com
www.embarq.com
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Tim
Ertl
Sent: Thursday, July 24, 2008 4:39 PM
To: ids@iiug.org
Subject: ONPLADM , IPLOAD & ONPLOAD scripts [12892]
Does anyone already have scripts to make up the files to make up the files
to input to onpladm that would create the backup jobs/devices/etc.... for
onpladm?
I wish to backup with onpload all of my 126 tables with a single command. I
can make up the jobs with the GUI and then run them at the time of
conversion but there is a fair amount of typing.
Thanks for your help. Onpload really goes FAST! Amazing fast!
Tim Ertl
413-442-9000 x6211
****************************************************************************
***
Forum Note: Use "Reply" to post a response in the discussion forum.
*******************
Bob, I spent the day today using your utility to perform our conversion in 5
hours TOTAL without removing any data before hand. My previous attempt was
estimated to be over 32hours. This was GREAT! Many Many thanks for your
help!
Tim Ertl
413-442-9000 x6211
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of
Schmitz, Rob B [EQ]
Sent: Friday, July 25, 2008 12:20 PM
To: ids@iiug.org
Subject: RE: ONPLADM , IPLOAD & ONPLOAD scripts [12904]
Tim,
I have such a script. It is not intended to cover all situations, just those
our group has run into. It serves our needs. You might be able to use it as
is
or you might be able to start with it and then modify going forward. The
script is named hpl_create.ksh and it uses a text file named
hpl_create.list.
The hpl_create.list file has one line per table. Each line contains the name
of the table, the number of devices (unload files), the letter e or d
indicating express or deluxe mode, and the root name of the job and
device(s).
The job name will be the root name with "_job" tacked on the end and the
device name will be the root name with "_dvc" tacked on the end.
For example,
mytab 12 e mytab
Will create both a load and an unload job named mytab_job with 12 unload
files
defined by device name mytab_dvc. The jobs will use express mode.
There are restrictions on using express mode (row size, extended data types,
etc.) and the script performs a few rudimentary checks and may change an
express specification to deluxe if needed.
The script expects you to pass the instance name, the database name, and the
path for the location of your unload files (device path). With all of our
scripts, we source a script that sets our environment for a particular
instance. Also, what we call an instance is essentially the extent name of
the
onconfig file and we use it for many things to identify an instance. It is
different (usually shorter) than the INFORMIXSERVER value.
The first time a job is created for a table with this script, you will see
several errors related to the "delete job" lines in the script. As I say,
this
is a somewhat crude script that meets our needs and it has never been worth
our time to make it squeaky clean.
This script was not written to meet anyone's needs but our own; however, you
may find some useful bits in it. (caveat lector)
Here is the text of the script:
----------------------------------------------------------------------------
-------------
#!/usr/bin/ksh
#---------------------------------------------------------------------------
----
# hpl_create.ksh
#
# Notes on job creation options:
# -flu means create both an unload job and a load job
# adding an "a" means treat data source as a device array
# adding a "c" means deluxe mode
# adding an "N" means it is a deluxe mode without replication
# adding an "n" means no-conversion (this is fastest, must be express)
# Without the "c" option, the mode will default to express
#---------------------------------------------------------------------------
----
if [ "$#" -lt 3 ]
then
echo "USAGE: `basename $0` instance database device_path"
exit 1
else
export INST=$1
export DB=$2
export DEVICE_PATH=$3
fi
.. ~informix/infx_env $INST
#---------------------------------------------------------------------------
----
# Create the jobs
#---------------------------------------------------------------------------
----
cat hpl_create.list | tr '{A-Z}' '{a-z}' \\\\
| while read TAB NUM_DEVICES MODE JOBNAME_ROOT
do
echo " unload to hpl_create.tabinfo delimiter ' '
select rowsize
from systables
where tabname = '${TAB}'" \\\\
| dbaccess $DB > /dev/null 2>&1
if [ `cat hpl_create.tabinfo | wc -l` -eq 0 ]
then
echo "Table $TAB does not exist in $DB"
else
export ROWSIZE=`cat hpl_create.tabinfo`
export PAGESIZE=`onstat -b|grep "buffer size"|awk '{print ($(NF-2))}'`
if [ "$ROWSIZE" -ge "$PAGESIZE" -a "$MODE" = "e" ]
then
echo "Changing the mode to Deluxe for $TAB because row size exceeds page
size."
export MODE="d"
fi
export JOBNAME=${JOBNAME_ROOT}_job
export DVCNAME=${JOBNAME_ROOT}_dvc
onpladm delete device $DVCNAME
onpladm delete job $JOBNAME -fl -R
onpladm delete job $JOBNAME -fu -R
echo "BEGIN OBJECT DEVICEARRAY ${DVCNAME}" > ${TAB}_device_specfile
while [ "$NUM_DEVICES" -gt 0 ]
do
echo "BEGIN SEQUENCE" >> ${TAB}_device_specfile
echo "TYPE FILE" >> ${TAB}_device_specfile
echo "FILE $DEVICE_PATH/${TAB}.unl${NUM_DEVICES}" >> ${TAB}_device_specfile
echo "TAPEBLOCKSIZE 0" >> ${TAB}_device_specfile
echo "TAPEDEVICESIZE 0" >> ${TAB}_device_specfile
echo "PIPECOMMAND" >> ${TAB}_device_specfile
echo "END SEQUENCE" >> ${TAB}_device_specfile
let NUM_DEVICES=NUM_DEVICES-1
done
echo "END OBJECT" >> ${TAB}_device_specfile
onpladm create object -F ${TAB}_device_specfile
if [ "$MODE" = "d" ]
then
onpladm create job $JOBNAME -d ${DVCNAME} -fluaN -D $DB -t $TAB
else
onpladm create job $JOBNAME -d ${DVCNAME} -flua -n -D $DB -t $TAB
fi
fi
done
#---------------------------------------------------------------------------
----
# cleanup the files
#---------------------------------------------------------------------------
----
rm -f hpl_create.tabinfo
rm -r *_device_specfile
----------------------------------------------------------------------------
-------------
Rob Schmitz
Embarq Data Management
rob.b.schmitz@embarq.com
www.embarq.com
-----Original Message-----
From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On Behalf Of Tim
Ertl
Sent: Thursday, July 24, 2008 4:39 PM
To: ids@iiug.org
Subject: ONPLADM , IPLOAD & ONPLOAD scripts [12892]
Does anyone already have scripts to make up the files to make up the files
to input to onpladm that would create the backup jobs/devices/etc.... for
onpladm?
I wish to backup with onpload all of my 126 tables with a single command. I
can make up the jobs with the GUI and then run them at the time of
conversion but there is a fair amount of typing.
Thanks for your help. Onpload really goes FAST! Amazing fast!
Tim Ertl
413-442-9000 x6211
****************************************************************************
***
Forum Note: Use "Reply" to post a response in the discussion forum.
****************************************************************************
***
Forum Note: Use "Reply" to post a response in the discu