Quick help - I hope
Posted in 2008
A DBA on IDS 10 / Solaris 8 wanted a ksh script to total disk space used under /opt/informix and all its subdirectories, having been doing it tediously with 'ls -l | awk'. Replies immediately pointed out the obvious answer: just use 'du' (du -s for a single total, du -k for KB, piped through 'sort -rn | pg' to find the biggest consumers). Alternatives offered included find/ls/awk combinations (with caveats about block sizes, double-counting and hard links) and a custom C program. The poster accepted du and considered the problem solved.
Auto-generated by DrWatson from the posts below — may be imperfect; read the full thread.
Topics: Versions, Editions & End-of-Life
IDS 10.00.UC8 Solaris 2.8 I need some help with a ksh script. I need to determine how much file system space the db team is using on one of our Unix boxes. There are buttloads of directories I need to look at - I had started doing this but it is tedious: ls -l | awk '{ total += $5 } END {print "column total is", total}' I need the part of the script that will start an ls at the top of a given directory and works its way down, total the /opt/informix directory, now total any dirctories under opt/informix, and any under that... make sense? I am also working on this but if there are any homies out there that already have this please hook a brother up. MM
Just run du /opt/informix The bottom number will be the size of that directory and all subdirectories. Lennie On Wed, Nov 19, 2008 at 9:28 AM, MIKE MAGIE <jmmagie@yahoo.com> wrote: > IDS 10.00.UC8 > Solaris 2.8 > > I need some help with a ksh script. I need to determine how much file > system > space the db team is using on one of our Unix boxes. There are buttloads of > directories I need to look at - I had started doing this but it is tedious: > > ls -l | awk '{ total += $5 } END {print "column total is", total}' > > I need the part of the script that will start an ls at the top of a given > directory and works its way down, total the /opt/informix directory, now > total > any dirctories under opt/informix, and any under that... make sense? I am > also > working on this but if there are any homies out there that already have > this > please hook a brother up. > > MM > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > >
2008/11/19 Lennie Jarratt <lennie@jarratt.ws>: > Just run > > du /opt/informix > > The bottom number will be the size of that directory and all subdirectories. > > Lennie > > On Wed, Nov 19, 2008 at 9:28 AM, MIKE MAGIE <jmmagie@yahoo.com> wrote: > >> IDS 10.00.UC8 >> Solaris 2.8 >> >> I need some help with a ksh script. I need to determine how much file >> system >> space the db team is using on one of our Unix boxes. There are buttloads of >> directories I need to look at - I had started doing this but it is tedious: >> >> ls -l | awk '{ total += $5 } END {print "column total is", total}' >> >> I need the part of the script that will start an ls at the top of a given >> directory and works its way down, total the /opt/informix directory, now >> total >> any dirctories under opt/informix, and any under that... make sense? I am >> also >> working on this but if there are any homies out there that already have >> this >> please hook a brother up. >> >> MM >> Pipe it through sort -rn and then pg or more (or less) and get the high hitters at the top:- du /opt/informix |sort -rn |pg output is in 1/2 Kb blocks Keith
Keith Simmons wrote: > > output is in 1/2 Kb blocks Unless you use du -k ... :o) -- Cheers, Obnoxio The Clown http://obotheclown.blogspot.com
Here's a 'c' program that I wrote a while back that does this... The -d does sizing by directory. The -l says how far down the director= y tree to go. By default, it finds the largest files below your current directory. (See attached file: big.c) ------------------------------------- Madison Pruet, STSM IDS Replication Architect = "MIKE MAGIE" = <jmmagie@yahoo.co = m> = To Sent by: ids@iiug.org = ids-bounces@iiug. = cc org = Subj= ect Quick help - I hope [14046] = 11/19/2008 09:28 = AM = = = Please respond to = ids@iiug.org = = = IDS 10.00.UC8 Solaris 2.8 I need some help with a ksh script. I need to determine how much file system space the db team is using on one of our Unix boxes. There are buttload= s of directories I need to look at - I had started doing this but it is tedi= ous: ls -l | awk '{ total +=3D $5 } END {print "column total is", total}' I need the part of the script that will start an ls at the top of a giv= en directory and works its way down, total the /opt/informix directory, no= w total any dirctories under opt/informix, and any under that... make sense? I = am also working on this but if there are any homies out there that already have= this please hook a brother up. MM ***********************************************************************= ******** Forum Note: Use "Reply" to post a response in the discussion forum. =
duh forgot about du - thanks. I'm good... although I like threads like this - would love to see more scripters flex. Flex on. MM
If you only care about space and not content, the du command may work better. du reports the amount of file space used by the files indicated by the given pathnames. If pathname is a directory, du reports the total amount of file space used by all files in that directory and in each subdirectory in its hierarchy. If you do not specify any pathname, du assumes the current directory. Files with multiple links are only counted once. Dick Snoke IBM Data Management - ChannelWorks dsnoke@us.ibm.com (404) 487-1595 From: "MIKE MAGIE" <jmmagie@yahoo.com> To: ids@iiug.org Date: 11/19/2008 10:32 AM Subject: Quick help - I hope [14046] IDS 10.00.UC8 Solaris 2.8 I need some help with a ksh script. I need to determine how much file system space the db team is using on one of our Unix boxes. There are buttloads of directories I need to look at - I had started doing this but it is tedious: ls -l | awk '{ total += $5 } END {print "column total is", total}' I need the part of the script that will start an ls at the top of a given directory and works its way down, total the /opt/informix directory, now total any dirctories under opt/informix, and any under that... make sense? I am also working on this but if there are any homies out there that already have this please hook a brother up. MM ******************************************************************************* Forum Note: Use "Reply" to post a response in the discussion forum.
Almost there: find /opt/informix -exec ls -l {} \\\\; | awk 'BEGIN {total = 0;}{ total += $5 } END {print "column total is", total}' Should do the trick. Of course you could just: du -s /opt/informix This reports in 'blocks'. The block size differs on different platforms - don't remember what Solaris uses. The answer will be the same from both runs (though du will tend to be faster) except that the find uses actual file sizes which ignores the waste between EOF and the end of the physical disk blocks. You can use ls to take account of this with the -s option and use field $1 instead and multiply by the block size (which you can back out of any ls -ls listing. This works on my Linux system where the block size is 1K: find /opt/informix -exec ls -ls {} \\\\; | awk 'BEGIN {total = 0;}{ total += $1 * 1024 } END {print "column total is", total}' Returning the more accurate 1484152832 where the first query returns 1463203965 a difference of 21MB. Art On Wed, Nov 19, 2008 at 10:28 AM, MIKE MAGIE <jmmagie@yahoo.com> wrote: > IDS 10.00.UC8 > Solaris 2.8 > > I need some help with a ksh script. I need to determine how much file > system > space the db team is using on one of our Unix boxes. There are buttloads of > directories I need to look at - I had started doing this but it is tedious: > > ls -l | awk '{ total += $5 } END {print "column total is", total}' > > I need the part of the script that will start an ls at the top of a given > directory and works its way down, total the /opt/informix directory, now > total > any dirctories under opt/informix, and any under that... make sense? I am > also > working on this but if there are any homies out there that already have > this > please hook a brother up. > > MM > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Art S. Kagel Oninit (www.oninit.com) IIUG Board of Directors (art@iiug.org) Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Oninit, the IIUG, nor any other organization with which I am associated either explicitly or implicitly. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.
Hmm, for some reason 'find' on my machine is listing every file twice. Once with a full path and once without one, so the number's it's producing are doubled. Well, as others have stated also, the du is accurate and faster. Art On Wed, Nov 19, 2008 at 11:27 AM, Art Kagel <art.kagel@gmail.com> wrote: > Almost there: > > find /opt/informix -exec ls -l {} \\\\; | awk 'BEGIN {total = 0;}{ total += $5 > } END {print "column total is", total}' > > Should do the trick. Of course you could just: > > du -s /opt/informix > > This reports in 'blocks'. The block size differs on different platforms - > don't remember what Solaris uses. The answer will be the same from both > runs (though du will tend to be faster) except that the find uses actual > file sizes which ignores the waste between EOF and the end of the physical > disk blocks. > > You can use ls to take account of this with the -s option and use field $1 > instead and multiply by the block size (which you can back out of any ls > -ls > listing. This works on my Linux system where the block size is 1K: > > find /opt/informix -exec ls -ls {} \\\\; | awk 'BEGIN {total = 0;}{ total += > $1 > * 1024 } END {print "column total is", total}' > > Returning the more accurate 1484152832 where the first query returns > 1463203965 a difference of 21MB. > > Art > > On Wed, Nov 19, 2008 at 10:28 AM, MIKE MAGIE <jmmagie@yahoo.com> wrote: > > > IDS 10.00.UC8 > > Solaris 2.8 > > > > I need some help with a ksh script. I need to determine how much file > > system > > space the db team is using on one of our Unix boxes. There are buttloads > of > > directories I need to look at - I had started doing this but it is > tedious: > > > > ls -l | awk '{ total += $5 } END {print "column total is", total}' > > > > I need the part of the script that will start an ls at the top of a given > > directory and works its way down, total the /opt/informix directory, now > > total > > any dirctories under opt/informix, and any under that... make sense? I am > > also > > working on this but if there are any homies out there that already have > > this > > please hook a brother up. > > > > MM > > > > > > > > > > ******************************************************************************* > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > -- > Art S. Kagel > Oninit (www.oninit.com) > IIUG Board of Directors (art@iiug.org) > > Disclaimer: Please keep in mind that my own opinions are my own opinions > and > do not reflect on my employer, Oninit, the IIUG, nor any other organization > with which I am associated either explicitly or implicitly. Neither do > those opinions reflect those of other individuals affiliated with any > entity > with which I am affiliated nor those of the entities themselves. > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Art S. Kagel Oninit (www.oninit.com) IIUG Board of Directors (art@iiug.org) Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Oninit, the IIUG, nor any other organization with which I am associated either explicitly or implicitly. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.
Of course, du works better if you have links in that directory. Hrvoje > -----Original Message----- > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On > Behalf Of Art Kagel > Sent: Wednesday, November 19, 2008 5:28 PM > To: ids@iiug.org > Subject: Re: Quick help - I hope [14055] > > Almost there: > > find /opt/informix -exec ls -l {} \\\\; | awk 'BEGIN {total = > 0;}{ total += $5 } END {print "column total is", total}' > > Should do the trick. Of course you could just: > > du -s /opt/informix > > This reports in 'blocks'. The block size differs on different > platforms - don't remember what Solaris uses. The answer will > be the same from both runs (though du will tend to be faster) > except that the find uses actual file sizes which ignores the > waste between EOF and the end of the physical disk blocks. > > You can use ls to take account of this with the -s option and > use field $1 instead and multiply by the block size (which > you can back out of any ls -ls listing. This works on my > Linux system where the block size is 1K: > > find /opt/informix -exec ls -ls {} \\\\; | awk 'BEGIN {total = > 0;}{ total += $1 > * 1024 } END {print "column total is", total}' > > Returning the more accurate 1484152832 where the first query returns > 1463203965 a difference of 21MB. > > Art > > On Wed, Nov 19, 2008 at 10:28 AM, MIKE MAGIE > <jmmagie@yahoo.com> wrote: > > > IDS 10.00.UC8 > > Solaris 2.8 > > > > I need some help with a ksh script. I need to determine how > much file > > system space the db team is using on one of our Unix boxes. > There are > > buttloads of directories I need to look at - I had started > doing this > > but it is tedious: > > > > ls -l | awk '{ total += $5 } END {print "column total is", total}' > > > > I need the part of the script that will start an ls at the top of a > > given directory and works its way down, total the /opt/informix > > directory, now total any dirctories under opt/informix, and > any under > > that... make sense? I am also working on this but if there are any > > homies out there that already have this please hook a brother up. > > > > MM > > > > > > > > > ************************************************************** > ***************** > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > -- > Art S. Kagel > Oninit (www.oninit.com) > IIUG Board of Directors (art@iiug.org) > > Disclaimer: Please keep in mind that my own opinions are my > own opinions and do not reflect on my employer, Oninit, the > IIUG, nor any other organization with which I am associated > either explicitly or implicitly. Neither do those opinions > reflect those of other individuals affiliated with any entity > with which I am affiliated nor those of the entities themselves. > > > ************************************************************** > ***************** > Forum Note: Use "Reply" to post a response in the discussion forum. >
Art, you should use LSD :o) ls -lsd because ls -ls lists all files in directory and lists every single file. Hrvoje > -----Original Message----- > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On > Behalf Of Art Kagel > Sent: Wednesday, November 19, 2008 5:43 PM > To: ids@iiug.org > Subject: Re: Quick help - I hope [14056] > > Hmm, for some reason 'find' on my machine is listing every > file twice. Once with a full path and once without one, so > the number's it's producing are doubled. Well, as others have > stated also, the du is accurate and faster. > > Art > > On Wed, Nov 19, 2008 at 11:27 AM, Art Kagel > <art.kagel@gmail.com> wrote: > > > Almost there: > > > > find /opt/informix -exec ls -l {} \\\\; | awk 'BEGIN {total = > 0;}{ total > > += $5 } END {print "column total is", total}' > > > > Should do the trick. Of course you could just: > > > > du -s /opt/informix > > > > This reports in 'blocks'. The block size differs on different > > platforms - don't remember what Solaris uses. The answer > will be the > > same from both runs (though du will tend to be faster) > except that the > > find uses actual file sizes which ignores the waste between EOF and > > the end of the physical disk blocks. > > > > You can use ls to take account of this with the -s option and use > > field $1 instead and multiply by the block size (which you can back > > out of any ls -ls listing. This works on my Linux system where the > > block size is 1K: > > > > find /opt/informix -exec ls -ls {} \\\\; | awk 'BEGIN {total = > 0;}{ total > > += > > $1 > > * 1024 } END {print "column total is", total}' > > > > Returning the more accurate 1484152832 where the first query returns > > 1463203965 a difference of 21MB. > > > > Art > > > > On Wed, Nov 19, 2008 at 10:28 AM, MIKE MAGIE > <jmmagie@yahoo.com> wrote: > > > > > IDS 10.00.UC8 > > > Solaris 2.8 > > > > > > I need some help with a ksh script. I need to determine how much > > > file system space the db team is using on one of our Unix boxes. > > > There are buttloads > > of > > > directories I need to look at - I had started doing this but it is > > tedious: > > > > > > ls -l | awk '{ total += $5 } END {print "column total > is", total}' > > > > > > I need the part of the script that will start an ls at > the top of a > > > given directory and works its way down, total the /opt/informix > > > directory, now total any dirctories under opt/informix, and any > > > under that... make sense? I am also working on this but > if there are > > > any homies out there that already have this please hook a brother > > > up. > > > > > > MM > > > > > > > > > > > > > > > > > ************************************************************** > ***************** > > > Forum Note: Use "Reply" to post a response in the > discussion forum. > > > > > > > > > > -- > > Art S. Kagel > > Oninit (www.oninit.com) > > IIUG Board of Directors (art@iiug.org) > > > > Disclaimer: Please keep in mind that my own opinions are my own > > opinions and do not reflect on my employer, Oninit, the > IIUG, nor any > > other organization with which I am associated either explicitly or > > implicitly. Neither do those opinions reflect those of other > > individuals affiliated with any entity with which I am > affiliated nor > > those of the entities themselves. > > > > > > > > > ************************************************************** > ***************** > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > -- > Art S. Kagel > Oninit (www.oninit.com) > IIUG Board of Directors (art@iiug.org) > > Disclaimer: Please keep in mind that my own opinions are my > own opinions and do not reflect on my employer, Oninit, the > IIUG, nor any other organization with which I am associated > either explicitly or implicitly. Neither do those opinions > reflect those of other individuals affiliated with any entity > with which I am affiliated nor those of the entities themselves. > > > ************************************************************** > ***************** > Forum Note: Use "Reply" to post a response in the discussion forum. >
DUH! I KNEW that <he says sheepishly>. A senior moment for sure. Thanks. Art On Wed, Nov 19, 2008 at 12:00 PM, Hrvoje Zokovic <hzokovic.iiug@gmail.com>wrote: > Art, you should use LSD :o) > ls -lsd because ls -ls lists all files in directory and lists every single > file. > Hrvoje > > > -----Original Message----- > > From: ids-bounces@iiug.org [mailto:ids-bounces@iiug.org] On > > Behalf Of Art Kagel > > Sent: Wednesday, November 19, 2008 5:43 PM > > To: ids@iiug.org > > Subject: Re: Quick help - I hope [14056] > > > > Hmm, for some reason 'find' on my machine is listing every > > file twice. Once with a full path and once without one, so > > the number's it's producing are doubled. Well, as others have > > stated also, the du is accurate and faster. > > > > Art > > > > On Wed, Nov 19, 2008 at 11:27 AM, Art Kagel > > <art.kagel@gmail.com> wrote: > > > > > Almost there: > > > > > > find /opt/informix -exec ls -l {} \\\\; | awk 'BEGIN {total = > > 0;}{ total > > > += $5 } END {print "column total is", total}' > > > > > > Should do the trick. Of course you could just: > > > > > > du -s /opt/informix > > > > > > This reports in 'blocks'. The block size differs on different > > > platforms - don't remember what Solaris uses. The answer > > will be the > > > same from both runs (though du will tend to be faster) > > except that the > > > find uses actual file sizes which ignores the waste between EOF and > > > the end of the physical disk blocks. > > > > > > You can use ls to take account of this with the -s option and use > > > field $1 instead and multiply by the block size (which you can back > > > out of any ls -ls listing. This works on my Linux system where the > > > block size is 1K: > > > > > > find /opt/informix -exec ls -ls {} \\\\; | awk 'BEGIN {total = > > 0;}{ total > > > += > > > $1 > > > * 1024 } END {print "column total is", total}' > > > > > > Returning the more accurate 1484152832 where the first query returns > > > 1463203965 a difference of 21MB. > > > > > > Art > > > > > > On Wed, Nov 19, 2008 at 10:28 AM, MIKE MAGIE > > <jmmagie@yahoo.com> wrote: > > > > > > > IDS 10.00.UC8 > > > > Solaris 2.8 > > > > > > > > I need some help with a ksh script. I need to determine how much > > > > file system space the db team is using on one of our Unix boxes. > > > > There are buttloads > > > of > > > > directories I need to look at - I had started doing this but it is > > > tedious: > > > > > > > > ls -l | awk '{ total += $5 } END {print "column total > > is", total}' > > > > > > > > I need the part of the script that will start an ls at > > the top of a > > > > given directory and works its way down, total the /opt/informix > > > > directory, now total any dirctories under opt/informix, and any > > > > under that... make sense? I am also working on this but > > if there are > > > > any homies out there that already have this please hook a brother > > > > up. > > > > > > > > MM > > > > > > > > > > > > > > > > > > > > > > > > ************************************************************** > > ***************** > > > > Forum Note: Use "Reply" to post a response in the > > discussion forum. > > > > > > > > > > > > > > -- > > > Art S. Kagel > > > Oninit (www.oninit.com) > > > IIUG Board of Directors (art@iiug.org) > > > > > > Disclaimer: Please keep in mind that my own opinions are my own > > > opinions and do not reflect on my employer, Oninit, the > > IIUG, nor any > > > other organization with which I am associated either explicitly or > > > implicitly. Neither do those opinions reflect those of other > > > individuals affiliated with any entity with which I am > > affiliated nor > > > those of the entities themselves. > > > > > > > > > > > > > > ************************************************************** > > ***************** > > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > > > > > -- > > Art S. Kagel > > Oninit (www.oninit.com) > > IIUG Board of Directors (art@iiug.org) > > > > Disclaimer: Please keep in mind that my own opinions are my > > own opinions and do not reflect on my employer, Oninit, the > > IIUG, nor any other organization with which I am associated > > either explicitly or implicitly. Neither do those opinions > > reflect those of other individuals affiliated with any entity > > with which I am affiliated nor those of the entities themselves. > > > > > > ************************************************************** > > ***************** > > Forum Note: Use "Reply" to post a response in the discussion forum. > > > > > > ******************************************************************************* > Forum Note: Use "Reply" to post a response in the discussion forum. > > -- Art S. Kagel Oninit (www.oninit.com) IIUG Board of Directors (art@iiug.org) Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Oninit, the IIUG, nor any other organization with which I am associated either explicitly or implicitly. Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.