[NTLUG:Discuss] disk usage by file age grave accent
Fred James
fredjame at fredjame.cnc.net
Thu Jul 1 12:45:28 CDT 2010
Michael Barnes wrote:
> On Thu, Jul 1, 2010 at 11:14 AM, Michael Barnes <barnmichael at gmail.com> wrote:
>
>> On Wed, Jun 30, 2010 at 8:42 PM, Fred James <fredjame at fredjame.cnc.net> wrote:
>>
>>> Michael Barnes wrote:
>>>
>>>> On Wed, Jun 30, 2010 at 2:34 PM, Fred James <fredjame at fredjame.cnc.net>
>>>> wrote:
>>>>
>>>>
>>>>> Michael Barnes wrote:
>>>>>
>>>>>
>>>>>> On Wed, Jun 30, 2010 at 12:58 PM, Fred James <fredjame at fredjame.cnc.net>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Michael Barnes wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> I'm trying to figure out how to gather some disk data. What I need is
>>>>>>>> to find the disk usage by subfolder, but only by files older than a
>>>>>>>> certain time. I can get a list of files with
>>>>>>>> find ./ftp/news +mtime 180
>>>>>>>> and get a list of everything older than six months.
>>>>>>>> I can do
>>>>>>>> du -sh ./ftp/news/*
>>>>>>>> and find the usage by subfolder.
>>>>>>>> What I need is to combine the two, so I get something like
>>>>>>>>
>>>>>>>> 15M ftp/news/4-Dallas
>>>>>>>> 40M ftp/news/4-Washington
>>>>>>>> 560M ftp/news/House
>>>>>>>> 1.1G ftp/news/Senate
>>>>>>>> 717M ftp/news/White House
>>>>>>>> 69M ftp/news/YIR 2009
>>>>>>>> 65M ftp/news/stuff
>>>>>>>>
>>>>>>>> knowing that the size is for files over six months old.
>>>>>>>>
>>>>>>>>
>>>>>>>> Any simple ideas for this?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Michael
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> Michael Barnes
>>>>>>> Have you considered the various 'time' options for 'du' (snip below)?
>>>>>>> Would
>>>>>>> any of those get you closer to what you want?
>>>>>>> You might also consider the '-printf' option (and it's formating
>>>>>>> options)
>>>>>>> of
>>>>>>> 'find' (under Actions). Hope that helps
>>>>>>> Regards
>>>>>>> Fred James
>>>>>>>
>>>>>>> --time show time of the last modification of any file in the directory,
>>>>>>> or
>>>>>>> any of its subdirectories
>>>>>>>
>>>>>>> --time=WORD
>>>>>>> show time as WORD instead of modification time: atime, access, use,
>>>>>>> ctime
>>>>>>> or
>>>>>>> status
>>>>>>>
>>>>>>> --time-style=STYLE
>>>>>>> show times using style STYLE: full-iso, long-iso, iso, +FORMAT FORMAT
>>>>>>> is
>>>>>>> interpreted like ‘date'
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> I tried the 'time' options and got
>>>>>> du: unrecognized option `--time'
>>>>>> and they are not listed as options in du --help.
>>>>>> It is an older machine running SLES 9. Apparently the 'time' options
>>>>>> are new. I don't know how I would upgrade du for this old SuSE
>>>>>> install.
>>>>>>
>>>>>> Thanks,
>>>>>> Michael
>>>>>>
>>>>>>
>>>>>>
>>>>> Michael Barnes
>>>>> A quick look at an old Unix in a Nutshell book also notes a lack of
>>>>> printf
>>>>> in find ... so ...
>>>>> find ./ftp/news +mtime 180 -print > fred
>>>>> ... should get you a text file of the list of files you are interested in
>>>>> and something like ...
>>>>> while read i
>>>>> do
>>>>> data=`du -h $i`
>>>>> echo "${data} ${i}" >> freddy.txt
>>>>> done < fred
>>>>> exit
>>>>> ... should produce a text file that contains something like this (see
>>>>> below)
>>>>> ... assuming that 'freddy.txt' does not already exist ...
>>>>> Does that help?
>>>>> Regards
>>>>> Fred James
>>>>>
>>>>> 724K ./tmp/orbit-fredjame
>>>>> 4.0K ./tmp/ksocket-fredjame/artsd-samples
>>>>> 12K ./tmp/ksocket-fredjame
>>>>> 4.0K ./tmp/kde-fredjame/kdenlive
>>>>> 16K ./tmp/kde-fredjame
>>>>> 4.0K ./tmp/keyring-pZD1S8
>>>>> 4.0K ./tmp/keyring-BivafU
>>>>> 4.0K ./tmp/keyring-HkLdP7
>>>>> 84K ./tmp/plugtmp
>>>>> 4.0K ./tmp/plugtmp-1
>>>>> 4.0K ./tmp/keyring-6gKCsT
>>>>> 4.0K ./tmp/keyring-22APTg
>>>>> 4.0K ./tmp/keyring-EClhPQ
>>>>> 4.0K ./tmp/keyring-4bEPT3
>>>>>
>>>>>
>>>>>
>>>> That is getting me closer. The problem now is it doesn't work well
>>>> with spaces in file names. It also lists all the files, while all I
>>>> need are the folders with size summary. But, I'm a lot closer now.
>>>>
>>>> Thanks,
>>>> Michael
>>>>
>>>>
>>> Michael Barnes
>>> Modify the while statement to something like ...
>>> while read i
>>> do
>>> if test -d "${i}"
>>> then
>>> data=`du -sh "${i}"`
>>> echo "${data}" >> freddy.txt
>>> fi
>>> done < fred
>>>
>>> Any better?
>>> Regards
>>> Fred James
>>>
>>> PS: Note that those are back ticks (grave accent) surrounding the 'du'
>>> statement.
>>>
>>>
>> Sorry, not yet. I start with creating the fred file with
>>
>> find /home/ftp/srnnews/ -mtime +180 -print > fred
>>
>> The fred file looks like this:
>>
>> /home/ftp/srnnews/House/HC-DEBATE-boehner-a4-SUNam.wav
>> /home/ftp/srnnews/House/THR-7-9--SAME-SEX-2w_22
>> /home/ftp/srnnews/House/HC-GOP-VOTE-doug-v-MONam_22
>> /home/ftp/srnnews/House/HC-BILL-PASSES-doug-w2-SUNam.wav
>> /home/ftp/srnnews/House/CLUNKERS-bond-a1-8-6.MP3
>> /home/ftp/srnnews/House/KENNEDY-MEMORIAL SERVICE-nats-a1.wav
>> /home/ftp/srnnews/House/CASH FOR CLUNKERS-feinstein-a2-8-3.MP3
>> /home/ftp/srnnews/House/THU-7-9--SAME-SEX-w2.wav
>> /home/ftp/srnnews/House/CLUNKERS-levin-a1-8-6.MP3
>> /home/ftp/srnnews/House/CLUNKERS-mccain-a1-8-6.MP3
>> /home/ftp/srnnews/House/KENNEDY TRIBUTE-byrd-a-9-10.MP3
>> /home/ftp/srnnews/stuff/WHEM Africa Feature.wav
>> /home/ftp/srnnews/stuff/9-11-SHANKESVILLE-powell-a1.wav
>> /home/ftp/srnnews/stuff/SRN_NewsA.mp3
>> /home/ftp/srnnews/stuff/9-11-NYC-DRUMS.wav
>> /home/ftp/srnnews/stuff/9-11-SHANKESVILLE-nats.wav
>> /home/ftp/srnnews/stuff/9-11-NYC-NAMES.wav
>> /home/ftp/srnnews/stuff/9-11-SHANKESVILLE-bells-names-a1.wav
>> /home/ftp/srnnews/stuff/SRN_NewsBC.mp3
>> /home/ftp/srnnews/stuff/9-11-NYC-bagpipes.wav
>> /home/ftp/srnnews/stuff/audacity-win-1.2.6.exe
>> /home/ftp/srnnews/stuff/9-11-BLOOMBERG-a1.wav
>> /home/ftp/srnnews/Senate/LIEBERMAN-kenyon-TUES.wav
>> /home/ftp/srnnews/Senate/13269D2-56561_Admin_a_o_i_LIEBERMAN-SENATE-kenyon-w-THURSpm_22
>> /home/ftp/srnnews/Senate/COLEMAN-09306233 HOLD FOR LINDA.wav
>> /home/ftp/srnnews/Senate/KENNEDY-BENNETT-bennett-a2.wav
>> /home/ftp/srnnews/Senate/KENNEDY-BENNETT-bennett-a3.wav
>> /home/ftp/srnnews/Senate/KENNEDY-BENNETT-bennett-a4.wav
>> /home/ftp/srnnews/Senate/BAILOUT-paulson-a2-TUESam.wav
>> /home/ftp/srnnews/Senate/KENNEDY MEMORIAL-kennedy-j-a.wav
>> /home/ftp/srnnews/Senate/SOTOMAYOR-kenyon-w2-THURS 8-6.MP3
>> /home/ftp/srnnews/Senate/SOTOMAYOR-kenyon-w2-THURS 8-6.wav
>> /home/ftp/srnnews/Senate/13290F1-21767_snnapuser_a_o_i_HEALTH
>> CARE-various-2aSATam_22
>> /home/ftp/srnnews/Senate/LEAHY-for all americans.wav
>> /home/ftp/srnnews/White House/OBAMA-RAMADAN-09-01.MP3
>> /home/ftp/srnnews/White House/wh-religious leaders-08-19.wav
>> /home/ftp/srnnews/White House/BUSH - 11-05.wav
>> /home/ftp/srnnews/White
>> House/Obama_Irritated_With_White_House_Reporter_-_1_22_09.wmv
>> /home/ftp/srnnews/White House/maginnis reserves question for greg.wav
>> /home/ftp/srnnews/White House/AP-GAFFE-01-22.mp3
>> /home/ftp/srnnews/White House/bushism-1.wav
>> /home/ftp/srnnews/White House/bushism-2.wav
>> /home/ftp/srnnews/YIR 2009
>> /home/ftp/srnnews/YIR 2009/FORT HOOD-gordon.mp3
>> /home/ftp/srnnews/YIR 2009/FORT HOOD-gordon.rtf
>> /home/ftp/srnnews/YIR 2009/YIR 2009-OBAMA AFGHAN WAR-clug.wav
>> /home/ftp/srnnews/YIR 2009/YIR 2009-OBAMA ECONOMY-clug.wav
>> /home/ftp/srnnews/YIR 2009/1329149-40114_snnapuser_a_o_i_SCRIPT_22
>> /home/ftp/srnnews/YIR 2009/PALIN YIR.rtf
>> /home/ftp/srnnews/YIR 2009/ricK 12-4-09.wav
>>
>> Then I run the script above and freddy.txt says
>>
>> 69M /home/ftp/srnnews/YIR 2009
>>
>> My desired result would be
>>
>> 754M /home/ftp/srnnews/House
>> 75M /home/ftp/srnnews/stuff
>> 425M /home/ftp/srnnews/Senate
>> 1.2G /home/ftp/srnnews/White House
>> 69M /home/ftp/srnnews/YIR 2009
>>
>> What is real strange is if I edit the fred file to remove the
>> /home/ftp/srnnews/YIR 2009 files, then run the script, it generates
>> nothing. It sure has me confused.
>>
>> Thanks for your help
>> Michael
>>
>>
>
> I found part of the problem. In the file fred, the directories are
> not listed except for '/home/ftp/srnnews/YIR 2009'. If I go into fred
> and add /home/ftp/srnnews/House, /home/ftp/srnnews/stuff,
> /home/ftp/srnnews/Senate, /home/ftp/srnnews/White House then run the
> script again, I get
>
> 2.7G /home/ftp/srnnews
> 560M /home/ftp/srnnews/House
> 65M /home/ftp/srnnews/stuff
> 1.2G /home/ftp/srnnews/Senate
> 717M /home/ftp/srnnews/White House
> 69M /home/ftp/srnnews/YIR 2009
>
> which is the desired result. However, I need to run this on several
> folders with a ton of subdirectories. I don't want to have to go in
> and edit dozens of lines each time.
>
> Ideas?
>
> TNX,
> Michael
>
Michael Barnes
(1) If your search path is "/home/ftp/srnnews/", then the line
"/home/ftp/srnnews" should not appear in the output of find
(2) Looking at your output from find, I do admit I don't know why
"/home/ftp/srnnews/YIR 2009" is there but the others are not (except as
mentioned in item 1).
(2.1) Logically, if the find statement list one directory, one could be
forgiven for thinking it should list the others
(2.1.1) Unless it is a permissions issue, perhaps?
(2.1.1.1) Unless you are the privileged user (root, for example), find
will not find all files and directories for which you do not have
permission/privilege.
Any joy there?
I did try using the '-type' and '-xtype' options for 'find', but at
least on my machine those appear to be broken?
Regards
Fred James
More information about the Discuss
mailing list