[NTLUG:Discuss] disk usage by file age grave accent

[Michael Barnes]

On Thu, Jul 1, 2010 at 11:14 AM, Michael Barnes <barnmichael at gmail.com> wrote:
> On Wed, Jun 30, 2010 at 8:42 PM, Fred James <fredjame at fredjame.cnc.net> wrote:
>> Michael Barnes wrote:
>>>
>>> On Wed, Jun 30, 2010 at 2:34 PM, Fred James <fredjame at fredjame.cnc.net>
>>> wrote:
>>>
>>>>
>>>> Michael Barnes wrote:
>>>>
>>>>>
>>>>> On Wed, Jun 30, 2010 at 12:58 PM, Fred James <fredjame at fredjame.cnc.net>
>>>>> wrote:
>>>>>
>>>>>
>>>>>>
>>>>>> Michael Barnes wrote:
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> I'm trying to figure out how to gather some disk data.  What I need is
>>>>>>> to find the disk usage by subfolder, but only by files older than a
>>>>>>> certain time.  I can get a list of files with
>>>>>>> find ./ftp/news +mtime 180
>>>>>>> and get a list of everything older than six months.
>>>>>>> I can do
>>>>>>> du -sh ./ftp/news/*
>>>>>>> and find the usage by subfolder.
>>>>>>> What I need is to combine the two, so I get something like
>>>>>>>
>>>>>>> 15M             ftp/news/4-Dallas
>>>>>>> 40M             ftp/news/4-Washington
>>>>>>> 560M    ftp/news/House
>>>>>>> 1.1G            ftp/news/Senate
>>>>>>> 717M    ftp/news/White House
>>>>>>> 69M             ftp/news/YIR 2009
>>>>>>> 65M             ftp/news/stuff
>>>>>>>
>>>>>>> knowing that the size is for files over six months old.
>>>>>>>
>>>>>>>
>>>>>>> Any simple ideas for this?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Michael
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> Michael Barnes
>>>>>> Have you considered the various 'time' options for 'du' (snip below)?
>>>>>> Would
>>>>>> any of those get you closer to what you want?
>>>>>> You might also consider the '-printf' option (and it's formating
>>>>>> options)
>>>>>> of
>>>>>> 'find' (under Actions). Hope that helps
>>>>>> Regards
>>>>>> Fred James
>>>>>>
>>>>>> --time show time of the last modification of any file in the directory,
>>>>>> or
>>>>>> any of its subdirectories
>>>>>>
>>>>>> --time=WORD
>>>>>> show time as WORD instead of modification time: atime, access, use,
>>>>>> ctime
>>>>>> or
>>>>>> status
>>>>>>
>>>>>> --time-style=STYLE
>>>>>> show times using style STYLE: full-iso, long-iso, iso, +FORMAT FORMAT
>>>>>> is
>>>>>> interpreted like ‘date'
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> I tried the 'time' options and got
>>>>> du: unrecognized option `--time'
>>>>> and they are not listed as options in du --help.
>>>>> It is an older machine running SLES 9.  Apparently the 'time' options
>>>>> are new.  I don't know how I would upgrade du for this old SuSE
>>>>> install.
>>>>>
>>>>> Thanks,
>>>>> Michael
>>>>>
>>>>>
>>>>
>>>> Michael Barnes
>>>> A quick look at an old Unix in a Nutshell book also notes a lack of
>>>> printf
>>>> in find ... so ...
>>>>  find ./ftp/news +mtime 180 -print > fred
>>>> ... should get you a text file of the list of files you are interested in
>>>> and something like ...
>>>>  while read i
>>>>  do
>>>>          data=`du -h $i`
>>>>          echo "${data} ${i}" >> freddy.txt
>>>>  done < fred
>>>>  exit
>>>> ... should produce a text file that contains something like this (see
>>>> below)
>>>> ... assuming that 'freddy.txt' does not already exist ...
>>>> Does that help?
>>>> Regards
>>>> Fred James
>>>>
>>>> 724K    ./tmp/orbit-fredjame
>>>> 4.0K    ./tmp/ksocket-fredjame/artsd-samples
>>>> 12K     ./tmp/ksocket-fredjame
>>>> 4.0K    ./tmp/kde-fredjame/kdenlive
>>>> 16K     ./tmp/kde-fredjame
>>>> 4.0K    ./tmp/keyring-pZD1S8
>>>> 4.0K    ./tmp/keyring-BivafU
>>>> 4.0K    ./tmp/keyring-HkLdP7
>>>> 84K     ./tmp/plugtmp
>>>> 4.0K    ./tmp/plugtmp-1
>>>> 4.0K    ./tmp/keyring-6gKCsT
>>>> 4.0K    ./tmp/keyring-22APTg
>>>> 4.0K    ./tmp/keyring-EClhPQ
>>>> 4.0K    ./tmp/keyring-4bEPT3
>>>>
>>>>
>>>
>>> That is getting me closer.  The problem now is it doesn't work well
>>> with spaces in file names.  It also lists all the files, while all I
>>> need are the folders with size summary.   But, I'm a lot closer now.
>>>
>>> Thanks,
>>> Michael
>>>
>>
>> Michael Barnes
>> Modify the while statement to something like ...
>>   while read i
>>   do
>>           if test -d "${i}"
>>           then
>>                   data=`du -sh "${i}"`
>>                   echo "${data}" >> freddy.txt
>>           fi
>>   done < fred
>>
>> Any better?
>> Regards
>> Fred James
>>
>> PS:  Note that those are back ticks (grave accent) surrounding the 'du'
>> statement.
>>
>
> Sorry, not yet.  I start with creating the fred file with
>
> find /home/ftp/srnnews/ -mtime +180 -print > fred
>
> The fred file looks like this:
>
> /home/ftp/srnnews/House/HC-DEBATE-boehner-a4-SUNam.wav
> /home/ftp/srnnews/House/THR-7-9--SAME-SEX-2w_22
> /home/ftp/srnnews/House/HC-GOP-VOTE-doug-v-MONam_22
> /home/ftp/srnnews/House/HC-BILL-PASSES-doug-w2-SUNam.wav
> /home/ftp/srnnews/House/CLUNKERS-bond-a1-8-6.MP3
> /home/ftp/srnnews/House/KENNEDY-MEMORIAL SERVICE-nats-a1.wav
> /home/ftp/srnnews/House/CASH FOR CLUNKERS-feinstein-a2-8-3.MP3
> /home/ftp/srnnews/House/THU-7-9--SAME-SEX-w2.wav
> /home/ftp/srnnews/House/CLUNKERS-levin-a1-8-6.MP3
> /home/ftp/srnnews/House/CLUNKERS-mccain-a1-8-6.MP3
> /home/ftp/srnnews/House/KENNEDY TRIBUTE-byrd-a-9-10.MP3
> /home/ftp/srnnews/stuff/WHEM Africa Feature.wav
> /home/ftp/srnnews/stuff/9-11-SHANKESVILLE-powell-a1.wav
> /home/ftp/srnnews/stuff/SRN_NewsA.mp3
> /home/ftp/srnnews/stuff/9-11-NYC-DRUMS.wav
> /home/ftp/srnnews/stuff/9-11-SHANKESVILLE-nats.wav
> /home/ftp/srnnews/stuff/9-11-NYC-NAMES.wav
> /home/ftp/srnnews/stuff/9-11-SHANKESVILLE-bells-names-a1.wav
> /home/ftp/srnnews/stuff/SRN_NewsBC.mp3
> /home/ftp/srnnews/stuff/9-11-NYC-bagpipes.wav
> /home/ftp/srnnews/stuff/audacity-win-1.2.6.exe
> /home/ftp/srnnews/stuff/9-11-BLOOMBERG-a1.wav
> /home/ftp/srnnews/Senate/LIEBERMAN-kenyon-TUES.wav
> /home/ftp/srnnews/Senate/13269D2-56561_Admin_a_o_i_LIEBERMAN-SENATE-kenyon-w-THURSpm_22
> /home/ftp/srnnews/Senate/COLEMAN-09306233 HOLD FOR LINDA.wav
> /home/ftp/srnnews/Senate/KENNEDY-BENNETT-bennett-a2.wav
> /home/ftp/srnnews/Senate/KENNEDY-BENNETT-bennett-a3.wav
> /home/ftp/srnnews/Senate/KENNEDY-BENNETT-bennett-a4.wav
> /home/ftp/srnnews/Senate/BAILOUT-paulson-a2-TUESam.wav
> /home/ftp/srnnews/Senate/KENNEDY MEMORIAL-kennedy-j-a.wav
> /home/ftp/srnnews/Senate/SOTOMAYOR-kenyon-w2-THURS 8-6.MP3
> /home/ftp/srnnews/Senate/SOTOMAYOR-kenyon-w2-THURS 8-6.wav
> /home/ftp/srnnews/Senate/13290F1-21767_snnapuser_a_o_i_HEALTH
> CARE-various-2aSATam_22
> /home/ftp/srnnews/Senate/LEAHY-for all americans.wav
> /home/ftp/srnnews/White House/OBAMA-RAMADAN-09-01.MP3
> /home/ftp/srnnews/White House/wh-religious leaders-08-19.wav
> /home/ftp/srnnews/White House/BUSH - 11-05.wav
> /home/ftp/srnnews/White
> House/Obama_Irritated_With_White_House_Reporter_-_1_22_09.wmv
> /home/ftp/srnnews/White House/maginnis reserves question for greg.wav
> /home/ftp/srnnews/White House/AP-GAFFE-01-22.mp3
> /home/ftp/srnnews/White House/bushism-1.wav
> /home/ftp/srnnews/White House/bushism-2.wav
> /home/ftp/srnnews/YIR 2009
> /home/ftp/srnnews/YIR 2009/FORT HOOD-gordon.mp3
> /home/ftp/srnnews/YIR 2009/FORT HOOD-gordon.rtf
> /home/ftp/srnnews/YIR 2009/YIR 2009-OBAMA AFGHAN WAR-clug.wav
> /home/ftp/srnnews/YIR 2009/YIR 2009-OBAMA ECONOMY-clug.wav
> /home/ftp/srnnews/YIR 2009/1329149-40114_snnapuser_a_o_i_SCRIPT_22
> /home/ftp/srnnews/YIR 2009/PALIN YIR.rtf
> /home/ftp/srnnews/YIR 2009/ricK 12-4-09.wav
>
> Then I run the script above and freddy.txt says
>
> 69M     /home/ftp/srnnews/YIR 2009
>
> My desired result would be
>
> 754M  /home/ftp/srnnews/House
> 75M   /home/ftp/srnnews/stuff
> 425M /home/ftp/srnnews/Senate
> 1.2G  /home/ftp/srnnews/White House
> 69M     /home/ftp/srnnews/YIR 2009
>
> What is real strange is if I edit the fred file to remove the
> /home/ftp/srnnews/YIR 2009 files, then run the script, it generates
> nothing.  It sure has me confused.
>
> Thanks for your help
> Michael
>

I found part of the problem.  In the file fred, the directories are
not listed except for '/home/ftp/srnnews/YIR 2009'.  If I go into fred
and add  /home/ftp/srnnews/House,  /home/ftp/srnnews/stuff,
/home/ftp/srnnews/Senate,  /home/ftp/srnnews/White House then run the
script again, I get

2.7G	/home/ftp/srnnews
560M	/home/ftp/srnnews/House
65M	/home/ftp/srnnews/stuff
1.2G	/home/ftp/srnnews/Senate
717M	/home/ftp/srnnews/White House
69M	/home/ftp/srnnews/YIR 2009

which is the desired result.  However, I need to run this on several
folders with a ton of subdirectories.  I don't want to have to go in
and edit dozens of lines each time.

Ideas?

TNX,
Michael