Counting Log Items

I have a log file that rolls each day. It has logging from various threads in it. I wanted to count the frequency of an item hitting one of the threads for a certain hour over the last month. Log file is called blahproc.log. Rolled files are called blahproc.log.YYYY-MM-DD. Thread I’m interested in is called ‘listener2’. Log line I’m interested in counting has ‘INFO com.mrsock.feeds.NetControl’ in it. Hour I’m interested in is 12:00-13:00.

First grep out the matching lines from each day, which will include the file name at the beginning of each line:
fgrep 'INFO listener2 com.mrsock.feeds.NetControl' blahproc.log.2010-05* > ~/tactical/netcount01.txt

Then grab just the hour we’re interested in:
grep -E ' 12:.* INFO' ~/tactical/netcount01.txt > ~/tactical/netcount02.txt

Then count those babies and make into a usable csv format. File name is followed by a colon so use the filenames to count the occurances between 12:00 and 13:00 :
awk -F':' ' { print $1 } ' /~/tactical/netcount02.txt | sort | uniq -c | sort -nr | sed 's/ blahproc.log./ /' | awk ' { print $2 "," $1 } '

Result is something like:
2010-05-06,3059
2010-05-07,2407
2010-05-19,2186
2010-05-20,2043
2010-05-11,2036
2010-05-21,2006
2010-05-04,1548
2010-05-05,1268
2010-05-10,1114
2010-05-13,1092
2010-05-12,1022
2010-05-17,1001
2010-05-18,854
2010-05-24,753
2010-05-14,692
2010-05-03,172

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s