Reading time: 1 – 2 minutes
rsspipe is a simple python 2.3 script that reads stdin and outputs the last lines as an RSS 0.92 file, one RSS item (“headline”) per line.
It’s a general tool, but it may help if I give you a specific usage
example. I use it to keep track of recent referrers. I have a
long-running command like this:
tail -c 1000000000 -f access_log
|./weblog_parse -quiet referer
|./xuniq
|python rsspipe.py --title 'rentzsch.com referrers' referrers-rentzsch.xml
That’s one line — I inserted line breaks on the pipes to make it easier to follow. Let’s walk through it command-by-command:
- First I have tail read my entire log file (-c 1000000000) and continue reading it forever (-f).
- Those
log lines are fed to a slightly modified version of the excellent
weblog_parse, which extracts the referrer from the log line. - The
referrer lines are fed to xuniq, which is a lot like uniq except its
input doesn’t need to be sorted first. It only outputs unique lines
(lines it hasn’t seen before). - Those unique referrers are transformed into a rolling RSS file for ongoing consumption. You can see it in action here.