rsspipe: Pipe the command line to RSS

Reading time: 1 – 2 minutes

rsspipe is a simple python 2.3 script that reads stdin and outputs the last lines as an RSS 0.92 file, one RSS item (“headline”) per line.

It’s a general tool, but it may help if I give you a specific usage
example. I use it to keep track of recent referrers. I have a
long-running command like this:

tail -c 1000000000 -f access_log
|./weblog_parse -quiet referer
|python --title ' referrers' referrers-rentzsch.xml

That’s one line — I inserted line breaks on the pipes to make it easier to follow. Let’s walk through it command-by-command:

  • First I have tail read my entire log file (-c 1000000000) and continue reading it forever (-f).
  • Those
    log lines are fed to a slightly modified version of the excellent
    weblog_parse, which extracts the referrer from the log line.
  • The
    referrer lines are fed to xuniq, which is a lot like uniq except its
    input doesn’t need to be sorted first. It only outputs unique lines
    (lines it hasn’t seen before).
  • Those unique referrers are transformed into a rolling RSS file for ongoing consumption. You can see it in action here.