Author: Oriol Rius

Gentoo: Specifying only needed locales

Reading time: 2 – 3 minutes

[Gentoo Weekly News] The locales a user can choose from are built by the glibc. Usually all
available locales starting from aa_DJ (Afar locale for Djibouti) over
en_US (English locale for the USA) to zu_ZA.utf8 (Zulu locale for South
Africa) will be installed. Unless you’re working at the UN and administer
a central server for all member states, it is difficult to conceive why
you would need a system where all of these locales are installed. This
week’s tip was written with all those of you in mind who’d like to save 90
percent of the space occupied by locales in their system, by limiting the
number of installed locales to the bare minimum.

Ever since sys-libs/glibc-2.3.4.20040619-r2 has been in Portage, a
USE-flag called userlocales was provided to make sure only those locales
mentioned in /etc/locales.build are to be built and installed. As a
side-effect, this also leads to a much faster emerge of glibc, obviously.

Activate the userlocales USE flag especially for |
glibc

echo "sys-libs/glibc userlocales" >> /etc/portage/package.use

Now specify the locales you want to be able to use: (/etc/locales.build)

The format of the locales is described in the file itself.               
en_US/ISO-8859-1                                                         
en_US.UTF-8/UTF-8                                                        
de_DE/ISO-8859-1                                                         
de_DE@euro/ISO-8859-15                                                   
de_DE.UTF-8/UTF-8    

For further information about locale-handling make sure you read our
Gentoo Linux Localization Guide.

Another interesting tool is app-admin/localepurge which can clean out any
installed man-page or info-file in languages you don’t need on your
system. You should read the man-page to localepurge in any case, and
configure languages you intend to keep in /etc/locale.nopurge.

By the way, if you want to prohibit the installation of all man-pages,
info-files or documentation, for example when space on your disk is
severely limited, you can add noman, nodoc and/or noinfo to FEATURES in
your /etc/make.conf.

Generem RSS d’un directori cada cop que aquest canvii

Reading time: 4 – 6 minutes

Per temes interns de la xarxa d’eBosc, m’interessava generar un fitxer RSS cada vegada que un directori canvies el seu contingut. A continucació us explico com vaig resoldre el tema.

Per temes interns de la xarxa d’eBosc, m’interessava generar un fitxer RSS cada vegada que un directori canvies el seu contingut. A continucació us explico com vaig resoldre el tema.

Així doncs vaig instal·lar-me el dnotify que em permetia executar una comanda cada cop que es modifiques un directori i el podia deixar corrent com un dimoni (paràmetre -b a patir de la versió 0.18).

dnotify -b -C /directory -e /path/generarRSS.sh

Si detecta que s’ha creat un nou fitxer al directori, es llença la comanda indicada, en aquest cas és un petit shell script que m’he fet, amb el següent contingut:

#!/bin/sh
ls --color=no --sort=time -1 /path/ | /root/bin/rsspipe.py --limit 20 --url "http://localhost/" --title="Llistat del directori" --link="http://localhost/" --description="Descripció del contingut del directori" /path/fitxer.xml

Com podeu veure l’únic que faig és un llistat ordenat per temps i després envio la sortida del llistat del directori al programa RSSPipe. En realitat aquest és una versió que m’he modificat jo, ja que la versió original no feia tot el que volia. Malgrat no tinc ni idea de python sense mirar cap manual m’he pogut modificar el codi, ja que era fàcil deduir que s’havia de tocar perquè fes el que jo volia.

Codi de la meva versió del RSSPipe:

#! /usr/bin/python
#
# rsspipe.py v0
#   <http://rentzsch.com/code/rsspipe>
#   Copyright (c) 2004 Jonathan 'Wolf' Rentzsch.
#   Some rights reserved: <http://creativecommons.org/licenses/by/2.0/>
"""
Reads lines from stdin, outputting an RSS 0.92 file containing the last N lines as items.
usage example:
        tail -f /var/log/httpd/access_log|python rsspipe.py --title 'http traffic' httpd_access.xml
"""	
import sys, cgi
from optparse import OptionParser
	
parser = OptionParser()
parser.add_option("-t", "--title", dest="title", help="feed title", default="rsspipe")
parser.add_option("-l", "--link", dest="link", help="feed link", default="http://rentzsch.com/code/rsspipe")
parser.add_option("-d", "--description", dest="description", help="feed description", default="rsspipe")
parser.add_option("-i", "--limit", dest="limit", help="number of lines to export as RSS items", default=100)
parser.add_option("-u", "--url", dest="url", help="la url on es pot descarregar el fitxer", default="ftp://localhost/")
(options, args) = parser.parse_args()
	
outputFile = open(args[0],'w')
lineQueue = []
i=0;
while i<int(options.limit):
        readline = sys.stdin.readline()
        if readline != '':
                i=i+1;
                if len(lineQueue) >= int(options.limit):
                        del lineQueue[0]
                lineQueue.append(readline.rstrip())
	
                outputString = '<?xml version="1.0"?>\n<rss version="0.92">\n<channel>\n\t<title>' + options.title + '</title>\n\t<link>' + options.link + '</link>\n\t<description>' + options.description + '</description>\n';
                for theLine in lineQueue:
                        encodedLine = cgi.escape(theLine)
                        outputString = outputString + '<item>\n\t<title>' + encodedLine + '</title>\n\t<link>' + options.url + encodedLine + '</link>\n\t<description>' + encodedLine + '</description>\n</item>\n';
                outputString = outputString + '\n</channel>\n</rss>'
                outputFile.seek(0)
                outputFile.truncate(0)
                outputFile.write(outputString)
                outputFile.flush()	

Els canvis que he fet al RSSpipe són els següents:

• He afegit un nou paràmetre (-url) que serveix per indicar la ruta del fitxer. Ho necessito per referenciar com baixar el fitxer per exemple via HTTP o FTP.
• Encompte de fer que el programa s’executes infinitament li he dit que només sexecuti per generar les linies que ha de fer, no infinitament, com estava previst originalment.
• Dins de cada item el tag de link ara fa referència a la URL del paràmetre -url més l’item en qüestió, així el link ja és usable.
• He netejat la sortida del fitxer RSS que era molt lletja. Tot estava en una solia línia, he posat els salts de línia i ho he identat.

dnotify – Execute a command when the contents of a directory change

Reading time: < 1 minute

dnotify is a simple program that makes it possible to execute a command every time the contents of a specific directory change in linux. It is run from the command line and takes two arguments: one or more directories to monitor and a command to execute whenever a directory has changed. Options control what events to trigger on: when a file was read in the directory, when one was created, deleted and so on.

rsspipe: Pipe the command line to RSS

Reading time: 1 – 2 minutes

rsspipe is a simple python 2.3 script that reads stdin and outputs the last lines as an RSS 0.92 file, one RSS item (“headline”) per line.

It’s a general tool, but it may help if I give you a specific usage
example. I use it to keep track of recent referrers. I have a
long-running command like this:

tail -c 1000000000 -f access_log
|./weblog_parse -quiet referer
|./xuniq
|python rsspipe.py --title 'rentzsch.com referrers' referrers-rentzsch.xml

That’s one line — I inserted line breaks on the pipes to make it easier to follow. Let’s walk through it command-by-command:

• First I have tail read my entire log file (-c 1000000000) and continue reading it forever (-f).
• Those
  log lines are fed to a slightly modified version of the excellent
  weblog_parse, which extracts the referrer from the log line.
• The
  referrer lines are fed to xuniq, which is a lot like uniq except its
  input doesn’t need to be sorted first. It only outputs unique lines
  (lines it hasn’t seen before).
• Those unique referrers are transformed into a rolling RSS file for ongoing consumption. You can see it in action here.