Ahir al vespre vaig fer una conferència a la FIB (Facultat d’Informàtica de Barcelona) dins de la UPC (Universitat Politècnica de Catalunya). En aquesta xerra vaig estar explicant què és i en que es diferència Arduino i Raspberry PI. A més de presentar tot un conjunt de solucions alternatives i experiències en el tema.
I have two full-HD displays as a desktop and I want to record the second one of them while I record mic too. The output format of the record has to be MKV with h264 as video codec and AAC as audio codec.
After some tests with VLC and FFMPEG finally I get the solution with this command:
when I finish recording the clip I have to convert the audio channel because if I try to convert the audio format while I’m recording the audio channel I have delays or sync problems with video channel.
Some times schemas and snippets don’t need large descriptions. If you think this is not enough in this case tell me and I’m going to add explanations.
Using a python library called kombu as an abstraction to talk with AMQP broker we are going to develop different message routes setting each type of Exchange. As a backend I used RabbitMQ with default configuration.
Kombu implements SimpleQueue and SimpleBuffer as simple solution for queues with exchange of type ‘direct’, with the same exchange name, routing key and queue name.
Pusher:
from kombu import BrokerConnectionconnection = BrokerConnection("amqp://guest:guest@localhost:5672//")queue = connection.SimpleQueue("logs")payload = { "severity":"info", "message":"this is just a log", "ts":"2013/09/30T15:10:23" }queue.put(payload, serializer='pickle')queue.close()
Nowadays last version of browsers support websockets and it’s a good a idea to use them to connect to server a permanent channel and receive push notifications from server. In this case I’m going to use Mosquitto (MQTT) server behind lighttpd with mod_websocket as notifications server. Mosquitto is a lightweight MQTT server programmed in C and very easy to set up. The best advantage to use MQTT is the possibility to create publish/subscriber queues and it’s very useful when you want to have more than one notification channel. As is usual in pub/sub services we can subscribe the client to a well-defined topic or we can use a pattern to subscribe to more than one topic. If you’re not familiarized with MQTT now it’s the best moment to read a little bit about because that interesting protocol. It’s not the purpose of this post to explain MQTT basics.
A few weeks ago I set up the next architecture just for testing that idea:
weboscket gateway to mosquitto mqtt server with javascrit mqtt client
The browser
Now it’s time to explain this proof of concept. HTML page will contain a simple Javascript code which calls mqttws31.js library from Paho. This Javascript code will connect to the server using secure websockets. It doesn’t have any other security measure for a while may be in next posts I’ll explain some interesting ideas to authenticate the websocket. At the end of the post you can download all source code and configuration files. But now it’s time to understand the most important parts of the client code.
Last part is very simple, the client connects to the server and links some callbacks to defined functions. Pay attention to ‘useSSL’ connect option is used to force SSL connection with the server.
There are two specially interesting functions linked to callbacks, the first one is:
function onConnect() { client.subscribe("/news/+/sport", {qos:1,onSuccess:onSubscribe,onFailure:onSubscribeFailure});}
As you can imagine this callback will be called when the connections is established, when it happens the client subscribes to all channels called ‘/news/+/sports’, for example, ‘/news/europe/sports/’ or ‘/news/usa/sports/’, etc. We can also use, something like ‘/news/#’ and it will say we want to subscribe to all channels which starts with ‘/news/’. If only want to subscribe to one channel put the full name of the channel on that parameter. Next parameter are dictionary with quality of service which is going to use and links two more callbacks.
The second interesting function to understand is:
function onMessageArrived(message) { console.log("onMessageArrived:"+message.payloadString);};
It’s called when new message is received from the server and in this example, the message is printed in console with log method.
The server
I used an Ubuntu 12.04 server with next extra repositories:
After installation it’s very easy to run mosquitto in test mode, use a console for that and write the command: mosquitto, we have to see something like this:
# mosquitto1379873664: mosquitto version 1.2.1 (build date 2013-09-19 22:18:02+0000) starting1379873664: Using default config.1379873664: Opening ipv4 listen socket on port 1883.1379873664: Opening ipv6 listen socket on port 1883.
The configuration file for lighttpd in testing is:
Remember to change ‘ssl.pemfile’ for your real certificate file and ‘server.name’ for your real server name. Then restart the lighttpd and validate SSL configuration using something like:
openssl s_client -host ns.example.tld -port 443
You have to see SSL negotiation and then you can try to send HTTP commands, for example: “GET / HTTP/1.0” or something like this. Now the server is ready.
The Test
Now you have to load the HTML test page in your browser and validate how the connections is getting the server and then how the mosquitto console says how it receives the connection. Of course, you can modify the Javascript code to print more log information and follow how the client is connected to MQTT server and how it is subscribed to the topic pattern.
If you want to publish something in MQTT server we could use the CLI, with a command mosquitto_pub:
mosquitto_pub -h ns.example.tld -t '/news/europe/sport' -m 'this is the message about european sports'
Take a look in your browser Javascript consle you have to see how the client prints the message on it. If it fails, review the steps and debug each one to solve the problem. If you need help leave me a message. Of course, you can use many different ways to publish messages, for example, you could use python code to publish messages in MQTT server. In the same way you could subscribe not only browsers to topics, for example, you could subscribe a python code:
Pay attention to server port, it isn’t the ‘https’ port (443/tcp) because now the code is using a real MQTT client. The websocket gateway isn’t needed.
The files
mqtt.tar.gz – inside this tar.gz you can find all referenced files
Another title for this post could be: “Getting audio from video clips”. Because you could do it with MP4 (Mpeg4), WebM, Mov, FLV, etc. We are going to use ffmpeg to that:
I just solved a very specific problem and I have to write some notes here to remember the solution. Given a RTMP source we have to stream the content to Apple devices like iPad, iPhone and iPod because RTMP couldn’t be played using Safari browser.
If we need to play streaming on Apple devices the best solution is convert it to HLS and publish generated files using HTTP server.
To solve this issue I use rtmpdump and vlc. Firstly rtmpdump gets video stream from source. Secondly the stream is sent to vlc and finally vlc transcodes de video and audio and outputs small .ts files and one .m3u8 index file.
Celery logs are colorized by default, the first big idea is disable color logs. It’s as easy as setting ‘CELERYD_LOG_COLOR’ to ‘False’ in ‘celery.conf’. The code could be something like this:
celery.conf.update('CELERYD_LOG_COLOR' = False)
Secondly we need a function where we set up a new handler and other settings to celery logging system. For example, the code could be:
from __future__ import absolute_importfrom logging import BASIC_FORMAT, Formatterfrom logging.handlers import SysLogHandlerfrom celery.log import redirect_stdouts_to_loggerdef setup_log(**args): # redirect stdout and stderr to logger redirect_stdouts_to_logger(args['logger']) # logs to local syslog hl = SysLogHandler('/dev/log') # setting log level hl.setLevel(args['loglevel']) # setting log format formatter = Formatter(BASIC_FORMAT) hl.setFormatter(formatter) # add new handler to logger args['logger'].addHandler(hl)
Pay attention to ‘redirect_stdouts_to_logger’ it’s used to send all outputs like print’s or something else to syslog.
Thirdly we want to use those settings in our celery tasks, then we have to connect ‘setup_log’ code to some celery signals. Those signals are launched when ‘task_logger’ and ‘logger’ are configured. To connect signals:
from celery.signals import after_setup_task_logger, after_setup_loggerafter_setup_logger.connect(setup_log)after_setup_task_logger.connect(setup_log)
Fourthly we have to get the ‘logger’, we can have more than one if we are interested in records with task context or without it. For example:
Finally we only have to use those loggers with common methods DEBUG, INFO, WARN, ERROR and CRITICAL:
@celery.taskdef the_task(): logger.info('this is a message without task context') logger_with_task_context.debug('this record will have the prefix "name_of_the_task_to_be_recorded_in_logs" in syslog')
I just want to share with you a small and powerful Windows tool I found in my last trip to US. The best feature IMHO is that permits to change the MAC address of your NIC interface without rebooting, safely and fast. It could be useful when you have a limit time to connect to internet in a free Wi-Fi network; after changing your MAC address you should be like a new device. If you have to do something like this, remember to remove the browser cookies.
Other interesting features of this tool is network presets. You can change your NIC settings very fast just changing a preset profile. As you can see in next screenshot it has a simple chart of your real time network traffic. And finally I want to stand out you can see all your network devices configuration very fast.
