Page 3 of 27‹ Previous 1 234 5 6 Next ›Last »

Dec 23

Raspberry PI and OpenWRT flash partition proposal and rescue boot support for embedded systems

Posted on 2015/12/23 by Oriol Rius

Reading time: 4 – 7 minutes

Introduction

Main target of this post is describe how to organize flash partitions and how to modify default OpenWRT boot sequence to support a flexible and powerful rescue mode for Raspberry PI based projects. Just to clarify the explaination. When OpenWRT is build on a flash card for Raspberry, there are only two partitions.

The first one is vFat partition with kernel, firmware and other configuration files; the second one is a ext4 partition with root filesystem. Boot sequence loads the kernel and then mount root partition and run the init script. If ext4 filesystem is corrupted or could not be mounted boot sequence is stoped and there is no solution without extracting the flash card.

Features

In this blog entry I’m going to describe a partition table and boot sequence strategy to avoid this kind of problems. Of course, there are other solutions to get similar results but I think this one is simple and powerful at the same time.

Summarizing features of this solution:

reduce risk when using intensive writing app
reduce damage risk on flash memories
fail-safe mode pressing a button
support application upgrades using opkg packages
support operative system upgrades using opkg packages

This solution proposal assume:

wear leveling protection solved by flash card
button connected to GPIO pins

The idea

Raspberry PI requires a vfat partition as its first flash partition where there are several required files for booting process, this is a bootloader substitution. For example, in that partition there are files like: start*.elf and bootcode.bin which are the GPU firmware and bootloaders. Another key file is kernel.img; this is the kernel used for booting. Bootloader parameters for kernel booting are in a file called cmdline.txt and firmware parameters are set in config.txt.

At this point the most important think to take into account is kernel.img file and cmdline parameters. Because kernel is loaded and executed by default with cmdline parameters set. When kernel boot process finishes root filesystem and init process sequence will be figured out from cmdline parameters.

At this point take a look on proposed partition table could be useful: (spaces are just as a reference, use what you need)

p1 - vfat (~50MB)
p2 - ext4 - operative system base (read-only) (~150MB)
p3 - ext4 - operative system (read-write) (~250MB)
p4 - logical partition
  p4.1 - ext4 - your_application files (usually read-only)
  p4.2 - ext4 - your_application data (usually read-write)

Fail-safe boot process key is partition p2 where a minimal OpenWRT installation with a modified init sequence is found. Main idea here is detect if a GPIO shortcut is done, usually this is done just pressing a physical button and you can interact with the user emitting some beep, for example, you can tell the user when you are waiting for button press using a beep and then emit two beeps when button press is detected or nothing if no button is pressed in 3 seconds. Finally the idea is detect if you need a regular boot or a fail-safe boot.

My suggestion for minimal OpenWRT is a small footprint installation of OpenWRT without kernel modules, just the monolitic kernel loaded. Then reduce init sequence to the minimum and add fail-safe logic (GPIO button capture); if button is pressed stop boot sequence and give a shell to the user. Regular way will be invoke init file of the rootfs (p3 in the partition table).

I think the idea is simple and the complexity is reduced in two parts both of them are the init file. To be more precise the p2 partition table has its own init file and p3 the other one. p2 init file load the minimum hardware to control button and give rescue environment when it’s needed. And p3 init file mounts read-write partition and the regular filesystem with regular boot processes and all kind of stuff that you need.

Final notes

I know this is not a very practical post, but my intention is only share some ideas that I have in mind. I spend most of my time designing architectures and I think this is a very powerful architecture of a boot sequence for some professional projects based on Raspberry PI and OpenWRT.

The best way to do what I describe in this post is putting p2 in a initrd file which is referenced in kernel parameters. Because then all read-only system is a RAM partition and rootfs init file has the PID 1 dropping dual-init file complexity. But I decided to modify this part because in the past I had some problems creating initrd files specially when required space for that partition is bigger than RAM. Anyway it’s important to take in account that initrd files has the same purpose as the proposed p2 partition.

Useful links

Building a JFFS2 OpenWRT Raspberry Pi image with hard float support – useful to understand how to modify default OpenWRT boot squence in a Raspberry PI
ramfs, rootfs and initramfs – understanding RAM filesystems
How to create initrd.gz to boot OpenWRT-x86 from USB flash – inspirational script about how to create p2 partition content or initrd file
OpenWRT: Boot/Init Requirements – what the initscripts must and should do for the new init system being developed for OpenWrt

Jul 13

Conferència Summer Camp Garrotxa 2015: SmartHome usant Arduino, Raspberry PI i més

Posted on 2015/07/13 by Oriol Rius

Reading time: 1 – 2 minutes

Aquest cap de setmana vaig tornar a donar la meva conferència sobre “SmartHome” en aquesta ocasió vaig introduïr la novetat del sistema de feedback basat en “Tasker + Auto-notification”, a més de l’execusió de tasques basades en “crontab”. També he inclòs un petit avanç sobre el nou projecte en el que estic treballant per tal d’integrar les dades del descalcificador dins del OpenHAB usant una webcam i OpenCV per processar les imatges optingudes.

Agraïr al Xavi, Gerardo i la Laura per haver-me donat aquesta oportunitat. Tant el SCG15 com el SAX2015 són events molt familiars a més l’entorn és espectacular. Tot un descobriment del que malgrat tenir referències no havia pogut disfrutar en primera persona.

May 08

Free dynamic DNS service

Posted on 2015/05/08 by Oriol Rius

Reading time: < 1 minute A long time ago there were several free dynamic DNS services but nowadays it's difficult to find one of them. And when you find the service usually you have some important restrictions like: number of updates per day or only few subdomains per account. But in the end I found a good free service of that, it's part of the project guifi.net and is called: Qui; you only need a guifi.net account to use the service and it’s really simple and clear. From my part the compatibility with “ddclient” and the “mikrotik” script are really useful and I want tu highlight this functionality.

Apr 19

Conference: SmartHome using Arduino, Raspberry PI and more

Posted on 2015/04/19 by Oriol Rius

Reading time: 1 – 2 minutes

I did this conference twice but this time it’s in English as I said before in Catalan, this is how I’m getting real a childhood dream. Thanks to open hardware like Arduino and Raspberry PI and open source software like OpenHab I have a smarthome with several automations. Last Thursday I spoke in “IoT Barcelona meetup” sharing my personal experience automating my house.

If you lost it and you want to watch the presentation video is here:

and the presentations slides are here:

Smart home using arduino, raspberry pi and more from Oriol Rius

Mar 12

VMWare trick: add new hard drive without restarting the virtual machine

Posted on 2015/03/12 by Oriol Rius

Reading time: < 1 minute As simple as that, if you add a new virtual hard drive using VMWare in your virtual machine with Linux and you want to force the re-scan SCSI bus to see the new hard drive when you run, for example: "fdisk -l". You can force the SCSI bus re-scan with:

# take into account that your new hard drive could be added in different point than “host0”
echo “- – -” > /sys/class/scsi_host/host0/scan

Feb 18

Conferència: SmartHome usant Arduino, Raspberry PI i més

Posted on 2015/02/18 by Oriol Rius

Reading time: 1 – 2 minutes

El dia 12 de febrer al vespre vaig fer una conferència a la FIB (Facultat d’Informàtica de Barcelona) dins de la UPC (Universitat Politècnica de Catalunya). En aquesta xerra vaig estar explicant com vaig convertint en realitat un somni que vaig tenir quan era petit, estic parlant de domotitzar la casa. Gràcies a enginys com l’Arduino i la Raspberry PI a més d’aplicacions com l’OpenHab podem fer assequible allò que si anem a grans marques comercials es fa caríssim pel meu pressupost.

En aquest enllaç podeu trobar les transparències de: SmartHome usant Arduino Raspberry i més i el video el teniu disponible al servidor de la FIB.

Ara també teniu disponible el video a youtube:

i podeu veure les transparències des d’aquest mateix post:

Espero els vostres feedbacks als comentaris, desitjo que ús sigui útil.

Feb 10

Turn on virtual machines in VMWare ESXi

Posted on 2015/02/10 by Oriol Rius

Reading time: < 1 minute Next commands are very useful when you don't have access to the vSphere UI and you have to access to VMWare Hypervisor using SSH or console:

# get the list of virtual machines
vim-cmd vmsvc/getallvms

# get the state of a VM with #id: VM_ID
vim-cmd vmsvc/power.getstate VM_ID

# turn on the virtual machine with #id: VM_ID
vim-cmd vmsvc/power.on VM_ID

Another option to turn on the virtual machine using an Ansible playbook:

- hosts: vmware
  gather_facts: false
  tasks:
    - vsphere_guest:
        vcenter_hostname: "X.X.X.X"
        username: "{{ hostvars[inventory_hostname].ansible_ssh_user|quote }}"
        password: "{{ hostvars[inventory_hostname].ansible_ssh_pass|quote }}"
        guest: "NAME_OF_THE_VM"
        state: "powered_on"
      delegate_to: localhost

Jan 30

New Ansible Role uploaded to Ansible Galaxy

Posted on 2015/01/30 by Oriol Rius

Reading time: 2 – 2 minutes

A long time ago I wrote an entry post about how to set up the SMTP in linux boxes using a relay system you can find the post here: Relay mail from your server without MTA. Remember that SSMTP is not a SMTP service for your system but it’s more than enough for all servers that don’t work as a mail servers. Historically Unix/Linux uses sendmail command to send system notifications but usually this mails are lost because system configurations are not completed. My advice in this sense is use SSMTP.

In the past I used to use SSMTP with a GMail account but security constraints in Google mail services make it difficult to configure today. The new alternative is set up a free Mandrill account as a relay host. Mandrill is a Mailchimp service that allows you to send a lot of emails without problems and there is a free account that allows to send up to 12.000 mails per month free, more than enough usually. If you don’t know how to set up a Mailchimp account the best option to learn how to do it is follow the support documentation it’s very good IMHO.

When you have a lot of linux machines to administer you need something fastly replicable. As you know use Ansible is a very good option. Then I developed a new Ansible role to set up Mandrill accounts to SSMTP services massively using Ansible.

The Ansible role has been uploaded here: ssmtp-mandrill and the source code of the roles is in my github. Remember to install the role in your Ansible is easy:

ansible-galaxy install oriol.rius.ssmtp-mandrill

Then you only need to create your own playbook using the role and don’t forget to setup the variables with the Mandrill account settings.

Jan 29

Ansible and Windows Playbooks

Posted on 2015/01/29 by Oriol Rius

Reading time: 3 – 5 minutes

Firstly let me introduce a Windows service called: “Windows Remote Manager” or “WinRM”. This is the Windows feature that allows remote control of Windows machines and many other remote functionalities. In my case I have a Windows 7 laptop with SP1 and PowerShell v3 installed.

Secondly don’t forget that Ansible is developed using Python then a Python library have to manage the WinRM protocol. I’m talking about “pywinrm“. Using this library it’s easy to create simple scripts like that:

#!/usr/bin/env python

import winrm

s = winrm.Session('10.2.0.42', auth=('the_username', 'the_password'))
r = s.run_cmd('ipconfig', ['/all'])
print r.status_code
print r.std_out
print r.std_err

This is a remote call to the command “ipconfig /all” to see the Windows machine network configuration. The output is something like:

$ ./winrm_ipconfig.py 
0

Windows IP Configuration

   Host Name . . . . . . . . . . . . : mini7w
   Primary Dns Suffix  . . . . . . . : 
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : ymbi.net

Ethernet adapter GigaBit + HUB USB:

   Connection-specific DNS Suffix  . : ymbi.net
   Description . . . . . . . . . . . : ASIX AX88179 USB 3.0 to Gigabit Ethernet Adapter
   Physical Address. . . . . . . . . : 00-23-56-1C-XX-XX
   DHCP Enabled. . . . . . . . . . . : Yes
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::47e:c2c:8c25:xxxx%103(Preferred) 
   IPv4 Address. . . . . . . . . . . : 10.2.0.42(Preferred) 
   Subnet Mask . . . . . . . . . . . : 255.255.255.192
   Lease Obtained. . . . . . . . . . : mi�rcoles, 28 de enero de 2015 12:41:41
   Lease Expires . . . . . . . . . . : mi�rcoles, 28 de enero de 2015 19:17:56
   Default Gateway . . . . . . . . . : 10.2.0.1
   DHCP Server . . . . . . . . . . . : 10.2.0.1
   DHCPv6 IAID . . . . . . . . . . . : 2063606614
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-15-F7-BF-36-xx-C5-xx-03-xx-xx
   DNS Servers . . . . . . . . . . . : 10.2.0.27
                                       10.2.0.1
   NetBIOS over Tcpip. . . . . . . . : Enabled
...

Of course, it’s possible to run Powershell scripts like the next one which shows the system memory:

$strComputer = $Host
Clear
$RAM = WmiObject Win32_ComputerSystem
$MB = 1048576

"Installed Memory: " + [int]($RAM.TotalPhysicalMemory /$MB) + " MB"

The Python code to run that script is:

#!/usr/bin/env python

import winrm

ps_script = open('scripts/mem.ps1','r').read()
s = winrm.Session('10.2.0.42', auth=('the_username', 'the_password'))
r = s.run_ps(ps_script)
print r.status_code
print r.std_out
print r.std_err

and the output:

$ ./winrm_mem.py 
0
Installed Memory: 2217 MB

In the end it’s time to talk about how to create an Ansible Playbook to deploy anything in a Windows machine. As always the first thing that we need is a hosts file. In the next example there are several ansible variables needed to run Ansible Windows modules on WinRM, all of them are self-explanatory:

[all]
10.2.0.42

[all:vars]
ansible_ssh_user=the_username
ansible_ssh_pass=the_password
ansible_ssh_port=5985 #winrm (non-ssl) port
ansible_connection=winrm

The first basic example could be a simple playbook that runs the ‘ipconfig’ command and registers the output in an Ansible variable to be showed later like a debug information:

- name: test raw module
  hosts: all
  tasks:
    - name: run ipconfig
      raw: ipconfig
      register: ipconfig
    - debug: var=ipconfig

The command and the output to run latest example:

$ ansible-playbook -i hosts ipconfig.yml 

PLAY [test raw module] ******************************************************** 

GATHERING FACTS *************************************************************** 
ok: [10.2.0.42]

TASK: [run ipconfig] ********************************************************** 
ok: [10.2.0.42]

TASK: [debug var=ipconfig] **************************************************** 
ok: [10.2.0.42] => {
    "ipconfig": {
        "invocation": {
            "module_args": "ipconfig", 
            "module_name": "raw"
        }, 
        "rc": 0, 
        "stderr": "", 
        "stdout": "\r\nWindows IP Configuration\r\n\r\n\r\nEthernet adapter GigaBit 
...
        ]
    }
}

PLAY RECAP ******************************************************************** 
10.2.0.42                  : ok=3    changed=0    unreachable=0    failed=0

As always Ansible have several modules, not only the ‘raw’ module. I committed two examples in my Github account using a module to download URLs and another one that runs Powershell scripts.

My examples are done using Ansible 1.8.2 installed in a Fedora 20. But main problems I’ve found are configuring Windows 7 to accept WinRM connections. Next I attach some references that helped me a lot:

pywinrm Github project notes., easy to trace errors than Ansible.
Ansible notes about Windows.
How to enable WinRM via Group Policy by Alan Burchill

If you want to use my tests code you can connect to my Github: Basic Ansible playbooks for Windows.

Oct 30

Conferència al Tecnocampus: Internet of Things (IoT) low cost

Posted on 2014/10/30 by Oriol Rius

Reading time: 1 – 2 minutes

Després d’arribar corrent de Cardiff on era aquest dilluns el dimarts vaig arribar a temps per fer la presentació “Internet of Things low cost” als companys del Tecnocampus de Mataró; per cert, unes intal·lacions brutals res a veure amb la EUPMT on vaig estudiar jo fa una colla d’anys. Jo encara diria més, molt millor que no pas la gran Universitat de Cardiff on vaig ser el dilluns.

Pel que fa a la xerrada agraïr a tothom que hi va assistir, a continuació adjunto les transparències pels que vareu ser-hi i pels que no.

~~Properament pujaré el video, o la part, del video que es va poder gravar. El problema és que són diversos gigues i he de deixar l’ordinador renderitzant cosa que encara no he pogut fer.~~

El video no conté tota la presentació però té una part important del contingut, així que desitjo que ús sigui útil:

Per cert, una gran part de la presentació esta reciclada de la conferència: Conferència: La revolució dels mini-PC: Raspberry PI, Arduino i més

oriolrius.cat

Oriol Rius :: Technology : Crecimiento Personal : Mussol

Tag Archives: linux

Raspberry PI and OpenWRT flash partition proposal and rescue boot support for embedded systems

Introduction

Features

The idea

Final notes

Useful links

Conferència Summer Camp Garrotxa 2015: SmartHome usant Arduino, Raspberry PI i més

Free dynamic DNS service

Conference: SmartHome using Arduino, Raspberry PI and more

VMWare trick: add new hard drive without restarting the virtual machine

Conferència: SmartHome usant Arduino, Raspberry PI i més

Turn on virtual machines in VMWare ESXi

New Ansible Role uploaded to Ansible Galaxy

Ansible and Windows Playbooks

Conferència al Tecnocampus: Internet of Things (IoT) low cost