Photoframe Hack

Sometimes you just want to get something done. Something for yourself.

You do not intend it to be reused, or even pretty.

You build a tool.

My tool was a photoframe with some basic overlays. I wanted the family calendar, some weather information (current temperature + forecast), time, and the next bus heading for the train station.

To make this acceptable in a home environment, I built it as a photoframe. You can find the sources in the hassframe-ui repository on my github.

A hidden feature is that if you tap the screen, a home automation control panel slides up. That way you can control all the lights, as well as heat in the garage and an AC in the bedroom. Very convenient.

All this is built using QML. Three somewhat useful models are available:

  • IcalModel, taking a URL and parsing whatever it gets back as ICAL data. It is a very naive parser and does not care about things such as time zones and other details.
  • YrWeatherModel, uses yr.no‘s public APIs to pull out a weather forecast for a given location.
  • ButStopModel, uses the APIs from resrobot to look for departures to the train station from two bus stops close to my home and then merge the results into a model.

I also have a bunch of REST calls to my local home assistant server. Most of these reside in the HassButton class, but I also get the current temperature from there. These are hardcoded for my local network, so needs refactoring to be used outside of my LAN.

All of these interfaces require API keys of one kind or another – be it a proper key, or a secret URL. These are pulled from environment variables in main.cpp and then exposed to QML. That way, you can reuse the components without having to share your secrets.

All in all the code is quite hacky. Especially main.qml. I refactor out parts from there now and then, but the photoframe works, so its not anything that I prioritize.

Currently it runs on a Raspberry Pi on top of Raspbian. I want to build an optimized Yocto image making it less hacky and more pre-packaged. Perhaps there will be a rainy day this summer and I’ll get around to it. Burkhard has prepared the instructions needed over at embedded use.

The Cost of no Architecture

Like many others, I enjoy various reverse engineering and tear-down stories. Personally, I mean things like iFixit tear-downs and Ken Shirriff’s blog, so I started following this tweet thread by foone.

This continues with another tweet sequence about getting software running on the remote control. Having enjoyed these tweets, I started thinking.

The Harmony remotes are quite expensive in my mind. I can’t find any exact numbers for the number of sold devices, but I found this 2018 Q4 earnings report. Looking at the net sales, I guess the remotes are either “Tablets & Other Accessories” or “Smart Home”. They represent sales net sales of ~107 and ~89 MUSD over 12 months. Let’s pick the lower number and just look at magnitudes. The Harmony 900 seems to have retailed for ~350 USD back when it was new. So, if all the Smart Home stuff was harmonies, we’re looking at 250k units over a year. So I’m guessing the magnitude is around 10k – 100k units annually – but the Harmony 900 is from 2013, so I assume that it sold closer to the lower number, if not below. The market was new and so on.

Then we look at the tweets again. What have we got? Let’s put aside security issues, unencrypted communications, and other clear mistakes and just look at how the device is built.

Flash to drive the UI, double web servers on-board, Lua, QNX and what not. A 233 MHz CPU and ~64MB of FLASH – for a remote control. From an engineering perspective, this sounds like a fun system to work on – from an architecture perspective, it looks like a ball of mud.

Back in 2013, QNX might have been a good choice compared to Linux. Today, with Yocto and similar tools for developing embedded Linux systems, it feels like an odd choice to add a license cost to such a device. But no biggie. Back in the day this was not an unreasonable choice (and still isn’t for certain applications).

The Flash stuff. There were alternatives back in 2013, but sure, there were plenty of developers at hand and things like Qt QML was still probably a bit clunky (I can’t recall the state of it back then – it required OpenGL ES, which I guess was a big ask back then).

But the mix of techniques and tools. The on-board web servers. The complexity of a small system and the costs it brings to maintenance and testability. If this is the foundation for Harmony remotes and a platform that has been used for the better past of the past decade, I wonder if the added engineering costs for architecture the platform to be more optimized early on would not have paid off in lower maintenance costs, as well as lower hardware costs.

I know how it is when you’re in a project. The deadline is there in big writing on one of the walls. You can get something working by stringing what you have together with duktape and glue. The question I’m asking myself is more along the lines of how do we run embedded systems engineering projects? Where did we go wrong? Why don’t we prioritize the thinking and refactoring over the just-get-this-thing-out-of-the-door?

The answer is time to market and such, but over a decade of building on a ball of mud, the economical numbers start adding up in favour for the better engineered product. For continuous improvement. For spending time thinking about how to improve the system as a whole.

The Embedded Talks

The foss-north conference strives to have an assortment of various talks. The point is that visitors should see something unexpected and that the conference should attract all types of visitors to ensure that we as a community can meet across various industries and problem spaces.

This time I’ve selected three talks about embedded systems from foss-north 2020. The talks touch on building embedded systems around Linux. If your reader does not show you the embedded videos, make sure to follow the actual page or go to our conf.tube channel to see all the contents.

First out was Ron Munitz talk on understanding and building minimal Linux systems. This talk proved to be a real deep dive into the Linux kernel – including setting up a debugger to the kernel itself.

The next embedded speaker on the program was Chris Simmonds. He discussed if going with Yocto or Debian is best for your embedded Linux project. This an interesting topic – how much is customization worth compared to other aspect such as build-time.

The embedded set of talks ended with Drew Fustini talking about running Linux on the RISC-V. This talk dives deep into the hardware part of embedded systems, but also Linux. By being able to run Linux on RISC-V, which is open hardware, we are very close to an completely open eco-system.

The three talks are already available on conf.tube, and the presentation material can be found by following the links to each speaker. For those of you who prefer YouTube, the talks will be made available shortly on the foss-north channel. Subscribe to get notified when they are.

foss-north call for papers

The summer is flying by and it is already August. The call for papers for foss-north IoT and Security Day is still open for a few more days, so make sure to get your talk in. We are looking for talks touching on connected embedded devices and how to do them securely.

The foss-north IoT and Security Day will be a one day event, October 21st, in central Stockholm. The venue, WTC, is located right by the central train station, so it is very easy to get there. Tickets will be made available soon. Make sure to save the date!

foss-north 2019: Community Day


I don’t dare to count the days until foss-north 2019, but it is very soon. One of the changes to this year is that we expand the conference with an additional community day.

The idea with the community day here is that we arrange for conference rooms all across town and invite open source projects to use them for workshops, install fests, hackathons, dev sprints or whatever else they see fit. It is basically a day of mini-conferences spread out across town.

The community day is on April 7, the day before the conference days, and is free of charge.

This part of the arrangements has actually been one of the most interesting ones, as it involves a lot of coordination. I’d like to start by thanking all our room hosts. Without them, the day would not be possible!

The other half of the puzzle is our projects. I am very happy to see such a large group of projects willing to try this out for the first time, and I hope for lots and lots of visitors so that they will want to come back in the future as well.

The location of each project, as well as the contents of each room can be found on the community day page. Even though the day is free of charge, some of the rooms want you to pre-register as the seats might be limited, or they want to know if they expect five or fifty visitors. I would also love for you to register at our community day meetup, just to give me an indication of the number of participants.

Also – don’t forget to get your tickets for the conference days – and combine this with a training. We’re already past the visitor count of the 2018 event, so we will most likely be sold out this year!

foss-north 2019: Training Day

The 2019 incarnation of foss-north is less than a month away. This year we’re extending the conference in two directions: a training day and a community day. This time, I wanted to write about the training day.

The training day, April 10, is an additional day for those who want to extend
the conference with a day of dedicated training. I’m very happy to have two experienced and well known trainers on side: Michael Kerrisk and Chris Simmonds. Both has years of training experience.

Michael will teach about the details in dynamic linking. The topic may seem trivial, but when you start scratching the surface, there are a lot of details to discover such as how to handle version compatibility, how symbol resolution really works, and so on. You can read more about the Building and Using Shared Libraries on Linux training here.

Chris will run a getting started with embedded Linux training. Using BeagleBone Black devices the participants will learn how to build linux for the target, how to get devices such as GPIO and i2c working and more. You can read more about the Fast Track to Embedded Linux training here.

The best part of enrolling for training at foss-north is that you also get full access to the two conference days, and that you help fund the conference itself. If you are interested, check out the tickets page.

QML Weather

I recently took some time to develop a photo frame style home automation control panel. The idea is to control some common tasks of my home assistant setup from a panel instead of having to rely on the phone. To hide the panel, it currently act as a photo frame until touched.

The build is based on a Raspberry Pi 2 with the official touch screen attached and a USB wifi dongle. Nothing fancy, but still good enough.

One of the features that I wanted was a weather forecast, so I decided to use Yr’s xml weather as a base for this. The result is the YrWeatherModel QML item.

The weather forecast overlay.

The presentation side of things is the fairly straight forward piece of QML shown below, resulting in the overlay shown above.

Row {
    anchors.bottom: dateText.bottom
    anchors.right: parent.right
    anchors.rightMargin: 40

    spacing: 20
    Repeater {
        delegate: Column {
            spacing: 2
            Text {
                anchors.horizontalCenter: parent.horizontalCenter
                color: "white"
                font.pixelSize: 16
                font.bold: true
                text: {
                    switch (period) {
                    case 0:
                        "00 - 06"
                        break;
                    case 1:
                        "06 - 12"
                        break;
                    case 2:
                        "12 - 18"
                        break;
                    case 3:
                    default:
                        "18 - 00"
                        break;
                    }
                }
            }
            Image {
                anchors.horizontalCenter: parent.horizontalCenter
                source: symbolSource
            }
            Text {
                anchors.horizontalCenter: parent.horizontalCenter
                color: "white"
                font.pixelSize: 16
                font.bold: true
                text: precipitation + "mm"
            }
            Text {
                anchors.horizontalCenter: parent.horizontalCenter
                color: "white"
                font.pixelSize: 16
                font.bold: true
                text: temperature + "°C"
            }
        }

        model: weatherModel.model
    }
}

This is followed by the model itself, and a small notice of the data source.

YrWeatherModel {
    id: weatherModel
    place: "Sweden/V%C3%A4stra_G%C3%B6taland/Alings%C3%A5s"
}

Text {
    anchors.bottom: parent.bottom
    anchors.right: parent.right
    anchors.bottomMargin: 5
    anchors.rightMargin: 40
    text: weatherModel.dataSourceNotice
    color: "white"
    font.pixelSize: 16
    font.italic: true
}

Diving into the model itself, we hit the interesting parts. The structure looks like this:

Item {
    id: root

    property alias model: weatherModel
    property int refreshHour: 1     // How often is the model refreshed (in hours)
    property int dataPoints: 6      // How many data points (max) are expected (in 6h periods)
    property string place           // Place, URL encoded and according to Yr web site, e.g. Sweden/V%C3%A4stra_G%C3%B6taland/Alings%C3%A5s
    readonly property string dataSourceNotice: "Data from MET Norway"

    ListModel {
        id: weatherModel
    }

    Timer {
        interval: 3600000 * root.refreshHour
        running: true
        repeat: true
        onTriggered: {
            _innerModel.reload();
        }
    }

    XmlListModel {
        id: _innerModel

        query: "/weatherdata/forecast/tabular/time"

        source: (place.length === 0)?"":("https://www.yr.no/place/" + root.place + "/forecast.xml")

        XmlRole { name: "period"; query: "string(@period)" }
        XmlRole { name: "symbol"; query: "symbol/string(@number)"; }
        XmlRole { name: "temperature"; query: "temperature/string(@value)"; }
        XmlRole { name: "precipitation"; query: "precipitation/string(@value)"; }

        onStatusChanged: {
            // ...
        }
    }
}

As you can see, the model consists of an inner model of the type XmlListModel. This model is refreshed by a timer (don’t refresh too often – you will most likely be auto-banned by Yr). At the top, there is also a ListModel that is the actual model used by the user interface.

The reason for the ListModel to exist is that I wanted to be able to limit how many data points I show. Each data point represents a six hour window, and I’d like 6 of them, i.e. one and a half day of forecasting.

The onStatusChanged handler in the XmlListModel takes care of this in the following for loop:

onStatusChanged: {
    if (status === XmlListModel.Ready)
    {
        for(var i = 0; i< root.dataPoints && i < count; ++i)
        {
            var symbol = get(i).symbol;
            var period = parseInt(get(i).period);
            var is_night = 0;

            if (period === 3 || period === 0)
                is_night = 1;

            weatherModel.set(i, {
               "period":period,
               "symbol":symbol,
               "symbolSource":"https://api.met.no/weatherapi/weathericon/1.1/?symbol=" + symbol + "&is_night=" + is_night + "&content_type=image/png",
                "temperature":get(i).temperature,
                "precipitation":get(i).precipitation
                });
        }
    }
    else if (status === XmlListModel.Error)
    {
        console.warn("Weather error")
        console.warn(errorString());
    }
}

As you can tell, this code has *very* limited error handling. It is almost as it has been designed to break, but it works. The code also shows how convenient it is to connect to online services via QML to build simple, reusable, models that can be turned into beautiful user interfaces.

Next time I have some free time, I’ll look at interfacing with the public transport APIs. Then I will have to deal with JSON data and make explicit XmlHttpRequest calls.

D-Link and the GPL

Recently my router died and to get netflix flowing again, I went out and got the cheapest router I could find in the vicinity. I ended up with a D-Link DIR-842 on discount for 499 SEK. In the box, I discovered this:

It tells me to go to D-Link’s page for GPL licensed software to get the source code. It also lets me write a request the source code on physical media for a nominal fee for the media and handling. Something I naturally did (being an engineer on vacation).

While waiting for a reply, let’s have a look at the online version. When entering the URL provided you have to click through an agreement that I understand what GPL and LGPL means and that the files distributed comes with no warranties (they spend more words saying this – read it if you want the details). Clicking “I Agree” I get a popup (back to the 90’s) asking me to register my product to enjoy all the benefits of doing so. At the same time the main window continues to a list of all D-Link products containing (L)GPL software – very nice.

The list of products is 24 pages long, so I searched for my model name (DIR-842) and clicked the resulting link. This got me to the following table:

I wonder what separates firmware version 2.0CN (China?) from version 3.0. Having 950MB of translation tables seems odd, so something else must be the case. As I have firmware version 2.02 for hardware revision B1 I decided to download that bundle consisting of 590 MB of open source code as a tarball. At least, that was what I expected. Apparently, I don’t only get the sources, I also get a test report – great!

The test report rar file contained a pdf documenting the tests. On November 16, 2016, test engineer Mason Wu carried out the GPL SC tests consisting of the steps Firmware upgrade, Firmware downgrade, Compile the Open Source Software Licenses code, Check list (Before test Open Source Software Licenses), Open Source Software Digital Signature check, and License file check. All tests passed – time for celebrations!

To be completely honest, this report tells me nothing, as I don’t know what has been tested or what the test cases do. The only thing I learn is that I just put a device with almost two years old software on as my interface towards the Internet…

So what is in the source code tarball? You can see the directory structure of the tarball in the picture below (I guess someone named Lisa created the tarball). First of all, there are some licenses, then the source code.

The source code is split into open source and private, where private is a set of prebuilt binaries, while open source comes with the whole source code. The open source software is licensed under the following licenses according to the LICENSE.txt:

  • GNU General Public License Version 2 (GPLv2.0)
  • GNU Lesser General Public License Version 2.1 (LGPLv2.1)
  • BSD 2-clause license
  • BSD 3-clause license
  • Apache License 2.0
  • zlib/libpng License
  • MIT License v2.0

Looking into the realtek SDK (rtl819x-SDK-v3.4.5.1) I found the base Linux system (under rtl819x-SDK-v3.4.5.1/rtl819x). This is open source software found outside the open source directory.

There does not seem to be a license for the proprietary stuff. Not for the realtek related directories (rtk_wifi_patch and rtl819x-SDK-v3.4.5.1), nor the directory named private. I’m not sure what that means from a licensing perspective. I guess it is complicated. The nice thing with this is that I should be able to rebuild a new firmware image from this.

Continuing down the rabbit hole I’m getting really worried. Remember that this is an internet facing device. There are so many things I want to point out, but I’m on vacation so I can’t dig through the whole source code. Here are some snippets:

  • The Makefiles outputs “It’s builded” when done. Kind of cute.
  • Building is supported on CentOS 5.9 (32-bit version) with GCC 4.1.2 20080704 (Red Hat 4.1.2-54). This is a release from 17 January 2013.
  • Building has to be done as root.
  • For the proprietary stuff, there are .c.dep files showing what source files where used and their dependencies. Also, some headers are included without copyright information.
  • The open source versions are really old. Some highlights:
    • Samba 3.0.24 – from 2003. The CVE list for Samba is scary – this is a piece of software that should be updated.
    • Kernel 2.6.30 – from 9 June 2009. End-of-lifed in October 2009).

However, the most critical issue is that code is not included in the release. Looking at the directory rtl819x-SDK-v3.4.5.1/rtl819x/toolchain there are a number of GCC tools (GPLv3 licensed, so the license list is incomplete) as well as binutils delivered only as binaries. These also include realtek confidental documents (see screenshot below).

I’m stopping my dig here, but I will have to follow up my written request for the source code, unless the optical medium contains more. I thought they had learned

Five days left

I use to joke that the last week before foss-north is the worst – everything is done, all that is left is the stress.

This year, we have the broadest program yet. 25 speakers talking about everything from community policies, GPU isolation, blockchain, historical KDE software, retro computers, IoT, Android, SailfishOS, bug triaging, crowd funding, software updates, yocto, home automation, design to sub-atomic particles.

You can still get a ticket (and make sure to bring a friend) at foss-north . Welcome!