Some Vue + Django tips

As I wrote last time, I currently develop a web app based on Django and Vue using the django-compressor-parceljs. I’d like to mention some small things that I’ve learned since last.

Vue in development mode

I found it interesting that parceljs described how to get Vue into development mode for several bundlers, but failed to mention parcel. After some digging around, I figured out that parcel respects the NODE_ENV environment variable, so starting the django server with NODE_ENV=development ./manage.py runserver does the trick. Now I get proper error messages from Vue.

Constantly rebundling

Another annoyance with the default configuration of the compressor, was that it compiles (bundles / compresses) the Vue files once every time the server is started. This really lengthened the change-build-test cycle time, so something needed to be done. It turns out that the solution is to set the COMPRESS_REBUILD_TIME to a really low value in the django settings, and the contents is compressed every time. I set it to one second on my development machine.

Disabling compression would break the Vue files, as the browser cannot run the Vue files natively, i.e. they need a compression to be possible to run.

The consequence of this configuration is that loading a view based on Vue takes a bit longer, but it takes way shorter time than restarting the django server, recreating the backend state and so on.

Exposing Django context to Javascript

Django exposes a set of context variables when loading a view. This avoids an unnecessary round-trip over the server to asynchronously request the info that I already know the view will need. I’ve developed a small convention for this.

In the Django template, I add something along these lines:

<script>
var context_title = '{{ title }}';
var context_actors = [
{% for actor in actors %}'{{ actor }}',
{% endfor %} ];
...
</script>

This copies the django context into Javascript space with the variable name prefix of context_. I then consume these in the Vue app’s mounted method, e.g.:

    ...
    mounted() {
        this.title = context_title;
        this.actors = context_actors;
        ...
    },
    ...

This copies the state into the state of the app, which means that it is now a part of what the Vue interface is reactive to.

Looking at Vue

All applications are more or less connected today. The time of files on a disk, or moving them with a USB stick (or floppy) are over. Even file based programs are often synced using Nextcloud, dropbox, google drive, etc.

At Eperoto I’m busy building a backend for a React frontend, but there I stay in my comfort zone at the backend. It is Python, databases and files, just as I know and like things to be. I also have my normal toolbox for debugging and know how to execute a rich set of unit and integration tests to ensure that things stay sane over time.

However, I have another side project. Finally I’ve reached a point where have to do take a dip in the sea of web frontend. I don’t mean messing about with the odd Javascript snippet or fighting the windmil^Wcss.

The choices I was looking at where React, Angular and Vue. I did not do a deep analysis when picking a framework to try. Instead my reasoning was a long these lines:

  • React, I can pick this up from Manos at Eperoto.
  • Angular, requires npm and a setting up an environment to get started.
  • Vue, just needs a script to be included into your web page.

So, me being lazy, I choose to view Vue, which is pronounced view. It seems that I like a bit of pronunciation ambiguity in my frameworks.

Starting Slow

The reason for choosing Vue was the ability to run right in the browser. The strength of today’s browsers is the easy of deployment, but a hidden strength takes me back to the 1980s and 90’s. Everyone has a development environment sitting right in front of them. However, instead of being used to load programs like with the C64, you now I have to look for the development tools to bring up the console.

https://ih0.redbubble.net/image.537660612.0177/pp,550x550.u10.jpg

I’m building a tool that is currently in closed alpha stage, so I cannot tell you too much about it. What I can show is one of my first Vue components. It is a text element, that when clicked turns into a text edit. I made two of these, one for editing a line, and one for editing a paragraph. It also sports a placeholder text if it is empty.

Vue.component('editable-text', {
     props: ['value', 'placeholder'],
     data: function() { return { editing: false, } },
     template: '{{value}}' + 
               '{{placeholder}}' +
               '',
     methods: {
         on_start_editing: function(e) {
             this.editing = true;
         },
         on_end_editing: function() {
             this.editing = false;
         },
     },
     directives: {
         focus: {
             inserted(el) {
                 el.focus();
             }
         }
     }
 });

The code above makes it possible to use an editable-text tag. In order to be able to use the new tag, a Vue app needs to be created and mounted into the DOM of the web page. That means making it take over a div somewhere in the page, and let Vue manage its contents.

This is partially done in the Javascript file:

var app = Vue({
     el: '#app',
     data: [ text: '' ]
});

And the other half goes into the html:

<div id="app">
<editable-text v-model="text" placeholder="enter text"></editable-text>
</div>

<script src="js/vue-test.js"></script>

Finally I sprinkled some css on top of this to make it look sane. For instance, defining a placeholder class.

.placeholder {
     color: #999999;
}

Now we have something that works! I can include a development version of Vue into my page, which provides helpful feedback and point me in the right direction when I make mistakes. And then I can point this to a minified production build of Vue once I’m done.

Bring on the Django

For my backend needs I choose to go with Django. It might because I know Python, it might because of the extensive docs, the great intro tutorial (no damn video – text that I can read back and forth in my own order and pace), or maybe it is just because I’m old.

I like Django. Django plays nice. It is my choice of backend framework. End of discussion.

So, how do we make Vue and Django play nice?

First up, the index.html needs to be served from Django, and the CSRF cookie needs to be served along with it:

from django.views.decorators.csrf import ensure_csrf_cookie
@ensure_csrf_cookie
def my_view(request):
     context = { … }
     return render(request, 'app/index.html', context=context)

Then the index.html needs to be adapted, e.g. use {% static 'app/index.html' %} instead of direct URLs and such. Then I place the js files in the app/static directory tree and run migrate.py collectstatic to get it all in the correct location for nginx to serve it on the production server.

There is, of course, more to the app. I provide a REST-ish API passing JSON around via a set of views served under /api/…. I could probably have used the Django REST Framework here, but I tend to have to adapt things as the API isn’t really proper REST. I just picked out the convenient bits from it, so I roll my own for now. To make these requests, I use Axios, which plays nicely with the Django CSRF cookie.

What is the Point?

When using Vue like this, the question is really what the point is. For smaller stuff I’ve rolled this in plain Javascript. Including the AJAX (AJAJ?) calls and updating the DOM. What Vue helps me with is that I can share a single JSON state between the server and client, and Vue reactively renders it as needed on the client side.

Still, writing Vue as shown above kind of sucks. The template part of a component is html wrapped into a Javascript string. This means loads of fun when handing various string enclosing chars – ” and ‘, I’m looking at you two! Also, just stuffing everything in to a Javascript map makes everything a bit clumsy. It is really had to misspell something and it just stops working.

So, it was time to take it to the next level. Vue can handle single file components. This sounds a lot like classes to an old C++ guy like me, so bring it on!

The Computer Says No

Vue might know what a *.vue file is, but Firefox sure does not. It does not know what an include is either. It has lots of interesting errors about it, so it knows about a similar concept, but not what I needed it to know. Instead, I need a compiler and linker, or what the webistas call a bundler.

When mucking about with npm to get React or Vue things to work, I always ended up with webpack. But apparently web pack is old, boring, and slow (a bit like me). Having asked a friend, I learned that all the new kids use Parcel – a project that has an icon next to every heading on the whole site. Amazing…

What Parcel (and webpack or any other bundler) does is that it transform assets into Javascript, HTML and CSS, that the browser understands. It also combines the assets into fewer files, e.g. one js-file instead of a pile of them, thus reducing the number of files the browser has to request, thus making things a bit quicker.

If this sounds a lot like what a compiler (or transpiler) and linker would do – it is exactly what it is.

So, with a bundler like Parcel, I can build my *.vue files into Javascript, which I then can run in my Browser.

What is the Point – part 2

So, what does a Vue file look like?

<template>
    <span v-if="(!editing) && (!!value)" @click="on_start_editing()" >{{value}}</span>

    <span v-else-if="(!editing) && (!(!!value))" @click="on_start_editing()" class="placeholder">{{placeholder}}</span>

    <span v-else-if="editing">
        <input v-focus="" :value="value" @input="$emit('input', $event.target.value)" @keydown.enter="on_end_editing()" @blur="on_end_editing()" type="text" class="form-control">
    </span>',
</template>

<script>
export default {
    name: "editable-text",
    components: {},
    props: ['value', 'placeholder'],
    data: function() { return { editing: false, } },
    methods: {
        on_start_editing: function(e) {
            this.editing = true;
        },
        on_end_editing: function() {
            this.editing = false;
        },
    },
    directives: {
        focus: {
            inserted(el) {
                el.focus();
            }
        }
    }
}
</script>

Apparently you can stuff some css in there as well. But still, the whole script part is just the same mess as before. Why, when making a transpiler, didn’t anyone come up with a sane syntax for properties, data, methods and directives?

Django and the Modern Frontend

Ignoring my annoyances with the the single file Vue components, it is still an improvement, so let’s continue.

I would like to build my frontend and backend in one go. I’d like a single git hash and know that the frontend and backend there fits together. At the same time, I don’t want to deploy my entire development toolchain to the production server. What I want is:

  • The Django stuff running an an uwsgi application.
  • All static files served directly from nginx.
  • All the frontend stuff to be a set of static files.

Enter: django-compressor-parceljs. Using this Django extension I can have parceljs do it’s work on the fly, in the Django app – and it allows me to build for production as well. More on this later.

However, going for parceljs took me one step beyond what I could do with simply including scripts into my web pages, I needed to install npm. I have a lot to say about this tool. To some extent it feels like it is trying to solve a problem with tooling more complex than the system, but let’s get to some specifics:

  • parceljs (and lots of command line tools) wants to get installed globally. This will collide head on with the distro, so it is a no-go for me.
  • When installing locally (in your dev tree) with npm, the executables end up in node_modules/.bin. A hidden directory. Thanks. Still, fixable by updating PATH.
  • The number of dependencies pulled in for a fairly small project is quite astonishing. The number of dependencies is huge. My package-lock.json clocks in at 6243 lines… That is for Vue, Axios and Parcel.
  • Many dependencies likes to not install their dependencies, but instead to tell me to do so in the form of error messages. I’m not sure I get the point of this – just install it, or have a recommends option or something.

Having setup django-compressor-parceljs and updated my PATH to include node_modules/.bin, it actually works. However, since the vue-files are compressed, the development experience is not perfect. I’d much rather be able to debug the vue-files directly via the Django server. If anyone knows how to, please tell :-)

I also created a small script that sets the COMPRESS_OFFLINE to true and then compil^Wcompresses the frontend on my development machine, meaning that I don’t have to install npm and all the dependencies of vue, parceljs, and axios on the server.

Conclusions

I’m still learning. I bet there is a lot that can be improved in this workflow. I also need to work on my CI/CD to do the offline compression automatically, and such. Still, I have some reflections to make from the viewpoint of a C++, Qt, QML, Python perspective:

  • All my web frontend friends appreciate the separation of behaviour from style from state. This can be done in QML, but requires discipline.
  • To me, the vue files are only half-way there. The script part of the file would benefit from a domain specific language such as QML, instead of putting everyting into a dictionary based on conventions.
  • To a large extent it feels like the web frontend world reinvents terminology – I really hope that there are some people carrying over experiences (in both directions). For instance, a bundler is just a transpiler + linker. Why not reusing common Makefile systems instead of re-inventing the wheel?
  • Why can’t I do nice inheritance and templating within Vue components? I.e. why do I have to duplicate all code for a line edit version of a clickable text when I have the text area version already? What am I missing?

Is anyone else doing this? How does your setup look and what does your working process look like?

Also, Happy Holidays and God Jul!

foss-north pod – a look at the stats

The foss-north pod about Licenses and Copyright has been around since May 1st, so I decided to talk a look at the stats. We gather very little statistics, but what I know is that we have 635 followers on YouTube and 108 over at conf.tube (a peertube instance). We also serve the pod directly from foss-north.se/pod, where we keep 14 days of access logs. What can we read out from them?

First of all – we decided to provide the pod as ogg or mp3, and it seems like a majority of you prefer the ogg version.

The downloads per day is a mess. From the episodes page I can see that we released the last two episodes on Nov 20, and Dec 4. I was a bit surprised not to see a spike on the 4th or 5ph, nor any apparent weekend vs workday pattern.

So, what was downloaded? Keep in mind that this is a two weeks window, and episode 26 was available for the last 3 days only. It seems like we have an even spread of listeners across many episodes, with a focus on episode 25, which was the latest during the time window.

Does this mean that we have a steady flow of new listeners? Not sure – the YouTube subscriber count raises steadily, so it might be the case.

Finally, let’s have a look at the user agent strings. I’ve tried to classify this into client OS for browsers, Apps for obvious pod listening apps, Bot for bots and other for the unidentified ones.

To my surprise, quite a few of you are listening from Windows machines. Then we have the Linux devices followed by Android, and Apps. Unless you count the bots, of course.

Another surprise is that OpenBSD is more common than OSX among our listeners.

It is possible to dig out more from the logs, but the evening is approaching. There are some surprises here, but it is good to see that we have had 800+ downloads over the past two weeks. To be honest, I was a bit worried when we shifted from YouTube to a podcasting format in August. Our views dropped quite dramatically on YouTube, but it seems that you found your way to the pod instead.

At the end of the day, the positive feedback given over social media and email is worth more than stats, so we will keep on going. Also, clocking in at almost 200 views on our episode on the definition of copyright and 250+ on the history of free and open source is quite amazing in my book, as this is a quite a narrow meta-topic inside the free and open source movement.

Advent of Code and Learning

So, I decided to do Advent of Code this year too. I usually get stuck part of the way, but I still think that it is a fun exercise.

This year the plan is to use python and pytest the whole way through. Every day that i learn something that I want to remember, I add a til.txt file in that sub-directory. You can follow my progress and learnings in the git repository.

The lessons this far includes:

  • When using readline to read lines, the line-break is included, so len(text) will be one character more than expected. Strip your strings!
  • When doing number of ifelifelifelif, make sure to include an else, even though you know that all cases are covered. I run assert False in the else clause.

As you can see, these are on the level of small snippets of wisdom right now. I’m sure it will be more interesting as the problems become more complex.

QmlBook: Felgo Service Integration

Felgo has kindly sponsored the QmlBook, which has resulted in a new chapter. The topic this time around is the Felgo Qt extensions for
integrating various services that are commonly used by app developers, the Felgo cloud builds, as well as their live reloading technology.

When building modern apps there are many things that you might want to integrate – in-app purchases, ads, analytics, user accounts, user settings, real-time sharing of data between devices. Felgo provides integrations of common solutions for this which let’s you focus on developing your app. In the Felgo Plugins chapter, we look at some of them.

Another hassle when developing apps is that you need a Mac to build for Apple devices – unless you use Felgo cloud builds. Felgo cloud builds is a CI/CD solution for building and deploying Qt apps directly to an app store.

In addition to this, the chapter contains a deep dive into the Felgo live reloading solution. We had a quick look at using this in the first Felgo chapter. In this chapter, we look at how you can integrate it into your own executables, as well as how you can use it to develop on multiple devices simultaneously.

Intense weeks

End of October turns out to be one of the highs when it comes to workload this year. Everything happens at once – there are two public events that I’d like to tell you about.

The first one is running lights. This is an annual running competition organized by AIF Friidrott, the sports club my kids are active in. This year, this means organized by me and postponed due to COVID-19, but the virtual races started this weekend and the arena race will take place on the 24th.

If anyone of you are in the Alingsås area and enjoy I highly recommend you to join. The weather looks nice, and we will light up the arena with live fire, so it will be a great evening.

The second one is the foss-north 2020 take II event. This spring, we decided to try to organize a physical foss-north event this fall, as obviously the pandemic must be over by November. This seems to not be the case. :-)

Instead we are running a single day event on November 1 with six handpicked speakers. The event is virtual and free for all.

I would like to tell you about the speakers one by one, because I’m very excited about each and everyone of them.

Andrew 'bunnie' Huang

In the morning we welcome Bunnie Huang who will talk about the precursor project. Precursor is an open hardware platform for secure, mobile communications and computations. The focus is on security aiming to create a trustable platform.

Simon Ser

Next up is Simon Ser. He will talk about how to get pixels onto the screen in a modern Linux stack. This means a deep dive in the Kernel Mode Setting (KMS) interface. How it exposes hardware blocks and how to use it to get images shown on the screen.

Ramón Soto Mathiesen

The morning session then ends with Ramón Soto Mathiesen taking us into the land of Domain Driven Design (DDD) using Algebraic Data Types (ADT). Ramón has a background in functional programming languages and brings this knowledge into the world of multi-paradigm languages such as C#, Rust, and Swift.

Carol Chen

The afternoon session starts with Carol Chen from Red Hat Ansible. She works as a community manager for Ansible. She will be talking about how they move have moved from collections to contributions to conferences.

Lars Brinkhoff

We then continue with Lars Brinkhoff who will talk about the Incompatible Timesharing System (ITS). Lars works with restoring ITS and recreating the history from these early days of computing. ITS is of particular interest at foss-north at is the platform where tools such as Lisp, Logo, Scheme, Emacs and Zork where developed. This is where the foundations for the free software movement where born – quite literally.

Tor-logo by Stanchenko on DeviantArt

The day then ends with Alexander and Georg who will talk about Tor, the anonymity network. They will discuss why diversity is essential for reaching security and anonymity.

So, the next days will be crazy hectic, but it is all for something good. First a cosy evening of running on an arena lit by live fire, and then a day of talks about various FOSS projects.

I hope to see you there!

Felgo in the QML Book

Over the past year I’ve been bumping into the Felgo crew at various Qt events. They take Qt to the next level. It all started as a game development framework for Qt, but has turned into a powerful app development framework taking a lot of the rough corners of Qt, and extending the tooling with a powerful live reloader at the same time.

https://qmlbook.github.io/_images/felgo-logo.png

To showcase how the Felgo SDK fits into the bigger picture, and how much it actually helps creating apps, we agreed to extend the QML Book with a Felgo chapter. The chapter is built around a messaging app and showcases the whole UI creation from initial skeleton to working app.

https://qmlbook.github.io/_images/messaging-stage-2.png

We also cover a bunch of other things such as how to get started, the QML Live reloader, as well as some more advanced topics such as native dialogs, simplified networking and the JsonListModel.

Big thanks goes out to Felgo for supporting this work and helping to make QML Book better.

Akademy 2020

I had the pleasure of speaking at Akademy 2020 this weekend. This year Akademy is virtual, but I still got the feeling of a very interactive event. Interesting questions, greenroom for the speakers, and generally a nice experience. Big thank you to the organizers!

The video below should start roughly when I go on stage.

For the interested listener, you can find the slides here: https://e8johan.se/presentations/2020-09%20akademy%20v1.1.pdf.

The API wars – 16 years later

It is more than 16 years since Joel Spolsky wrote How Microsoft Lost the API War. The bonds of the win32 API lock-in is broken and the free web is here to take over.

The web has come a long way in the past 16 years. Richer APIs, dramatic performance improvements, and an ubiquity that surpasses anything else that we as a human race have experienced. Easy of deployment is king and the easiest deployment of all is to simply browse to a web page.

Creating web apps has always been riddled by browser compatibility caveats. Various services have been around to test rendering across browsers and versions, and frameworks to address common scenarios have evolved to create a write-once, deploy-everywhere story.

The modern web browser has become our universal runtime environment. It is what Java and .net aspired to on a crazy scale. However, it is not only a runtime environment. It is the perfect client server setup to provide everything as a service.

With the focus shifting from the browser to the actual contents, the value of controlling your own browser engine has become less and less attractive, and last week, Mozilla begun what I think is the final downward spiral of the last alternative to the Google led Chrome family of browsers.

(There are so many things I’d like to say about this. For instance, you should know about the Mozilla manifesto, as well as their funding being secured for the next three years. But I digress).

A browser engine is a hugely complex beast these days. An incredible number of backwards compatibility hacks, while ensuring high performance on both rendering and Javascript execution. Add a broad range of APIs for integration into the native host platform. Combine that with privacy and security concerns, and the code base is starting to turn into a beast.

Now, it seems that Google controls the leading browser engine and thus, holds the direction of the web as we know it in their hands. Google has not only won the search, contents, and personal data collection wars. They also won the API war.

Having a single, almost monopolistic entity controlling all these aspects of life makes me feel very uncomfortable.

I’ve started my own personal de-googling journey, and I know that there are many others doing the same. Taking back ownership over their email, shifting from Google Drive and Google Apps to alternatives such as Nextcloud. But also moving from platforms such as Twitter to federated alternatives such as Mastodon.

A lot of this is probably seen as nostalgia from an earlier generation growing old. The web has moved on and many parts of what I love about the internet are no longer in broad use. For instance, small forums are migrated to Facebook groups, IRC is taken over by freemium alternatives such as Slack, RSS feeds become less and less common, and so on. The web is being centralized and has been so over the past decades.

However, I believe that the tide can be turned.

On the contents side, early adopters are moving to federated and self-hosted services where data lock-in is impossible. Privacy concerns become more common outside of the technology sector. What is needed is great alternatives that are easy to deploy. Examples that I use are Nextcloud, ttrss, fripost, and Signal.

But what can be done about the API war?

An attractive possibility, in my view, is the raise of WebAssembly. It enables the deployment of complex applications into the browser, really turning the browser into the universal run-time environment. It does so for compiled languages and at great performance.

What about deploying a bare bones wasm run-time environment, and then deploy the browser into it. That way, the complex beast that is the browser of today, turns into a much more manageable animal that is the wasm run-time.

What would this change? Short-term, very little. Even if the Chrome engine is compiled to wasm and executed inside an outer shell, the experience and value is still delivered through a very complex code base controlled by one of the most dominant companies in human history.

Long-term, it would mean that the ease of deployment would apply to not only the web, but to the wasm run-time. We would shift from the HTML/CSS/JS world to a wasm world.

Not only would this mean that the universal run-time becomes smaller and more manageable to maintain by multiple parties, it also opens the opportunity to shift to a optimized way to run software (the hardware requirements of the modern browser isn’t really environmentally friendly – it drives energy usage, as well as hardware obsolesce).

Now, all that is needed is time. An idea without execution, is merely a dream. I might be a dreamer, but I think that this is the way forward.