Mudmap Version 2

Over the last couple of month's Mudmap version 2 has been in the works. This work has been happening behind the scenes but is now ready for public announcement.

The new back end is much more performant than the current in service setup. For an application like Mudmap which makes a lot of costly connections (SSH has setup and tear down costs) it has become apparent that the software in use needs an overhaul.

What does this update look like, technically?

The back end is now written in Go which is replacing Django. Django is an excellent web framework written in python. It's the batteries-included best of breed standard for python and makes for an excellent choice for most web applications. After a lot of backward and forward, I have decided to move off Django, and for several reasons.

Django is opinionated

Meaning it wants and expects you the developer to follow certain practices. These standards are great when you're platform conforms to this standard. Unfortunately, Mudmap has some unique constraints which whilst workable have reduced the efficiency of Django. This detracted from its usually great developer experience.

Mudmap needs a great API handler, and Django has a de facto third party module which fits the bill - django-rest-framework (shortened to DRF). DRF, like Django, is opinionated and expects certain contracts to be fulfilled. Mudmap, again, has been able to make it work but not without some extra leg work. The rest framework is perfect for your typical Django application but its usually useful abstractions in this case have made development more difficult than necessary.

It's important to point out that all these complaints are not reason alone to embark on a full rewrite. Nonetheless, they are contributing factors.

Python is slow*

Python is an excellent language - easy to learn and powerful enough to run almost any application. But, it's concurrency (the ability to run multiple operations at the "same" time) is still a bit of a mess. The language was first developed in 1991 (or there abouts) when concurrency wasn't as important as it is today. The fact that python's core is built around a thing called the Global Interpreter Lock is a big part of its difficult path to concurrency. Without going more into it, this is important because of Mudmap's expensive network operations.

Switching to a concurrency first language such as Go ameliorates a lot of the issues associated with making n number of SSH connections per client per request. Further, Go through its use of goroutines makes executing background tasks much easier. To achieve the same thing in python usually requires either an event queue, or task runner such as Celery - meaning more complexity.

As a simple comparison, to send an email when a user connects a device in python requires three servers; Django, Celery and Redis. In Go, its just the Go server.

For users, the execution speed of Go, is noticeable. Go opens network sockets faster and with less resource exhaustion than python.

I don't like benchmarks but here is some.

Python uses more memory

The memory consumption for Django is much higher than a comparable Go web application.

I look forward to running a direct comparison between the two versions when they're in production.

Go is compiled

Go builds a compiled binary, and can do so for almost any architecture. Deploying a binary, or building a container using a Go binary is a hell of a lot easier than managing python's package management system. Container sizes are also much much smaller.