Installing multiple versions of Python with Pyenv

I write a lot of stuff in Python – some of it useful, some of it… not so useful. But as a relative newcomer to Python, it’s almost always in Python 3. Most modern Linux distributions come with Python 3, so what’s the problem?

Although yes, a lot of distributions do come with a version of Python 3, it’s often an older version. It’s not uncommon to see 3.4 installed, which can be a problem if you’re relying on the newer ways of using async in 3.5+“Okay”, you say. “I can just build the newer version of Python 3 and install it alongside the system-provided one”. At that point, a large hand appears out of nowhere and slaps you across the side of the head. A loud voice, reminiscent of Death from the Discworld series, booms:-

MESSING WITH THE SYSTEM PYTHON WILL ONLY LEAD TO PAIN AND SUFFERING

Jack Nicholson in The Shining
Terrifyingly accurate representation of what yum will do if you change the version of Python on it

You look at your cup of coffee and wonder if someone slipped something into it when you weren’t looking, but there’s a serious point here – messing around with the system-installed versions of Python is a recipe for a world of hurt. There’s a fair few system utilities that are written in Python, and the distribution maintainers are doing their job by ensuring that the correct version of Python gets installed along with them. While it’s definitely possible to install a newer version of Python system-wide, it’s often not worth risking the potential for subtle fuckery if your newly-installed version of Python ends up being used for something that didn’t expect it.

Side note: at this point I should clarify that the imaginary situations I’m referring to here are when you need a particular version of Python for development. Making a new version of Python available for services deployed in production is definitely a problem you might run into, but is a topic for another post.

So what to do? Luckily, there’s a relatively straightforward way to make pretty much any version of Python available to you – and only you. Enter pyenv.

pyenv lets you easily switch between multiple versions of Python. It’s simple, unobtrusive, and follows the UNIX tradition of single-purpose tools that do one thing well.

This project was forked from rbenv and ruby-build, and modified for Python.

I won’t go over the installation instructions because I’d just be repeating the clear and concise instructions the pyenv project already has, but here’s what it’ll let you do:-

  • have multiple versions of Python installed at once – both Python 2 and Python 3
  • have multiple implementations of Python installed at once – for instance, Jython or Pypy alongside the standard CPython
  • set the version of Python in use on a per directory basis
  • set the global – for you – version of Python in use if you don’t otherwise specify a version

It does this by using shims – that is, inserting itself in your path and making sure that when you – or your application – calls the Python interpreter it will get the version that you intended it to get. It’s a lot safer than installing a version of Python system-wide, it doesn’t need root[0] and there’s no endless manual fucking around with PATH or LD_LIBRARY_PATH. It’s so useful that I even have the following in my dotfiles repo[1] I drag around between systems I work on:-

#!/bin/bash

function install-pyenv() {
  if [[ ! -d ~/.pyenv ]]; then
    echo "Downloading pyenv..."
    git clone https://github.com/pyenv/pyenv.git ~/.pyenv
    . ~/.bashrc
  else
    echo "~/.pyenv already exists!"
  fi
}

if [[ -d ~/.pyenv ]]; then
  echo "~/.pyenv found, initialising pyenv..."
  export PYENV_ROOT="$HOME/.pyenv"
  export PATH="$PYENV_ROOT/bin:$PATH"

  eval "$(pyenv init -)"
fi

This lets me just run install-pyenv on a system I haven’t already installed it on, and it will pull pyenv from GitHub and then do the things it needs to make it available in my session. Far nicer than being slapped around the face by a character from a fantasy novel series.

[0] while you don’t need to be root to install pyenv, since it will build the version of Python from source you may need to install additional packages (such as gcc) if they aren’t already installed.
[1] one of these days I will tidy it up enough to make it publicly-available, but it doesn’t do anything that these public projects already do, and probably do better.

Resetting user passwords on Mastodon

I recently installed Mastodon (yay!), didn’t enable email sending, then promptly forgot my password (duh!). Since Mastodon is using Rails, it’s pretty straightforward to reset it.

Log into the docker container

I’m running Mastodon in Docker with docker-compose (see here for more details). If you are too, you’ll need to log in to the web container to do this. If you’re not, you can skip this.

First, find the name of the web container. It’s probably mastodon_web_1, but check with:-

$ docker ps | grep mastodon_web | awk '{ print $NF }'

…and the output should look something like this:-

mastodon_web_1

Log in to the container with the name you just got:-

$ docker exec -ti mastodon_web_1 bash

Use the rails console to reset the user’s password

Start the rails console with:-

bash-4.3$ rails c
Default type scope order, limit and offset are ignored and will be nullified
Creating scope :cache_ids. Overwriting existing method Notification.cache_ids.
Chewy console strategy is `urgent`
Loading production environment (Rails 5.2.1)
irb(main):001:0>

Get the Account object, with the username of the account whose password you want to reset. The details will also be echoed to the screen:-

irb(main):001:0> account = Account.find_by(username: 'myuser')
=> #<Account id: 1, username: "myuser", do...

Next, get the associated User object:-

irb(main):002:0> user = User.find_by(account: account)
=> User(id: integer, email: string, created_at: datetime, updated_at: datet...

Again, the details of the User object will be echoed to the screen as well as placed in the user variable. Now we can change the password attribute on the User record:-

irb(main):003:0> user.password = 'dontforgetitthistime'
=> "dontforgetitthistime"

…and then save it:-

irb(main):004:0> user.save!
[ActiveJob] Enqueued ActionMailer::DeliveryJob (Job ID: 8975893d-ba20-3453-b5ed-2911e846276a) to Sidekiq(mailers) with arguments: "UserMailer", "password_change", "deliver_now", #<GlobalID:0x00005573acbdb148 @uri=#<URI::GID gid://mastodon/User/1>>
=> true

And you’re done! If you haven’t tried Mastodon, feel free to head over to https://mastodon.nsnw.ca/ and sign up for an account.

Dokku and KVM on Ubuntu quickstart

Install prerequisites

Install qemu and supporting utilities:- sudo apt install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virtinst

Download Ubuntu

At the time of writing, 18.04 is the latest. I usually store ISOs in a separate directory to the VM images, so feel free to adjust the below to suit. wget http://releases.ubuntu.com/18.04.1/ubuntu-18.04.1-live-server-amd64.iso

Create a guest

Create a guest VM. Again, the below settings are really a minimum requirement, so adjust to your tastes. virt-install --name=dokku --vcpus=1 --memory=1024 --cdrom=ubuntu-18.04.1-live-server-amd64.iso --disk size=10 This will open up virt-viewer so you can see the console. Alternatively, you can use the --graphics vnc option to enable the VNC server.

Install Ubuntu

Install Ubuntu. You can accept the defaults. You might need to enable the universe repository, which you can do by editing /etc/apt/sources.list (see here for more information).

Install Dokku

Download the Dokku installer:- wget https://raw.githubusercontent.com/dokku/dokku/v0.12.12/bootstrap.sh Have a look at bootstrap.sh before running it, making sure nobody’s snuck anything nefarious in there. If you’re feeling particularly paranoid, bootstrap.sh itself pulls the Docker install script, so you might want to grab that too and give it a once-over before continuing. Run the installer:- chmod +x bootstrap.sh sudo ./bootstrap.sh

Configure Dokku

Open http://<your vm IP>/ in a browser, and tell Dokku:-
  • your SSH public key
  • a hostname
  • whether you want to enable virtualhost naming
Once you hit Finish Setup, Dokku will shut down the web server and redirect you to http://dokku.viewdocs.io/dokku~v0.12.12/deployment/application-deployment/, giving you the first steps to deploying apps to Dokku.

Two new (unrelated) Docker Compose definitions

docker-elk

docker-elk, a fork from deviantony/docker-elk. This fork includes an rsyslog container running on port 10514 so you can pump syslog into Elasticsearch out of the box.

docker-teamspeak

docker-teamspeak, a fork of the now-defunct overshard/docker-teamspeak. I’ve updated this to support Teamspeak 3.3.0.