Use pyenv for local Python development

Brett Weir • Jan 27, 2023 • 6 min read

I'm a huge fan of containers. They solve a great deal of problems with portability and deployment. One thing they are not great for is developer ergonomics. The hard separation between the inside and outside of the container puts most developer tooling and IDE integrations on hard mode, and good luck ever using GUI tools again. There are some glimmers of hope with things like VSCode's Dev Containers extension, but that's not going to do you a lot of good if you're not developing in VSCode.

Despite what anyone tells you, not everything needs to run in a container. Desktop environments are what they are, we're adults, and we're allowed to use a local development environment if we understand the consequences of doing that (and are prepared to format our workstations on a regular basis!)

There are lots of tools that live somewhere in between the everything-goes ethic of the desktop work environment and the airtight seal of a container. For Python developers, we have pyenv.

pyenv is a tool that allows you to bootstrap a Python installation from scratch on your system and use it as if it came pre-installed as part of your OS. It makes it very easy to try out a variety of Python configurations, like a container does, but plays so nicely with your local dev tools that you'll forget you had to install it at all. You can have as many Python versions as you like, and quickly switch between them without changing any code.

Install pyenv

Download pyenv's automatic installer:

curl https://pyenv.run | bash

Add this magic snippet to your ~/.bashrc:

echo '
export PYENV_ROOT="$HOME/.pyenv"
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
' >> ~/.bashrc

This will cause pyenv to add itself to your $PATH variable whenever you open a new shell.

Close and reopen your shell. Afterwards, pyenv will be available as a command. You can test it with the --version flag:

pyenv --version

And the python3 command will now be provided by pyenv:

$ which python3
/home/brett/.pyenv/shims/python3

Pick a Python version

With pyenv installed, we're ready to install a Python distribution. But which one do we need?

What makes an appropriate Python version depends on your use case, but oftentimes that choice will be dictated by availability of supported third party packages that you'd like to use. It takes awhile for package maintainers to update their packages to support new Python versions, and new versions are coming out all the time.

Looking at the latest versions

Let's take a look at what's recently available with pyenv install -l:

$ pyenv install -l
Available versions:
  2.1.3
  2.2.3
  2.3.7
  2.4.0
  2.4.1
...
...

According to pyenv install -l | wc -l, there are 673 Python versions available.

Picking a good one

If you're not sure, a good rule of thumb is to use the latest patch version that is one or two minor versions behind the latest minor version. If that doesn't make sense, how about an example?

  • Latest version: 3.11.x

  • One or two minor versions behind: 3.10 or 3.9

  • The one you want: 3.10.x (look for the latest x)

This gives you access to newer Python features, but with a good base of supported third party packages. And of course, if you're starting a new package, let's make it Python 3!

After thumbing through the list for awhile, I've chosen Python 3.10.9.

Install Python

Install build dependencies. We'll follow pyenv's suggested build environment page. Yes, you need all of them (I've tried):

sudo apt update
sudo apt install build-essential libssl-dev zlib1g-dev libbz2-dev \
  libreadline-dev libsqlite3-dev curl libncursesw5-dev xz-utils \
  tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev

Start the installation. It'll take awhile, but the output will be short:

$ pyenv install 3.10.9
Downloading Python-3.10.9.tar.xz...
-> https://www.python.org/ftp/python/3.10.9/Python-3.10.9.tar.xz
Installing Python-3.10.9...
Installed Python-3.10.9 to /home/vagrant/.pyenv/versions/3.10.9

Select the new Python installation

You can see all Python versions available to pyenv using the versions subcommand:

pyenv versions

The output is shown here:

$ pyenv versions
* system (set by /home/brett/.pyenv/version)
  3.10.9

After the installation, you'll be able to use your new Python installation at both the project level and user level, using the local and global subcommands:

# configure for a specific directory
pyenv local [VERSION]

# use for every call to python by default for your user
pyenv global [VERSION]

In general, it's a good idea to leave the system Python alone as much as possible. The system Python belongs to your system and is critical to system stability. Since I don't want to reinstall my OS before I need to, I'm going to configure my user account to use the pyenv Python installation by default (as mentioned above, I'm using version 3.10.9):

pyenv global 3.10.9

If you ever need to use the system Python for whatever reason, set your version to system:

pyenv global system

Use Python

Congratulations! You are the proud owner of a new Python installation.

You can use it just like your system Python installation, so it'll play nicely with your IDE and other local development tools you may be using, while still allowing you to isolate your Python environment from your system Python.

Removing pyenv

There are situations where you may need to remove pyenv, because you messed up your installation somehow or just want to start fresh. pyenv never gets installed as root, so removing it is safe and can be done at any time. To do so:

  • Remove the ~/.pyenv directory.

  • Remove the pyenv snippet added above from your .bashrc.

All traces of pyenv will be gone, including commands installed via pip while using this Python installation.


Tags

#linux #python