I've contributed to a number of Python projects, but their foundations were already established by engineers familiar with the Python ecosystem. Working in these codebases helped me gain familiarity with Python, while minimizing the need to delve into Python tooling.
Recently I've been experimenting with "AI" projects more and more, and a lot of the libraries are either only available in Python, or the Python implementations are more straightforward. Since these are side projects, I'm now on the hook for setting up Python tooling, like dependency management, linting, and testing.
Virtual environments are similar to node_modules
npm install and having the packages installed within the project's directory. I found Python's approach less straightforward. You have to "activate" a virtual environment before running any packaging-related commands. A common theme I started to feel is there are a lot of different ways to do things in Python, and it's not always clear which is the best way. In this case, there are several popular conventions related to where you put the virtual environment, its naming, and activation methods. Do you run
python -m pip install or
pip install? Do you run
python -m venv or
This ambiguity prompted me to install Poetry, which decides a lot of these things for you.
Use Poetry for dependency management
The smoothest transition for someone familiar with
npm is to use Poetry in their Python project.
In my research, I encountered a post by James Bennett: Boring Python: dependency management. In it James recommends sticking to Python’s default tools:
Python’s packaging ecosystem has default tools for each of these. In order, they are: (1) setuptools, (2) pip, (3) virtual environments. These three tools are your foundation, and for truly “boring” Python development work, I would argue that you should stick to them almost exclusively.
This initially resonated with me — I find Node's default package manager,
npm, to be sufficient and often feel it's an unnecessary step to install and maintain usage of a different package manager like
That was until I tried using Python's default tools. I gave it a shot, but eventually succumbed to decision fatigue. You have to decide:
How to manage virtual environments?
How to define and install dependencies? A single or multiple requirements files,
How to pin dependency versions?
Poetry takes these decisions off your shoulders, and is closer to how I'm used to working with NPM. It's an additional tool to install, unlike
npm, but I think it eases the transition to working with Python. The Poetry documentation is also great.
A few tips I have in my notes:
pyproject.toml is like
package.json but on steroids.
poetry config virtualenvs.in-project true and
poetry config virtualenvs.prefer-active-python true so the virtual environment is created within your project, and uses the same Python version your project is configured to use.
DRY'ing common commands
A Makefile can be used to encapsulate common commands, like
make install and
make test. Makefiles aren't exclusive to Python! A basic example of a
This is similar to using the
scripts section of
package.json, although more powerful.
Another post from James, Boring Python: code quality, was helpful here:
flake8 is generally faster and will raise fewer false positives, but checks/enforces fewer things. Pylint is generally slower and will have more false positives, but checks/enforces a lot more things.
Pylint [...] requires everything, including all of your dependencies, to be importable during the run (in order to check for things like correct usage of imported modules/functions/classes)
I ended up going with flake8 for my side projects, and I've been on projects that have utilized both.
The equivalent to Prettier is a combo of Black and isort.
Some tips from James' post:
For isort, I recommend setting the “profile” option to "black" to tell isort to match Black’s preferences. If it has trouble recognizing which modules are part of your codebase and which aren’t, consider setting
known_third_party to help it out
[put] the configuration in a top-level
pyproject.toml file in your repository. In the case of Black this is the only supported configuration file, and most other tools support using
pyproject.toml as a centralized configuration file now
Pytest seems to be the Python equivalent to Jest. If you're used to
jest.spyOn, you can install the
pytest-mock plugin. If you're a heavy user of Jest's watch mode, you can add this to Pytest with the
Documentation / Comments
Whenever string literals are present just after the definition of a function, module, class or method, they are associated with the object as their doc attribute. We can later use this attribute to retrieve this docstring.
A docstring is akin to JSDoc, except it's within the function definition.
There doesn't appear to be a standardized format for these strings, but reStructuredText (reST) seems popular and Google also has their own format.
This Programiz course provides a helpful overview of Python docstrings.
If you know a better way, send them my way, but so far I find myself often setting the following in a
.vscode/settings.json file in the project directory. In this example, it's a monorepo where the project is in an