Is this clickbait? Understand, I love Python, and this isn’t to say Python is dead or you can’t be productive with Python. There’s a lot of lessons to be learned from Python, and I couldn’t have learned them without Python. But I hope to make the case that going forward, we should now know better. Let’s go in heavy…
Are trade-offs for quick iteration worth it?
Unsurprisingly, and conforming to Betteridge’s law of headlines, the answer is no. But why? There’s nothing wrong with a quick script or a prototype. The problem is where to go from there. No good deed goes unpunished, and successful prototypes get used. You might say this is a good trade-off! But it gets old really quickly having to do the same steps to “productise” a prototype. And the cost of doing it afterwards instead of up-front is disproportionate. Which brings me to the next point.
Static type checking is a boon
Run-time type checking sucks for big projects. Period. It can work though, it just needs very diligent programmers. So a common trope is hire better programmers. This is obviously flawed. One reasoning might be this isn’t cost-effective - slightly dehumanising, but ok. No, the reason I find persuasive is I can’t be bothered to be that diligent! It’s a waste of my time, attention, and brain power. Type checking is something that computers are great at, it can be automated very well.
There is a flip-side, I do think in a dynamic context, you have to think about control flow and data structures a bit more, so they may turn out more simple. With type checking, you can get away with more convoluted, horrible types that still seem workable but probably should’ve been refactored. I don’t think this is reason enough though, when the more common class of errors is typos or misunderstanding.
So you want static type checking, which is much easier to do from the start. Just add Mypy, a strict configuration, annotate everything like you would in a static type system, and you’re good to go. Right?
Tribal knowledge tooling
Setting up a Python project with best practices is now non-trivial. If you know how to do it, the results are quite nice. Currently, my minimum setup is using Poetry for dependency management, Black for auto-formatting, and the aforementioned Mypy for static type checking. Maybe some linting. And then you have all these dependencies and configuration files to sort out for e.g. running CI checks.
But compare this to Rust (or Golang, if you must. I have a strong dislike for it, which I’ll gloss over to not digress too hard). You get this all by default. Ignore the fact that Rust has the borrow-checker, and isn’t garbage collected. Java has a similar problem: Gradle vs Maven, although this debate is kinda settled. No decent/standardised auto-formatting. The point is the tooling!
Regardless of the results, in some languages you need to know how to do it, and you need to do it every time. It’s tedious. It’s error-prone.
Is it an unfair comparison? Sure, Python is a much older language. Of course newer languages should’ve learned from these issues. But where does that leave Python?
Is there a way forward?
Maybe. Before Python 3, virtualenv was a common requirement. But it got integrated into the standard library as venv. The same could be done for Poetry, Black, and Mypy in e.g. Python 4. If only it were that easy.
For Poetry, there’s also Pipenv, which is actually under the Python Packaging Authority (PyPA). I now strongly prefer Poetry. But also, pip is being improved, for example with an improved dependency resolver. I guess there’s also Flit and Enscons (?).
For Mypy, there’s Google’s Pytype, Facebook’s Pyre, and Microsoft’s Pyright. Oh dear.
For Black, technically there’s others that nobody uses, like autopep8 and yapf. They are effectively dead. But, Black doesn’t sort imports. So you might also need isort - except it struggles when the tool is installed outside a virtual environment, but dependencies are inside the virtual environment.
There’s Pylint, Flake8 (somewhat redundant with Black, it wraps PyFlakes, pycodestyle, and McCabe), and more.
I haven’t even mentioned pytest (highly recommended) over the standard library’s unittest, and I’m sure I’ve forgotten others.
What a mess.
Is there any hope?
Maybe. Black is now the de-facto auto-formatter. This is because it is opinionated. I still have to occasionally lay out why consistency is more important than the exact rules a codebase previously had but were unenforced, but most devs just get it.
I’d also call the pyproject.toml format a success (see PEP 518 and related PEP 517). Many tools already support it or are moving to support it. But you only really get the full benefit when using a tool like Poetry, and all other tooling uses it (rather than setup.cfg or custom dot files).
But on a whole, I’m not optimistic. Black is a miracle, and only because the previous tools were a lot worse. There was a void to fill. Python 4 probably can’t be too opinionated (for other reasons, too, you don’t want a reputation of breaking every major version).
Comparing apples to oranges
Do I wish there was a garbage-collected Rust? Yeah. Do I wish there was a Rust REPL, or other prototyping tools like IPython or Jupyter Notebook? You bet. Do I miss a robust, feature-complete standard library where I don’t have to pull in a crate for ZIP files that produces opaque errors that e.g. don’t include the filename that wasn’t found? Oh, and backtraces for errors?
Do I wish I could statically compile Python into a stand-alone executable? Absolutely. And have a way to type-check incoming JSON like serde does/having something like serde at all in Python? 100% percent. Enums/algebraic data types (ADTs)?
Closing
You can’t always get what you want / But if you try sometimes / Well, you might find / You get what you need
Some rando Brits
In 2020/2021, what I need looks more like Rust and less like Python to me. It just isn’t fun to program any more in Python. But it is in Rust. And a big part of that is tooling.