Xavier Olive research teaching python blog til cli

Life changers in Python

11 January 2020

When programming is part of your daily routine comes the choice of a preferred programming language. I have been practicing several common languages and paradigms in my professional life, and finally converged to Python. I am used to facing disapproval from people from the real-time programming or model checking communities: semantics is flawed, typing is inexistant and performance is poor.

I would argue that Python is well equipped to bypass these prejudices, and unites the strength from many worlds. Python is slow and error-prone if you don’t know have a proper programming background. Python is not the best languages to learn about programming, but a pleasure to work with once you went through other languages.

Third-party libraries developed by the community bring performance and safety to the language. They provide ease of use and a new way to think about programming. I would call them life changers.

  • numpy brings the performance from C arrays to Python. Avoid indexing at all cost and then Python is not slow anymore. If you can’t avoid indexing, consider Cython: add typing annotations and compile your code to native C code. Yet for most cases, numba infers the type itself and compiles Python methods just in time;

  • pandas ports data frames from R to Python. The learning curve is quite steep at first, but once you get to master method chaining, you get new ideas about comfort in programming;

  • altair is a recent declarative framework for data visualisation. They bring a grammar of graphics to Python, and translate descriptions for the Javascript Vega framework. I consider it one of the most promising alternative to matplotlib, the library I enjoy criticising the most for their arguably poor API choices;

  • pytorch also provides a clean object-oriented interface for Machine Learning and artificial neural network. requests wraps up the poor behaviour of the built-in urllib library for networking interactions and provides an intuitive API instead;

  • last but not least, mypy parses type annotations attached to variables and functions and checks consistency in your code. The true power of typing in Python is comes from this “add type annotations when you are ready” paradigm. You may write code without any type annotations. You may annotate only part of the code and run code which is not properly typed. This is probably an heresy for skilled OCaml programmers but I find it efficient and pragmatic instead: your first shot at writing code probably works in most cases when you are ready to share it. Adding type annotations helps identifying tricky corner cases but does not hamper the execution of the code.

Of course, all these libraries are precious for the functionalities they provide. Yet there is more than just that functionalities. Were it just about API and services, these libraries will soon end up being irrelevant, replaced by better tools yet to be written. However, they become truly powerful and elegant when we look more in depth into the choices that were made to design them.

JIT compilation, method chaining, grammar of graphics, functional programming, and soft typing are (among others) powerful concepts to study, remember, practice and leverage, when writing your own programs. Blindly translating Matlab scripts or code originally designed for another language only brings frustration without brushing up any programming skill.