Onboarding to Python, FastAPI, SQLAlchemy (2021)

Several key pieces in Pinwheel’s tech stack are:

• Python (CPython) - Language + Runtime
• FastAPI - Web framework
  • Pydantic - I/O data object validation
  • Starlette - ASGI framework
  • uvicorn - Event loop ASGI server
• SQLAlchemy - ORM
  • psycopg2 - Postgres DB driver

Here are a collection of articles, notes, books, etc. I thought were particularly helpful while I was learning this tech stack.

Python

Several new language features were added in Python 3.8, particularly f-strings (i.e., es6’s template strings) and assignment expressions (i.e., the walrus operator in if (a := 2) > 1:). Here’s a 2019 article with brief reminders of other new Python 3 features.

How do Python imports work? The article “Python behind the scenes #11: how the Python import system works (2021)” provides a detailed run through.

A reminder that Async Python Is Not Faster (2020) with brief architectural explanations accompanying benchmarks of various Python web frameworks. Here’s another 2020 article, we have to talk about this Python, Gunicorn, Gevent thing, discussing limitations of Python’s user-space async pattern. And another 2020 article, Latency in Asynchronous Python, which explains how sync tasks are queued up for execution at the end of the event loop which can lead to high latency.

For a deep dive of the CPython runtime, checkout Inside the Python Virtual Machine.

Checkout the switch-statement for dispatch_opcode in ceval.c to understand the core operations in CPython.

How are complex types in the types standard library constructed? It constructs the objects and then calls type() on them (HN comment).

A run through of the Python GIL, practical implications, theoretical issues, attempts to fix it, and future plans (2021)

Instagram has teased the idea of strict modules (2019), which are side-effect free on import.

This 2019 blog post provides summaries of deterministic and statistical profilers for Python.

FastAPI / Starlette / uvicorn

FastAPI creator’s brief HackerNews comment on what FastAPI evolved from.

How do the layers connect?

FastAPI is implemented as a Starlette application with most logic implemented as middleware. The APIRouter middleware implements dependency resolution, request validation, and response validation and serialization.

In Starlette, HTTPEndpoint dispatch calls run_in_threadpool which in turn calls asyncio’s run_in_executor on the default event loop.

If you use uvicorn as your ASGI server it defines the event loop execution model. When uvicorn’s workers/servers start they call setup_event_loop which in turn either configures asyncio using an unparametrized new_event_loop call or uvloop’s EventLoopPolicy.

Pydantic

The Pydantic source code is quite legible. See validate_model which implements the main logic.

SQLAlchemy

Mike Bayer, the original author of SQLAlchemy, authored a chapter on SQLAlchemy in The Architecture of Open Source Applications. He provides a breakdown of the core vs. ORM abstractions, SQL expression compilation, the Session object, etc.

psycopg2

The Python DB API, PEP 249, provides the base interface of psycopg2. Review this PEP to understand the philosophy behind its API.

Why does the wheel only psycopg2-binary package exist alongside the source only psycopg2 package?

Wheel packages are a Python standard to distribute self-contained binary package. […] Unfortunately, after the packages were released, it was reported of occasional segfaults of Python processes using Psycopg from the Wheel package.

See the 2018 blog post for more details.