pycompile takes in unmodified Python files and compiles them directly into a system binary instead of needing to execute through an interpreter. This results in faster execution time and allows for compile-time optimizations. Importantly, this does not require a Python DSL. pycompile compiles a subset of vanilla Python into a binary.
To avoid needing to re-write and maintain a Python interpreter from scratch, the compiler front-end relies directly on CPython to translate Python source code into Python bytecode. On the back-end, this bytecode is then lowered using a custom PrIR MLIR dialect to LLVM IR. From there, it is compiled and linked to a custom builtin standard library to form a full executable. This builtin library implements the Python builtins in C++ that are themselves implemented in C (in CPython's implementation). For easier portability, the builtin library is statically linked to each output program. This increases binary size slightly, but removes the need of managing a dynamic library.
To download pycompile, simply navigate to the latest release and download the executable. Currently, this is only available for x86 Linux machines. pycompile links to LLVM 21.1 and Python 3.14, so the program may fail to launch if the system is unable to find the relevant shared libraries.
pycompile takes in Python source code files and compiles them into a single executable. This compiler also supports saving temporary files, similar to gcc or clang. This set of flags can halt the compilation process after generating bytecode, MLIR, or LLVM IR.
pycompile [options...] module1.py module2.py ...This repository contains a variety of example files in examples/ that demonstrate how to utilize the majority of pycompile's capabilities.
- CMake 3.22 or higher
- C++23 compatible compiler
- C++20 should also work with minimal modifications
- Python 3.14 development libraries
- Git (for fetching dependencies)
# Clone the repository
git clone https://github.com/matthew-pisano/pycompile
cd pycompile
# Configure and build
cmake -S . -B build
cmake --build buildThis project is built against Python 3.14, but should work with minimal modifications for versions after Python 3.11. This is due to the fact that Python makes few guarantees on its internal bytecode representation between versions. Bytecode may change without backward compatibility from one version to another. Therefore, changes to the required Ops may be needed for older versions.
To generate an executable from unmodified Python source code, several translations and lowerings are required:
- Python Bytecode - generated by the CPython API.
- PyIR - a custom MLIR dialect with Python-specific operations defined.
- LLVM IR - the intermediate representation used by clang.
- An executable - linked with the pystd static builtin library.
The translation between unmodified Python bytecode to PyIR (MLIR) requires some special handling. The CPython (bytecode) interpreter is a stack-based virtual machine while MLIR generates instructions for register-based machines. This means that during compilation, the stack must be simulated, with operations being popped from the stack before being translated to PyIR.
The remaining lowering process follows the standard for custom MLIr dialects in order to get the LLVM IR and eventually an executable. The builtin library is implemented in C++ instead of MLIR directly. This can be compiled and linked to the main executable as any other static library would. This library also contains the main() function for the program which sets up global variables and handles exceptions before handing control off to the compiled MLIR Python module.