Day 44 — Packaging Driven Development

Today I started looking into building cross-platform wheels for my Python C extension. I've never built binary extension wheels so I decided to start simple by building wheels for this snake game I want to write in C. Right now it's just a "hello world" ncurses program.

I started looking into wheels for some existing Python C extensions like opencv-python and xmlstarlet, and saw that the shared libraries are stored inside a .libs (Linux) or .dylibs (macOS) directory.

I also checked out their setup.pys and build processes, and soon it all began to make sense! I just needed to write a setup.py that handles compiling the C extension, and run python setup.py bdist_wheel (or pip wheel) on different operating systems!

opencv-python uses multibuild and xmlstarlet uses cibuildwheel to automate building wheels for different OSes. I went ahead with cibuildwheel as it had more detailed documentation.

Since I'm using pybind11 to wrap my C code, I used the setup.py from their python_example. And based on the cibuildwheel docs, I added this GitHub workflow where I install ncurses, and use auditwheel and delocate to bundle libncurses with the Linux and macOS wheels. (I couldn't find tools like auditwheel and delocate for Windows :( )

After downloading and installing the Linux wheel built by the workflow, I found that I needed to link against another shared library on Linux, so I had to add this to my setup.py:


  unix_l_opts = ["-lncurses"]
  if sys.platform == "linux":
      unix_l_opts.append("-ltinfo")

I also used fastmac to check if the macOS wheel worked. I wish there was a similar thing for Windows. fastwin? winfast?

Later, I found out that this game wouldn't work on Windows, because Windows doesn't support ncurses! I thought that maybe I could write some extra code for Windows using the curses module from the Python standard library (assuming that it's portable), but the Windows version of Python doesn't contain that module!

The Windows version of Python doesn’t include the curses module. A ported version called UniCurses is available. You could also try the Console module written by Fredrik Lundh, which doesn’t use the same API as curses but provides cursor-addressable text output and full support for mouse and keyboard input. — docs.python.org/3/howto/curses.html

There's PDCurses though, which is kinda like a curses port for Windows. Maybe I could try compiling the extension with PDCurses (or one of the alternatives from the Python docs above), and bundle the associated shared library in the Windows wheel.

Now that I have the packaging pipeline set up, I should focus on writing a snake game!