Day 53 — A tool to list PE file dependencies

Today I read up on some numpy docs to try and find how they bundle the OpenBLAS DLL with the numpy wheel. I've updated the day 51 post with the things that I found.

I also packaged the find_dll_dependencies function from wheel_repair.py into a convenient CLI tool which can be used to list dependencies for a PE file.

  $ pedep pdftopng.cp38-win_amd64.pyd
  Imports for pdftopng.cp38-win_amd64.pyd:
    - MSVCP140.dll
    - python38.dll
    - KERNEL32.dll
    - VCRUNTIME140_1.dll
    - VCRUNTIME140.dll
    - api-ms-win-crt-runtime-l1-1-0.dll
    - api-ms-win-crt-stdio-l1-1-0.dll
    - api-ms-win-crt-string-l1-1-0.dll
    - api-ms-win-crt-heap-l1-1-0.dll
    - api-ms-win-crt-convert-l1-1-0.dll
    - api-ms-win-crt-time-l1-1-0.dll
    - api-ms-win-crt-math-l1-1-0.dll
    - api-ms-win-crt-multibyte-l1-1-0.dll
    - api-ms-win-crt-locale-l1-1-0.dll
    - api-ms-win-crt-filesystem-l1-1-0.dll
    - freetype.dll
    - libpng16.dll
    - jpeg62.dll
    - ADVAPI32.dll

There's also a --json option which will print all the imports as a JSON string.

And if you pass in a --dll-dir which might contain some of the DLL dependencies, pedep will recursively list down dependencies for every file it finds in the --dll-dir:

  $ pedep --dll-dir C:\path\to\dll\directory pdftopng.cp38-win_amd64.pyd
  Imports for pdftopng.cp38-win_amd64.pyd:
    - MSVCP140.dll
    - python38.dll
    - KERNEL32.dll
    - VCRUNTIME140_1.dll
    - VCRUNTIME140.dll
    - api-ms-win-crt-runtime-l1-1-0.dll
    - api-ms-win-crt-stdio-l1-1-0.dll
    - api-ms-win-crt-string-l1-1-0.dll
    - api-ms-win-crt-heap-l1-1-0.dll
    - api-ms-win-crt-convert-l1-1-0.dll
    - api-ms-win-crt-time-l1-1-0.dll
    - api-ms-win-crt-math-l1-1-0.dll
    - api-ms-win-crt-multibyte-l1-1-0.dll
    - api-ms-win-crt-locale-l1-1-0.dll
    - api-ms-win-crt-filesystem-l1-1-0.dll
    - freetype.dll
    - libpng16.dll
    - jpeg62.dll
    - ADVAPI32.dll

  Imports for freetype.dll:
    - zlib1.dll
    - bz2.dll
    - libpng16.dll
    - brotlidec.dll
    - VCRUNTIME140.dll
    - api-ms-win-crt-convert-l1-1-0.dll
    - api-ms-win-crt-string-l1-1-0.dll
    - api-ms-win-crt-utility-l1-1-0.dll
    - api-ms-win-crt-environment-l1-1-0.dll
    - api-ms-win-crt-stdio-l1-1-0.dll
    - api-ms-win-crt-heap-l1-1-0.dll
    - api-ms-win-crt-runtime-l1-1-0.dll
    - KERNEL32.dll

  Imports for zlib1.dll:
    - VCRUNTIME140.dll
    - api-ms-win-crt-stdio-l1-1-0.dll
    - api-ms-win-crt-heap-l1-1-0.dll
    - api-ms-win-crt-convert-l1-1-0.dll
    - api-ms-win-crt-runtime-l1-1-0.dll
    - KERNEL32.dll

  Imports for bz2.dll:
    - VCRUNTIME140.dll
    - api-ms-win-crt-stdio-l1-1-0.dll
    - api-ms-win-crt-heap-l1-1-0.dll
    - api-ms-win-crt-runtime-l1-1-0.dll
    - api-ms-win-crt-string-l1-1-0.dll
    - api-ms-win-crt-math-l1-1-0.dll
    - KERNEL32.dll

  Imports for libpng16.dll:
    - zlib1.dll
    - VCRUNTIME140.dll
    - api-ms-win-crt-heap-l1-1-0.dll
    - api-ms-win-crt-stdio-l1-1-0.dll
    - api-ms-win-crt-math-l1-1-0.dll
    - api-ms-win-crt-runtime-l1-1-0.dll
    - api-ms-win-crt-convert-l1-1-0.dll
    - api-ms-win-crt-filesystem-l1-1-0.dll
    - api-ms-win-crt-time-l1-1-0.dll
    - KERNEL32.dll

  Imports for brotlidec.dll:
    - brotlicommon.dll
    - VCRUNTIME140.dll
    - api-ms-win-crt-heap-l1-1-0.dll
    - api-ms-win-crt-runtime-l1-1-0.dll
    - KERNEL32.dll

  Imports for brotlicommon.dll:
    - VCRUNTIME140.dll
    - api-ms-win-crt-heap-l1-1-0.dll
    - api-ms-win-crt-runtime-l1-1-0.dll
    - KERNEL32.dll

  Imports for jpeg62.dll:
    - VCRUNTIME140.dll
    - api-ms-win-crt-environment-l1-1-0.dll
    - api-ms-win-crt-heap-l1-1-0.dll
    - api-ms-win-crt-stdio-l1-1-0.dll
    - api-ms-win-crt-runtime-l1-1-0.dll
    - KERNEL32.dll

If I can somehow get the dependency file paths from the PE file itself (I'm not sure if that's possible), or emulate the behavior of the Windows dynamic library loader to look for DLLs in every possible Windows directory where they might occur (C:\Windows, C:\Windows\System32, etc. or maybe every directory on the PATH variable?!), I could remove the need for a --dll-dir option!

I also watched this talk by Steve Dower where he shows some statistics on how more than 50% VS Code and PyCharm users are on Windows, and talks about how we should make the code we release more inclusive by supporting Windows as a platform. He mentions the following things that can help us do that:

• Support and document the usage of python -m to run tools because that's the most reliable way to run things with the Python version that you want.
• On Windows, paths are formed with a backslash C:\Users\Steve\Documents, so a path.split('/') in your code won't work. Use pathlib from the standard library to do path manipulations and let that module handle all the different path formats on various operating systems.
• Use appdirs to store app settings instead of storing it manually in something like ~/.config/app.config (Windows has no concept of ~ for home dirs). (If you're using click, then you can use click.get_app_dir() which is built on top of appdirs!)
• Always supply an encoding when opening user files, don't guess and throw away parts of their language! He also mentioned some bits about how Windows 1.0 was released before Unicode 1.0, and how utf-16 is the default on Windows. My Unicode understanding has gotten a bit rusty, so I need to watch Ned Batchelder's awesome Unicode talk again and then get back to this point again!
• Everyone can install PyPI packages on Windows (initally, wheels on PyPI were only supported on Windows!) Use a free CI like Azure pipelines / Appveyor to test your packages on Windows, and even build wheels when it's time for a release! You can also collaborate with your Windows users to test the code you release, and even build wheels!