Do you resent the fact that Go programmers can whip up an executable and distribute it to end users with relative ease? The ability to use your software without downloading or installing anything is a huge plus. That is the goal, and the Python environment has a tool called PyInstaller that can help you get there.
When developing Python libraries, it’s helpful to have access to the many available guides on topics such as virtual environment creation, dependency management, avoiding dependency traps, and publishing to the Python Package Index (PyPI). Python programmers have significantly less resources at their disposal. This guide is intended for programmers who plan to share their work with end users who may or may not themselves be proficient in Python. The following are the topics covered in this guide:
- How PyInstaller can simplify application distribution
- How to use PyInstaller on your own projects
- How to debug PyInstaller errors
- What PyInstaller can’t do
PyInstaller allows you to make a folder or executable that can be launched immediately without any further setup on the user’s end. It is helpful to review some of the distribution issues that PyInstaller solves in order to properly grasp its potential.
avoid.
Distribution Problems
When you’re not a coder, setting up a Python project can be a real pain. Opening a Terminal is frequently the first step in the setup process, which is a dealbreaker for a large population of potential customers. Before the installation guide even begins to get into the specifics of virtual environments, Python versions, and the plethora of possible dependencies, this stumbling block prevents customers from proceeding.
Consider the steps you normally take to prepare a computer for Python development. It most likely goes like this:
this:
- Download and install a specific version of Python
- Set up pip
- Set up a virtual environment
- Get a copy of your code
- Install dependencies
Think for a second about whether any of the above actions make sense to someone who is not a developer, much less a Python coder. Most likely not.
If your user makes it to the requirements section of the installation, these issues will balloon out of control. Wheels have made this a lot easier in recent years, but there are still some dependencies that need C/C++ or even FORTRAN compilers.
If you want to reach the largest potential audience with your app, this is an extremely high hurdle to overcome. There “has to be a better way,” as Raymond Hettinger says in several of his insightful speeches.
way.”
PyInstaller
PyInstaller hides this complexity from the user by tracking down and aggregating all of the necessary dependencies. Because the Python Interpreter is included in your app, your users won’t even notice they’re working with a Python program. Put away your manuals with their convoluted instructions!
PyInstaller accomplishes this remarkable feat by inspecting your Python code, finding your dependencies, and subsequently packaging everything up in a way that is appropriate for your operating system.
The intricacies of PyInstaller’s operation and use are fascinating, but for the time being we’ll stick to the fundamentals. If you need more information, you can always check out the PyInstaller documentation.
PyInstaller also has the capability of producing executables for Windows, Linux, and macOS. This means a.exe file for Windows users, a standard executable for Linux users, and a.app bundle for macOS users. However, there are exceptions to this rule. For more on this, see the caveats.
information.
Preparing Your Project
You need a command line interface (CLI) script to launch your application in order to use PyInstaller. Making a small script outside of your Python package that does nothing more than import your package and run main() is a common way to do this.
Python is used for the primary script that users will run. Almost anything can be done in the entry-point script, but explicit relative imports should be avoided. If you prefer to use relative imports in the remainder of your program, you can continue to do so. An entrance point is the part of your program that actually launches the application or project.
Try it out in your own endeavor, or use the Real Python feed reader as a guide. You can learn more about how to publish the reader project on PyPI by reading the tutorial on doing so.
Including the entry-point script is the first step in creating an executable version of this project. Fortunately, the feed reader project is neatly organized, and can be put to use with just a single external script. If you’re using the reader package, for instance, you can make a file named cli.py that contains the following:
code:
from reader.__main__ import main
if __name__ == '__main__':
main()
The feed reader is launched by main(), which is called from within cli.py.
When working on your own project, writing an entry-point script is simple because you already know the ins and outs of the code. Finding the starting point in someone else’s code, however, is more difficult. The setup.py file in the external project is a good place to start.
Check the setup.py file for a mention of the entry_points option. Here’s the reader project’s setup.py, for instance:
:
setup(
name="realpython-reader",
version="1.0.0",
description="Read the latest Real Python tutorials",
long_description=README,
long_description_content_type="text/markdown",
url="https://github.com/realpython/reader",
author="Real Python",
author_email="info@realpython.com",
license="MIT",
classifiers=[
"License :: OSI Approved :: MIT License",
"Programming Language :: Python",
"Programming Language :: Python :: 2",
"Programming Language :: Python :: 3",
],
packages=["reader"],
include_package_data=True,
install_requires=[
"feedparser", "html2text", "importlib_resources", "typing"
],
entry_points={"console_scripts": ["realpython=reader.__main__:main"]},
)
You can see that the function specified in the entry_points argument is the one called by the cli.py script’s entry point.
If you checked out your reader project into a folder also named reader, the directory structure after this edit should resemble this:
:
reader/
|
├── reader/
| ├── __init__.py
| ├── __main__.py
| ├── config.cfg
| ├── feed.py
| └── viewer.py
|
├── cli.py
├── LICENSE
├── MANIFEST.in
├── README.md
├── setup.py
└── tests
There is a new file, cli.py, but no modifications to the reader code. Typically, just this one script will get you up and running using PyInstaller for your project.
The use of __import__() or imports within a function is something else to watch out for. In the language of PyInstaller, these are known as “hidden imports.”
If modifying your program’s imports is too time-consuming or complicated, you can explicitly specify the hidden imports to have PyInstaller install those prerequisites anyway. Later on in this guide, you’ll learn how to accomplish this.
When your program can be run from a Python script outside of its package, you’re ready to try out PyInstaller.
executable.
Using PyInstaller
The first order of business is to grab PyInstaller from PyPI and set it up. The standard Python package installation tool, pip, can be used for this.
:
$ pip install pyinstaller
The prerequisites for running PyInstaller, as well as the pyinstaller command, will be installed by pip. While PyInstaller can be used as a library in your Python code, its primary purpose is on the command line.
If you want to write your own hook files, you’ll need to learn how to use the library interface.
If you have only pure Python dependencies, PyInstaller’s defaults are more likely to generate an executable. But don’t worry too much if your dependencies are more complex and involve C/C++ add-ons.
Many widely used packages, such as NumPy, PyQt, and Matplotlib, are automatically supported by PyInstaller. Refer to the PyInstaller docs for further information on the official list of packages that PyInstaller supports.
Don’t freak out if not all of your prerequisites can be found in the canonical documentation. There are a lot of usable Python packages. The popularity of PyInstaller has led to the inclusion of instructions for using it in a wide variety of projects.
In other words, you have a good shot at having your project succeed right away.
Simply provide PyInstaller with the name of your primary entry-point script, and it will attempt to generate an executable with all the default settings.
To begin, cd to the directory containing your entry point, and then run the pyinstaller command, which should now be on your PATH since PyInstaller was installed.
If you’re following along with the RSS reader, cd into the root directory, and then type:
project:
$ pyinstaller cli.py
If your executable’s build produces a lot of output, don’t worry about it. The default setting for PyInstaller’s verbosity is helpful for troubleshooting, but you can increase it to ridiculous levels if necessary.
later.
Digging Into PyInstaller Artifacts
PyInstaller is complex beneath the surface and will generate a great deal of logs. Therefore, prioritization is essential. The executable that can be sent to end users and any possible debugging data. The pyinstaller script will automatically generate a few
interest:
-
A
*.spec
file -
A
build/
folder -
A
dist/
folder
Spec File
By default, the name of your CLI script will be used for the spec file. To continue with the preceding example, a file with the name cli.spec will be displayed. After executing PyInstaller on cli.py, the resulting default spec file appears below.
file:
# -*- mode: python -*-
block_cipher = None
a = Analysis(['cli.py'],
pathex=['/Users/realpython/pyinstaller/reader'],
binaries=[],
datas=[],
hiddenimports=[],
hookspath=[],
runtime_hooks=[],
excludes=[],
win_no_prefer_redirects=False,
win_private_assemblies=False,
cipher=block_cipher,
noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
cipher=block_cipher)
exe = EXE(pyz,
a.scripts,
[],
exclude_binaries=True,
name='cli',
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=True,
console=True )
coll = COLLECT(exe,
a.binaries,
a.zipfiles,
a.datas,
strip=False,
upx=True,
name='cli')
The pyinstaller command will generate this file automatically. Even if your version’s routes won’t be identical to the original’s, they should be similar.
You can still use PyInstaller even if you don’t comprehend the preceding code.
You can make changes to this file and use it again to generate executables at a later time. If you replace the entry-point script with this spec file and run pyinstaller, the resulting builds will be a little quicker.
PyInstaller spec files have a limited number of applications. Unless your project is highly customized, however, you can ignore those concerns when working on a simple one.
built.
Build Folder
Most of the metadata and internal bookkeeping for generating your executable are stored in the build/ subdirectory by PyInstaller. The default content will resemble
this:
build/
|
└── cli/
├── Analysis-00.toc
├── base_library.zip
├── COLLECT-00.toc
├── EXE-00.toc
├── PKG-00.pkg
├── PKG-00.toc
├── PYZ-00.pyz
├── PYZ-00.toc
├── warn-cli.txt
└── xref-cli.html
In most cases, you may safely ignore the build folder unless you run into issues. Later on, you’ll get into more detail regarding debugging.
tutorial.
Dist Folder
After the construction completes, a dist/ folder resembling the
following:
dist/
|
└── cli/
└── cli
The user-facing deliverable is stored in the dist/ subdirectory. There is a folder with the name of your entry point inside the dist/ folder. To follow along with this example, our application’s executable and all required dependencies can be found in the dist/cli folder. To launch the program, navigate to the dist/cli/cli directory, or, if using Windows, dist/cli/cli.exe.
Depending on your OS, you may also encounter many files with the extensions.so,.pyd, and.dll. These are the shared libraries that PyInstaller compiled to represent your project’s dependencies. Add *.spec, build, and dist to your.gitignore file to prevent git from modifying these directories.
A clean status in git version control. The
The gitignore file that is used as the default for Python projects on GitHub takes care of this for you.
You should share the full dist/cli folder, but feel free to change the name to something more appropriate.
If you’ve been following along with the feed reader example, you may now try executing the dist/cli/cli executable.
Errors referencing the version.txt file will appear when you attempt to launch the program. This is because PyInstaller is unaware of the existence of some data files needed by the feed reader and its dependencies. Fixing that will involve informing PyInstaller that version.txt is necessary; you’ll find this out during testing of your new executable.
.
Customizing Your Builds
Spec files and regular CLI options are both acceptable means of providing options to PyInstaller. Some of the more popular and practical choices are listed below.
–name
You need to rename your program’s executable.
This prevents your entry-point script from being used as the name of directories containing your executable, spec file, and build artifacts. Using –name is helpful if, like me, you give your main script a name like cli.py.
With the command like, you may compile the cli.py script into an executable file named realpython.
this:
$ pyinstaller cli.py --name realpython
–onefile
Create a single executable file that contains the complete application.
In contrast to the usual settings, which generate a dependency folder in addition to the executable, the –onefile option simplifies distribution by generating only the executable.
There is no counterargument to this choice. The build command allows you to compile your work into a single executable.
this:
$ pyinstaller cli.py --onefile
The preceding command will consolidate all dependencies into a single executable, eliminating the need for a dist/ subdirectory.
–hidden-import
If there are several top-level imports that PyInstaller missed, please list them below.
Using __import__() and import inside functions is one approach to get around problematic code. Multiple instances of –hidden-import can be used in a single command.
To use this feature, you must specify the name of the package you wish to integrate into your program. For instance, PyInstaller will not incorporate the requests library into your executable if your project imports it inside a function. The following command could be used to compel requests to be
included:
$ pyinstaller cli.py --hiddenimport=requests
In your build command, you can provide this many times, once for each hidden import.
Options: –add-data, –add-binary
Tell PyInstaller to include specific files, either textual or binary, in the final product.
Include non-code data like configuration files, examples, and so on. If you’re keeping up with the feed reader project, you’ll see an example of this in the future.
–exclude-module
You can choose not to include some components in your executable.
This is helpful for excluding necessities that are exclusive to developers, such as testing frameworks. This is a fantastic method for minimizing the size of the delivered artifact to the user. You may wish to ignore this, for instance, if you use pytest.
executable:
$ pyinstaller cli.py --exclude-module=pytest
-w
Do not have stdout logging trigger the opening of a console window.
Only if you’re making a program with a graphical user interface will this be helpful. This ensures that end users never have to interact with a terminal, which is useful for keeping implementation details secret.
Like the –onefile option, -w does not require any input.
arguments:
Spec file
$ pyinstaller cli.py -w
.spec
As was previously indicated, you can modify your executable even more by reusing the.spec file that was generated automatically. Spec is just a standard Python script, and it uses the PyInstaller library API without even realizing it.
Because it’s just a regular Python script, you can pretty much do whatever in there. For further details, read the official documentation for the PyInstaller Spec file.
API.
Testing Your New Executable
Your new executable should be tested on fresh hardware. Your build machine’s operating system should also be installed on the new machine. This device should mimic the user experience as closely as feasible. The next best thing is to test it on your own machine, but that isn’t always feasible.
The trick is to launch the executable that is the result outside of your development environment. This means launching Python outside of any sandbox, be it virtualenv, conda, or another. Keep in mind that one of the primary tenets of a PyInstaller-built executable is that it not require any additional software to be installed on the user’s computer.
Continuing with the feed reader example, you’ll see that the default cli executable located in the dist/cli subdirectory produces an error when run. The error, however, directs you to the
problem:
FileNotFoundError: 'version.txt' resource not found in 'importlib_resources'
[15110] Failed to execute script cli
A version.txt file is necessary for the importlib_resources package. The –add-data option allows you to include this file in the build. See this working example of a version.txt file to see how it should look.
file:
$ pyinstaller cli.py \
--add-data venv/reader/lib/python3.6/site-packages/importlib_resources/version.txt:importlib_resources
PyInstaller will now include the version.txt file from the importlib_resources folder in your build, as instructed by the command. Please take note that the pyinstaller commands utilize the tilde () character to improve readability. If you’re executing these commands on your own, you may skip the and just copy and paste them as is, assuming you’re using the same paths.
Depending on where you put the feed reader’s prerequisites, you may need to change the path in the preceding command.
A new config.cfg file error will appear when the updated executable is launched.
The feed reader project expects you to include this file in your submission.
build:
$ pyinstaller cli.py \
--add-data venv/reader/lib/python3.6/site-packages/importlib_resources/version.txt:importlib_resources \
--add-data reader/config.cfg:reader
Again, the location of the file relative to the feed reader project will determine the correct path to use.
You should now have a fully functional executable that can be handed over to
users!
Debugging PyInstaller Executables
As you saw above, there are a number of potential issues that could arise while running your executable. The solutions could be as elementary as incorporating data files, like in the case of a feed reader, into your project. However, additional debugging methods may be required on occasion.
Listed here, in no particular sequence, are some standard tactics. If you’re having trouble debugging, try one of these methods, or try combining them.
sessions.
Use the Terminal
Launch the program in a terminal to examine the complete log.
If you want to view all of the stdout in a console window, remember to remove the -w build parameter. If a required library is missing, you may encounter ImportError issues.
missing.
Debug Files
Check for errors in the build/cli/warn-cli.txt file. PyInstaller generates copious amounts of output to clarify its actions. You can learn a lot by exploring the build/ subdirectory.
start.
Single Directory Builds
If you want to distribute multiple files instead of just one, use the –onedir option. This is the default setting, once again. Instead of having everything buried in a single executable, building with –onedir allows you to see each and every dependency.
While –onedir is helpful for debugging, –onefile is more convenient in most situations.
users to fully grasp. It’s possible that –onefile mode will be preferable after debugging.
distribution.
Additional CLI Options
You can modify the quantity of build-time output printed by PyInstaller. Build the executable again, this time telling PyInstaller to log at the DEBUG level, and inspect the results.
Using –log-level=DEBUG to make PyInstaller more verbose will cause it to produce a large amount of output. Instead of navigating through the Terminal, saving this output to a file will allow you to easily refer to it later. You can use the shell’s built-in redirection feature for this purpose. This is an example of
example:
$ pyinstaller --log-level=DEBUG cli.py 2> build.txt
The above command will generate a file named build.txt that contains numerous extra DEBUG messages. Please take note that the > redirection is insufficient. All of PyInstaller’s output is sent to the stderr file.
not a stodgy person. The stderr stream must be redirected to a file, which can be done with a 2 as in the preceding command.
An example build.txt file is shown below.
like:
67 INFO: PyInstaller: 3.4
67 INFO: Python: 3.6.6
73 INFO: Platform: Darwin-18.2.0-x86_64-i386-64bit
74 INFO: wrote /Users/realpython/pyinstaller/reader/cli.spec
74 DEBUG: Testing for UPX ...
77 INFO: UPX is not available.
78 DEBUG: script: /Users/realptyhon/pyinstaller/reader/cli.py
78 INFO: Extending PYTHONPATH with paths
['/Users/realpython/pyinstaller/reader',
'/Users/realpython/pyinstaller/reader']
What was included in your build, what was left out, and how the executable was packaged may all be found in this file.
You can get even more information by using the –debug option when you recompile your executable using the –log-level option. The -y and –clean options are helpful when recompiling, especially when setting up your builds for the first time or when utilizing
Integration That Never Stops. Using these settings, you can get rid of outdated builds and avoid providing any input to the build process.
process.
Additional PyInstaller Docs
There are many helpful resources and troubleshooting advice on the PyInstaller GitHub Wiki. The parts on correct packaging and what to do if something goes wrong stand out the most.
.
Assisting in Dependency Detection
If PyInstaller was unable to correctly discover all of your dependencies, you will most likely see ImportError problems. The use of __import__(), imports within functions, and other forms of hidden imports might lead to this issue.
The –hidden-import PyInstaller CLI option is a common solution to these sorts of issues. If a module or package is not found during the installation, you can force PyInstaller to add it by using this option. This is the simplest workaround for your application’s extensive use of dynamic import magic.
Hook files are another method for avoiding issues. These files provide extra information that PyInstaller can use to better package a dependency. The –additional-hooks-dir command-line option enables you to construct your own hooks and instruct PyInstaller to utilize them.
Since hook files are fundamental to how PyInstaller operates, they are abundant in the project’s source code.
.
Limitations
While PyInstaller’s capabilities are vast, it does have a few restrictions. Hidden imports and relative imports in entry-point scripts are two examples of these restrictions.
PyInstaller allows you to build Windows, Linux, and macOS executables, however it does not support cross compilation. Therefore, you can’t create an application for one OS to run on another OS. You will need a build machine for each supported operating system if you intend to distribute executables for those systems.
The fact that PyInstaller does not officially bundle everything your application needs to operate is useful to know, since it relates to the cross compile constraint. Your executable still requires glibc to run. The glibc restriction is usually avoidable by using the oldest version of each target operating system when constructing.
If, for instance, your intended users run a variety of Linux distributions, you can choose to construct your application on an older version of CentOS. This will ensure that your project is compatible with most versions newer than the one you are developing on. The PyPA approves this method for constructing compatible wheels because it is identical to that specified in PEP 0513.
Actually, you may want to look at utilizing the manylinux docker image available from the PyPA as your Linux build environment. You may create a build image that works with most distributions of Linux by starting with the base image and installing PyInstaller and all your dependencies.