This page describes how Indigo 2023.2.0 works with Python, specifically with respect to packages/libraries.

Indigo 2023.2 shipped with API version 3.4 and is the first version to contain support for Python version 3 (check the API Version Chart to see what version of the API is available in which Indigo versions). With this version of the API, Python 3 is now automatically installed as part of the Indigo installer. For Indigo 2023.2, we are including Python 3.11.6.

You may know that Indigo plugins run in a process called the IndigoPluginHost (IPH). With this release, there are now 2 versions of the IPH process: IndigoPluginHost (IPH2 for clarity), which runs the legacy Python 2 API, and IndigoPluginHost3 (IPH3) which runs the new Python 3 based API. We determine which version to use when starting a plugin (by which API the plugin specifies). API 3.0 and later will run the IPH3 process, anything earlier will run the older IPH2 process.

For this release, all embedded and external Python scripts will be run in IPH3 (and therefore Python 3) - so you will need to update your scripts if they aren't working (we can't automatically tell if they will work or not unfortunately). We believe that simple scripts will most likely work without change, but if not, you can walk through the document mentioned above and perhaps spot the changes you need. You can also post your scripts on the Help Converting to Python 3 forum to get help with any conversion issues.

The Indigo 2023.2 installer includes the Python installer (including Intel and Apple Silicon support) from https://www.python.org - the official Mac installers for the Python language. As a result, any installs of other versions of Python from that website may cause unexpected results in Indigo and potentially cause it to break. We've taken all precautions we can to try and avoid conflicts, but there is only so much we can do. So please use caution when installing other Python versions to your server machine. If you have any questions about other Python versions, post your questions on our forums.

As noted above, Indigo 2023.2 ships with Python 3.11 and which contains several modules in addition to Python's standard core modules. The specific modules installed with each version of Python are listed below. However, in some instances, you may need to manually install one or more additional Python packages to support a third-party plugin or your own custom scripts. The manner in which this is done, depends on the Indigo API version that the plugin supports.

Use pip3 install [package] commands to install packages:

Last login: Tue jan 1 12:34:56 on console
user@my mac ~ % pip3 install tensorflow
...

To see which packages are installed, use the list command:

Last login: Tue jan 1 12:34:56 on ttys000
user@my mac ~ % pip3 list
Package            Version
------------------ ---------
astroid            2.9.3
certifi            2021.10.8
charset-normalizer 2.0.11
cycler             0.11.0
fonttools          4.29.1
...
user@my mac ~ %

Note: Some Python packages – and specifically those that are included in IPH3 as described below – will not appear in the pip3 list output.

To see more detailed information about an individual package, use the show command.

Last login: Sun Mar 27 10:20:04 on ttys000
user@my mac ~ % pip3 show requests
Name: requests
Version: 2.27.1
Summary: Python HTTP for Humans.
Home-page: https://requests.readthedocs.io
Author: Kenneth Reitz
Author-email: me@kennethreitz.org
License: Apache 2.0
Location: /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages
Requires: charset-normalizer, certifi, idna, urllib3
Required-by: 
user@my mac ~ % 

Particularly helpful in the show output is the dependencies information – both what the package requires and what requires the package. Note that the dependencies information may not include information about packages that are included with the IPH.

Installing additional Python versions on the Indigo 2023.2 server machine is NOT recommended (see above). Installing additional Python modules should only be done when necessary.

Because the packages included with Indigo should not be modified (to ensure a smoothly running server), plugin developers must include any other desired version within the plugin package. Plugin developers should note that some Python packages will not integrate universally across all installations as they rely on code compiled for specific hardware configurations (for example, Intel vs. Apple silicon). The IPH processes use the following Module Search Paths (Python interpreters use PYTHONPATH to describe the order of directories to search for installed packages, the first matching package will be used):

The PYTHONPATH for IPH3 is:

1. Contents/Server Plugin (inside the plugin bundle)
2. Contents/Packages (inside the plugin bundle)
3. IndigoPluginHost3 (inside the app itself)
4. /Library/Application Support/Perceptive Automation/Python3-includes
5. /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages

Packages are installed in several ways:

1. You may add modules/packages that you develop yourself to the Python#-includes folders above, and those will be shared across all Indigo Plugins and Scripts.
2. Other packages are installed using a pip3 process. Warning: Indigo will preinstall several packages (listed below) at install time. You should avoid modifying those unless absolutely necessary as other 3rd party plugins may rely on them.

If you are going to instruct users to install packages that support your plugin using pip, you should probably do one of the following:

If your plugin requires libraries that are not part of the basic Python installation or packages installed by Indigo, the preferred method is to include a list of those modules so Indigo can install them automatically when your plugin is installed. This automatic process is available in Server API version 3.4 and later (prior API versions will not invoke this process). This is handled by the following process:

If a plugin contains the plain text file: Contents/Server Plugin/requirements.txt, we'll run pip to install the listed modules into Contents/Packages (an example is shown below).

1. When a plugin starts up, it'll look for the Contents/Server Plugin/requirements.txt file. If it's not there, we'll continue with plugin startup. If it is there, we'll proceed to the next step.
2. Indigo will check to see if this file exists: Contents/Packages/pip-install-log-success.txt. If it does exist (meaning that we've already successfully installed requirements before), we will continue plugin startup. If it doesn't exist, we'll proceed to the next step.
3. We run pip3 with the requirements file and target the Contents/Package directory (all libraries and dependencies will be installed there). If it fails, we output an error and log the install attempt to the Event Log window and stop the plugin. If it installs correctly, we write the install log results to the Contents/Packages/pip-install-log-success.txt file so that the next time it starts it won't do it again (see previous step above).

If processing the requirements.txt file has failed (say, if the network is down so that pip can't get to pypi.org to download), it will continue to try when the user reloads the plugin, so transient issues will eventually resolve themselves. If the error is caused by a problem with the requirements.txt file or with a specific requirement, the developer will need to address in a new version.

Here is a sample requirements.txt file. Note that the following includes references to specific versions of the packages to be installed. We strongly recommend this approach to minimize the possibility that a newer version could be installed which may cause problems with your plugin.

zeroconf==0.130.0
websocket-client==1.7.0

Troubleshooting:

If you find that packages are not being installed when a user installs/updates your plugin, one cause might be that you have a Contents/Packages/pip-install-log-success.txt file left over in your plugin bundle. If it's in the bundle, Indigo will assume the packages are already installed.

Using pip:

If you're going to instruct users to install needed packages individually, they should use a command similar to:

pip3 install package -t /Path/To/Plugin/Contents/Packages/

or, if a specific version is required, use:

pip3 install package==1.2.3 -t /Path/To/Plugin/Contents/Packages/

which will install the libraries to the specified folder that exists in your plugin bundle. It is strongly recommended that you use the Packages folder as this may be required in future versions of Indigo.

Logging Import Errors

If your plugin requires one or more libraries that your users will install manually, you should provide some helpful messages to guide them through the process. One such way is to monitor import failures and notify users that they will need to install some additional libraries in order to use your plugin, such as:

import_errors = []
try:
    import zeroconf
except ImportError:
    import_errors.append("zeroconf")

def startup(self):
    self.logger.debug("startup called")

    if len(import_errors):
        msg = f"Required Python libraries missing.  Run the following command(s) in a Terminal window to install them, then reload the plugin.\n\n"
        for i in import_errors:
            msg += f'pip3 install {i} -t "{self.pluginFolderPath}/Contents/Packages/"\n'
        self.logger.error(msg)
        return "Plugin startup cancelled due to missing Python libraries."

There are some important implications that result from having some libraries included in the IPH3. Namely, those packages will be available to plugins, Indigo scripts (embedded and linked), the Indigo python shell (from the menu item or by doing indigo-host in a terminal window), but IPH3 libraries will NOT be available if you start the python interpreter directly (by doing python2 or python3 in a terminal window). Refer to this chart to determine accessibility:

In this release, all Python scripts (either embedded or linked) are executed using the IPH3 process using Python 3. Therefore, the server will be able to locate any packages your scripts use in the same way as described above:

1. IndigoPluginHost3
2. /Library/Application Support/Perceptive Automation/Python3-includes
3. /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages

If you need a module/package for your plugin, you can install it via pip3 as shown above, which will install it into the Python site-packages folder. We recommend (but don't require) the following folders be used for custom scripts and modules:

• /Library/Application Support/Perceptive Automation/Scripts – any linked scripts you use for Indigo Schedules and Actions using the “Execute Script (Script and File Actions” command should be stored in this folder.
• /Library/Application Support/Perceptive Automation/Python3-includes – any modules you have developed (a module is a file that contains classes, functions, constants, etc.) that you want to share across multiple scripts, should be stored in the /Library/Application Support/Perceptive Automation/Python3-includes folder. (Indigo versions prior to 2022 should use the /Library/Application Support/Perceptive Automation/Python2-includes folder).

Warning 1: as mentioned above, Indigo will preinstall several packages (listed below) at install time. You should avoid modifying those unless absolutely necessary as other 3rd party plugins may rely on them.

Warning 2: you should use caution when naming your custom scripts to ensure they don't supplant other Python packages. For example, if you install the holidays Python package using pip3 install holidays, it will be installed in the site-packages folder. If you also have a custom script in the Python3-includes folder named holidays.py, the IPH will import your script instead of the site-packages version (because the Python3-includes folder is searched before the site-packages folder).

To provide users with a measure of safety regarding the Python packages that Indigo installs, the Indigo installer also includes the capacity to repair the Python installation if something should become damaged. For example, if a user were to accidentally upgrade one of the Python packages – and as a result cause the Indigo server to stop functioning properly – re-running the Indigo installer may be able to bring things back under control. There's a few things you can try should you run into conflicts between Python versions.

Re-running the Indigo installer on an existing server shouldn't affect user-installed libraries (existing user-installed libraries should be unaffected).

Depending on the method used, any Python modules installed via pip3 may need to be re-installed on the new machine. For more information, consult Moving an Indigo Installation to Another Mac.

Indigo 2023.2 ships with support for the following Python 3 packages in addition to the packages installed with the standard Python framework. Generally, if earlier versions of the listed packages are installed at the location Indigo uses, they will be upgraded to the version numbers below. The >= symbol means the version installed by Indigo may be newer than the version listed, but will not be older. pyserial is also installed but is installed differently than the other packages listed above and should be unmodifiable.