Tox template¶

The purpose of this template is to make it easy to map tox environments to Azure jobs. To use this template, your repository will need to have a tox.ini file. Note that you will also need to make sure you first load the templates as described in Loading the templates.

Basic setup¶

To use this template, you will need to add the following section to your azure-pipelines.yml file:

jobs:
- template: run-tox-env.yml@OpenAstronomy
  parameters:
    envs:
    - <os>: <tox env>
    - <os>: <tox env>

Where <os> is the operating system to test on, and <tox env> is the name of a tox environment. The operating system should be one of linux, linux32, macos, or windows. An example might be:

jobs:
- template: run-tox-env.yml@OpenAstronomy
  parameters:
    envs:
    - linux: pep8
    - macos: py37-test
    - windows: py36-docs

In many cases, this may be enough, but there are additional options you can specify, which we describe in the following sections.

Reporting coverage¶

To enable coverage reporting, add a parameter coverage that is set to the name of the service to use:

jobs:

- template: run-tox-env.yml@OpenAstronomy
  parameters:
    coverage: codecov
    envs:
    - ...

At this time, only codecov is supported.

Non-Python dependencies¶

To make sure that non-Python dependencies are installed before the tox environments are run, use the libraries parameter. This can have sections for the apt, yum, brew (or brew-cask), and choco tools which are used for linux, linux32, macos, and windows respectively, and each of these sections should contain a list of package names to install with these tools, e.g:

jobs:

- template: run-tox-env.yml@OpenAstronomy
  parameters:
    libraries:
      apt:
        - libopenjpeg5
      brew:
        - openjpeg
    envs:
    - ...

Note that as shown above, you don’t need to specify all tools, only the ones for which you need to install packages.

Tox Version¶

If you need to you you can add a tox version specifier eg. >=1 or ==3.2.1 , by defaul the most recent version is installed. This can be done via the toxverspec parameter.

jobs:

- template: run-tox-env.yml@OpenAstronomy
  parameters:
    toxverspec: <4
    envs:
    - ...

Pre-Tox Python dependencies¶

If you wish to install Python packages before tox is called, i.e. tox plugins, you can pass the toxdeps parameter. These packages are installed at the same time as tox itself.

jobs:

- template: run-tox-env.yml@OpenAstronomy
  parameters:
    toxdeps: tox-pypi-filter
    envs:
    - ...

Commandline Arguments to tox¶

If you wish to pass extra command line arguments to the tox command, you can specify toxargs.

jobs:

- template: run-tox-env.yml@OpenAstronomy
  parameters:
    envs:
    - linux: pep8
      toxargs: -i https://notpypi.org

X virtual framebuffer (Linux)¶

If you want to make use of the X virtual framebuffer (Xvfb) which is typically needed when testing packages that open graphical windows, you can set the xvfb parameter to true:

jobs:

- template: run-tox-env.yml@OpenAstronomy
  parameters:
    xvfb: true
    envs:
    - ...

This parameter only has an effect on Linux, and is ignored on other platforms.

Mesa OpenGL (Windows)¶

If you need to use OpenGL on Windows, you should set the mesaopengl option to install the Mesa OpenGL libraries:

jobs:

- template: run-tox-env.yml@OpenAstronomy
  parameters:
    mesaopengl: true
    envs:
    - ...

Conda¶

If you want tox to be run with tox-conda, include the string conda in your tox environment name. This will automatically result in conda getting set up, and tox-conda installed.

Positional arguments for tox¶

If you want to make use of the {posargs} functionality in your tox.ini file, you can specify positional arguments to pass to tox for each job using the posargs parameter:

jobs:
- template: run-tox-env.yml@OpenAstronomy
  parameters:
    envs:
    - linux: pep8
    - macos: py37-test
      posargs: -n=4
    - windows: py36-docs

Submodule Checkout¶

If you want to change the submodules setting to the Checkout task you can set the submodules parameter. For instance:

jobs:
- template: run-tox-env.yml@OpenAstronomy
  parameters:
    submodules: false
    envs:
    - linux: pep8

Setting or overriding options on a job by job basis¶

The coverage, libraries, posargs and xvfb parameters can be specified on a job by job basis instead of or as well as globally, and take precedence over global options:

jobs:
- template: run-tox-env.yml@OpenAstronomy
  parameters:
    coverage: codecov
    posargs: '-n=4'
    libraries:
      brew:
      - fftw
    envs:
    - linux: pep8
      coverage: false
      libraries: {}
      posargs: ''
    - macos: py37-test
    - linux: py36-test
      xvfb: true
    - windows: py36-docs
      libraries:
        choco:
          graphviz

In the above example, we have disabled coverage testing, posargs, and any libraries for the pep8 job, and overridden libraries so that graphviz gets installed on Windows.

Naming Jobs¶

Optionally you can name an env, which is useful if you want to refer to that job later in your pipeline, e.g. in the publish template’s dependsOn parameter.

jobs:
- template: run-tox-env.yml@OpenAstronomy
  parameters:
    envs:
    - linux: py36-test
      name: py36_test

Note, that job names in Azure pipelines can only contain A-Z, a-z, 0-9, and underscore. Which is why they are not automatically set from the tox env names, as they frequently have hyphens in.

Caching¶

Setting the cache_dirs parameter will cache all files in the specified directories. Caches are identified by a specified key. Once a cache is created with a particular key, it cannot be updated or replaced. The Azure documentation contains more information on how Azure manages caching.

A list of caches are defined according to the following specification. Each collection of mappings in the sequence under cache_dirs is passed to the input of a Cache@2 Azure task. See the Azure documentation for the full specification of input.

jobs:
- template: run-tox-env.yml@OpenAstronomy
  parameters:
    cache_dirs:
    - key: <cache name>
      path: <cached directory>
    - key: <cache name>
      path: <cached directory>
  envs:
  - <os>: <tox env>
    cache_dirs:
    - key: <cache name>
      path: <cached directory>
    - key: <cache name>
      path: <cached directory>

Azure will look for a cache with the key <cache name>. If it exists, it will be restored to the directory <cached directory>. If it doesn’t exist, at the end of the job the contents of <cached directory> will be cached with the key <cache name> and will be available for subsequent jobs.

The path specified by <cached directory> can be an absolute or relative path, with relative paths based at $(System.DefaultWorkingDirectory), which is usually the directory containing your package’s setup.py file.

By defining cache_dirs under parameters, the specified caches will be used for all envs. However, if cache_dirs is specified under a specific environment, that environment will only use this set of caches.

As an example, to cache pip packages you can set the PIP_CACHE_DIR environment variable and cache this directory. This will ensure that pip uses this directory as the cache, and that it is cached by Azure:

variables:
  PIP_CACHE_DIR: $(Pipeline.Workspace)/.pip

jobs:
- template: run-tox-env.yml@OpenAstronomy
  parameters:
    cache_dirs:
    - key: 'python | "$(Agent.OS)" | requirements.txt'
      restoreKeys: |
        python | "$(Agent.OS)"
        python
      path: $(PIP_CACHE_DIR)

Docker Jobs¶

This template has support for running tox inside docker containers. This was originally added to support testing against 32bit linux builds using the manylinux official docker images, so there is specific support for these images. When using docker the Xvfb, conda and libraries options will not work.

Manylinux¶

There are two options for using manylinux, you can set the os flag to either linux32 or manylinux. If it is set to linux32 all the commands will be prefixed with the linux32 command to set the architecture to i686.

By setting the OS flag to manylinux or linux32, the template will automatically select docker and use the manylinux2010_i686 image. Which can be overridden by specifying the manylinux_image parameter.

When using manylinux images, the libraries parameter will work, and you should use yum rather than apt as the tool name.

As a shortcut for the other docker options, when using manylinux you can set manylinux_image to the name of the container you want to use. This excludes the quay.io/pypa prefix and also excludes any tag (latest is always used).

Other Docker Images¶

You can also specify your own docker images in which to run tox. There are a few options available to control this behaviour, they all can only be specified on a per-env basis. The running of the docker commands are not dependant on the operating system, although setting the os to linux32 will cause all commands in the container to be prefixed with the linux32 setarch binary. The following example shows all the possible options even though some are redundant:

jobs:
- template: run-tox-env.yml@OpenAstronomy
  parameters:
    envs:
    - linux: <job name>
      docker_image: python:3.9.0rc1-slim-buster
      docker_name: python39
      docker_python: /usr/local/bin/python
      docker_cache: true

The options are as follows:

docker_image this is the name of the container to be created. It can be any valid argument to docker pull, i.e python or quay.io/pypa/manylinux2010_i686.
docker_name this is optional as long as docker_image is a valid container name. If you specify a tag in docker_image : and / will be replaced, so you will not need to specify docker_name. However, if you specify a more complex image you will need to manually specify the container name with docker_name.
docker_python this is the path inside the container to the docker executable.
docker_cache this enables caching of docker_image. This is optional and default is no caching.