Testing

Prerequisites

Our test suite is based on tox. It allows us to run tests in all supported Python environments with a single tox command and also automates checks for package build, docs, coverage, code style, and performance.

So, in order to run Xentica tests you at minimum have to set up the CUDA environment properly (as described above), and install tox:

pip3 install tox

If you are planning to run the full test suite, you also need to install all necessary Python interpreters: 3.5-3.7 and pypy3, along with dev headers to build NumPy and PyCUDA.

On Ubuntu, regular Python interpreters are available with the amazing deadsnakes repo:

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install python3.5 python3.5-dev
sudo apt-get install python3.6 python3.6-dev
sudo apt-get install python3.7 python3.7-dev

Pypy3, however, comes in binaries (make sure you downloaded the latest):

wget -q -P /tmp https://bitbucket.org/pypy/pypy/downloads/pypy3.5-v7.0.0-linux64.tar.bz2
sudo tar -x -C /opt -f /tmp/pypy3.5-v7.0.0-linux64.tar.bz2
rm /tmp/pypy3.5-v7.0.0-linux64.tar.bz2
sudo mv /opt/pypy3.5-v7.0.0-linux64 /opt/pypy3
sudo ln -s /opt/pypy3/bin/pypy3 /usr/local/bin/pypy3

Run tests

To run full tests:

git clone https://github.com/a5kin/xentica.git
cd xentica
tox

Or, if you are using optirun:

optirun tox

For the first time, it would take an amount of time to download/install environments and all its dependencies. Tox will automatically set up all necessary stuff for you, including NumPy and PyCUDA. Subsequent runs should be much quicker, as everything is already set up. In the developer’s environment (Ubuntu 18.04) it takes ~42 sec to finish the full test suite.

If you run tests often, it would also be helpful to get less verbose output. For that, you could execute a strict version of tox:

tox -q

Or if you’d like to skip all uninstalled interpreters:

tox -s

Or even quicker, for immediate test purposes, you could run the default Python3 interpreter tests only with code style and coverage:

tox -q -e flake8,coverage

You could also check the full list of available environments with:

tox -l -v

If you don’t mind, please update us with the metrics under the “Benchmark” section, along with the info about your GPU, environment and Xentica version. It would help us analyze performance and plan for future optimizations.

Note

When planning for a pull request in core Xentica repo, it is a good practice to run a full test suite with tox -q. It will be accepted at least if everything is green =)