How do I test multiple python distributions in one go?

I have the following python3 project structure:

project/
|-- dist1/
|   |-- tox.ini
|   |-- setup.py
|   |-- mynamespace/
|   |   |--package1/
|   |      |-- __init__.py
|   |      |-- ..
|   |-- tests/
|       |-- __init__.py
|       |-- test_package1.py
|
|-- dist2/
|   |-- tox.ini
|   |-- setup.py
|   |-- mynamespace/
|   |   |--package2/
|   |      |-- __init__.py
|   |      |-- ..
|   |-- tests/
|       |-- __init__.py
|       |-- test_package2.py
|
|-- tests/
    |-- __init__.py
    |-- test_dist1_dist2_integration.py

The distributions do not depend on each other but they can communicate with each other through other means, eg Redis cluster. The integration test tests this scenario.

What I would like to accomplish is:

  1. Run all the tests in one go. Do any of the python test frameworks (unit test, nose, nose2, etc..) support this out of the box?
  2. Easily run the integration test from the cmd line.

Some background, the reason for this structure is that each distribution (20+) has its own set of dependencies that need to be tested. Each distribution will be installed in its own unique environment. If I put all the tests on the top level project I cannot test whether all their dependencies are properly declared. If a different, more simple structure, would solve this - happy to adjust it.

What I've tried:

  • python -m unittest - discovers the integration test only and fails because it can't find 'mynamespace'
  • python -m unittest discover -s dist1 - works
  • PyCharm supports automatically adding Source Roots to PYTHONPATH - this makes the integration test work inside PyCharm, but I rather not depend on it though
  • nosetests & nose2 - similar problems. I couldn't find any cmdline options to ease the pain.