Skip to content

Extension_installation_and_testing.md

Latest commit

 

History

History
81 lines (64 loc) · 3.9 KB

Extension_installation_and_testing.md

File metadata and controls

81 lines (64 loc) · 3.9 KB
Raw

Development and testing

Local Installation

To run CKAN and develop and/or test extensions locally, one needs to set up development environment as the following:

  1. Set up virtual environment:
sudo mkdir -p /etc/ckan/default/  # or other directory of choice
sudo chown `whoami` /etc/ckan/default
python3 -m venv /etc/ckan/default
source /etc/ckan/default/bin/activate
pip install setuptools
pip install --upgrade pip

Note, virtual enviromnent should stay activated during the whole installation process.

  1. Install CKAN as a package into virtual environment:
pip install -e 'git+https://github.com/ckan/ckan.git@ckan-2.10.1#egg=ckan[requirements]'
# or for development purposes:
pip install -e 'git+https://github.com/ckan/ckan.git@ckan-2.10.1#egg=ckan[requirements,dev]'

Installation of dependencies may fail due to some known or other compatibility issues, below is an example of troubleshooting.

One of the known issues is psycopg2 building. If installation of psycopg2 fails, go to CKAN requirements.txt now cloned and located at <venv directory>/src/ckan/requirements.txt, change psycopg2==2.9.7 => psycopg2-binary==2.9.9, rerun installation of the dependencies and install CKAN itself separately:

pip install -r <venv directory>/src/ckan/requirements.txt
pip install -r <venv directory>/src/ckan/dev-requirements.txt
pip install -e 'git+https://github.com/ckan/ckan.git@ckan-2.10.1#egg=ckan'

Another case of dependencies incompatibility occurring for CKAN v2.9.10 is PyYAML. In requirements.txt it may be set as pyyaml==5.4.1 or pyyaml==6.0.1, if it causes the following error in the runtime TypeError: load() missing 1 required positional argument: ‘Loader', it should be downgraded to pyyaml==5.3.1.

  1. Install required extensions

Each extension should be installed as a package as described in README for regular installation or development installation. Alternatively an extension can be cloned first to a desired location and then installed to the virtual environment from file, e.g.:

pip install -e file:///<path to local extension repo>/ckan-fairdatapoint#egg=ckanext-fairdatapoint

Then install dependencies:

pip install -r <path to local extension repo>/requirements.txt
pip install -r <path to local extension repo>/dev-requirements.txt

Testing

Testing strategy for a plugin would depend on plugin functionality. CKAN provides a number of helper functions allowing to generate dummy data in the database or clean the database afterwards. Please review official documentation for detailed info.

To set up testing of a plugin make sure the following settings are configured correctly:

  1. Make sure pytest-ckan is installed to the virtual environment. Normally this dependency should be listed in dev-requirements.txt of a plugin and therefore got installed. If you do not see it in pip list output run pip install pytest-ckan.
  2. Each plugin has an autogenerated test.ini file. Make sure it points to test-core.ini file of CKAN installation, e.g.:
[app:main]
use = config:/etc/ckan/default/src/ckan/test-core.ini
  1. Modify test-core.ini so configuration is correct. Most likely you need to modify sqlalchemy.url parameter so it contains correct connection information: sqlalchemy.url = postgresql://<user>:<password>@localhost/<db_name> If testing of your plugin requires writing to the database it is recommended to set up a separate test instance of postgres database.

Running tests

Tests can be run via simple pytest --ckan-ini=test.ini command or an extended one with desired options. The following example is to run tests with coverage:

pytest --ckan-ini=test.ini --disable-warnings ./ckanext/fairdatapoint --cov ./ckanext/fairdatapoint