Failures in the collection converted tests

[nhosoi]

Collections conversion tool for linux-system-roles converts the linux-system-roles tree structure to the collection tree structure and replaces the role value with FQCN in the yaml files as well as in the modules/module_utils.

This is just a heads-up; there are 2 cases the conversion tool cannot support.

1. direct access expecting the roles tree structure

tests/tests_unit.yml
- hosts: all
  name: execute python unit tests
  tasks:
    - name: Copy python modules
      copy:
        src: "{{ item }}"
        dest: /tmp/test-unit-1/
        local_follow: false
      loop:
        - ../library/network_connections.py
        - unit/test_network_connections.py
        - ../module_utils/network_lsr

2. requires git repository

tests/tests_integration_pytest.yml
- name: Run Pytest tests
  hosts: all
  vars:
    - rundir: /run/system-roles-test
  tasks:
    - file:
        state: directory
        path: "{{ rundir }}"
        recurse: true

    - command: git rev-parse --show-toplevel
      register: git_top_directory
      delegate_to: localhost

[tyll]

Collections conversion tool for linux-system-roles converts the linux-system-roles tree structure to the collection tree structure and replaces the role value with FQCN in the yaml files as well as in the modules/module_utils.

This is just a heads-up; there are 2 cases the conversion tool cannot support.

1. direct access expecting the roles tree structure
2. requires git repository

What's the next step here? Do you have suggestions how to address this? Basically both cases need the same/similar solution. In both cases files from the role need to be copied on the system running the tests to be able to run the unit or integration tests. For this both all the necessary test definitions and the python code used by the module in the role need to specified. How are other roles running unit tests? Is the network role the only one with unit tests?
Regarding the tree structure, I suppose the tests can be adjusted to detect whether it is a role or collection structure and import files respectively.

[richm]

What's the next step here? Do you have suggestions how to address this?

We don't have any suggestions yet. We noticed this in collections testing and didn't want to forget about it.

How are other roles running unit tests? Is the network role the only one with unit tests?

No, other roles have unit tests. However, network is the only role that wants to run unit tests as part of integration testing.

[richm]

Regarding the tree structure, I suppose the tests can be adjusted to detect whether it is a role or collection structure and import files respectively.

Yes, something like that. I'm sure the are some sort of built-in variables or env. vars. (or even path structure) that can be used to determine if the test is being run against the "old-style" role or the role as a member of a collection.

[tyll]

Regarding the tree structure, I suppose the tests can be adjusted to detect whether it is a role or collection structure and import files respectively.

Yes, something like that. I'm sure the are some sort of built-in variables or env. vars. (or even path structure) that can be used to determine if the test is being run against the "old-style" role or the role as a member of a collection.

Path structure should be easy:

if ../library/network_connections.py exists:
	# assume it is a role
elif PATH_FOR_COLLECTION exists:
    # assume it is a collection
else
    # something is wrong

[nhosoi]

FYI, the converted network collections format looks like this.

ansible_collections/
└── fedora 
    └── system_roles
        ├── plugins
        │   ├── modules
        │   │   ├── .....
        │   │   ├── network_connections.py
        │   │   └── .....
        │   └── module_utils
        │       ├── .....
        │       ├── network_lsr
        │       │   ├── argument_validator.py
        │       │   ├── ethtool.py
        │       │   ├── __init__.py
        │       │   ├── nm_provider.py
        │       │   └── utils.py
        │       └── .....
        ├── roles
        │   ├── .....
        │   ├── network
        │   │   ├── .....
        │   │   └── tasks
        │   │       └── main.yml
        │   └── .....
        ├── tests
        │   ├── .....
        │   ├── network
        │   │   ├── .....
        │   │   └── unit
        │   │       ├── test_network_connections.py
        │   │       └── test_nm_provider.py
        │   └── .....
        └── tox.ini

[tyll]

How are other roles running unit tests? Is the network role the only one with unit tests?

No, other roles have unit tests. However, network is the only role that wants to run unit tests as part of integration testing.

Thanks. This made me realize that the affected tests probably do not need to run for collections as long as they already run in a role context for the platforms that would be used by the collection.

Since the tests directly test the python code, moving the files around should not affect the tests. Not sure, when/how the tests will run in connection with the migration to collections. Will ELN/CentOS Stream use collections? Will only the downstream CI run the tests after conversion or will this be part of the upstream CI?

[richm]

Since the tests directly test the python code, moving the files around should not affect the tests.

The conversion to collections rewrites the python code - specifically, the import paths for the network module_utils. Presumably, the unit tests will be testing individual methods/classes in the network module and module_utils files, so it doesn't matter what the module import paths are, because that isn't the subject under test of the unit test. The ability to lookup and import the module using the correct path at runtime is done as part of the integration test (albeit implicitly - that is, there isn't an explicit verification that the network_connections module was found and loaded by Ansible, either it works or it doesn't).

Not sure, when/how the tests will run in connection with the migration to collections.

We are planning to have CI that will convert the role into a collection then run the integration tests against the collection.

Will ELN/CentOS Stream use collections?

Not sure what you mean. What is ELN? How do ELN/CentOS Stream currently use the network role?

Will only the downstream CI run the tests after conversion or will this be part of the upstream CI?

Both

[tyll]

Will ELN/CentOS Stream use collections?

Not sure what you mean. What is ELN? How do ELN/CentOS Stream currently use the network role?

ELN is the project name for developing future RHEL releases as part of Fedora:
https://docs.fedoraproject.org/en-US/eln/

I guess I could also ask, will Fedora Rawhide ship a collection instead of a role in the linux-system-roles package? Will it use the upstream tests to test the collection in their CI?