Add support for Cargo workspaces

[ian-h-chamberlain]

Currently, it appears that support for Cargo workspaces is limited, due to the way resource files are discovered via relative paths. This is easiest to show with an example:

.
├── Cargo.lock
├── Cargo.toml
└── subcrate/
    ├── Cargo.toml
    ├── data/
    │   ├── a.txt
    │   └── b.txt
    ├── src/
    │   └── lib.rs
    └── tests/
        └── example_test.rs

example_test.rs contents:

extern crate test_generator;

use test_generator::test_resources;

use std::path::Path;

#[test_resources("data/a.txt")]
fn file_exists(resource: &str) {
    assert!(Path::new(resource).exists());
}

This results in a compile-time error:

error: custom attribute panicked
 --> subcrate/tests/example_test.rs:7:1
  |
7 | #[test_resources("data/a.txt")]
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = help: message: no resource matching the pattern data/a.txt

Okay, so it expects a path from the workspace root. Changing the path to #[test_resources("subcrate/data/a.txt")] compiles fine, but now the test fails:

test file_exists_subcrate_data_a_txt ... FAILED

failures:

---- file_exists_subcrate_data_a_txt stdout ----
thread 'file_exists_subcrate_data_a_txt' panicked at 'assertion failed: Path::new(resource).exists()', subcrate/tests/example_test.rs:9:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    file_exists_subcrate_data_a_txt

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

For now, a workaround is to manually change the working directory from within the test:

    let current_dir = std::env::current_dir().unwrap();
    std::env::set_current_dir(current_dir.parent().unwrap()).unwrap();

Edit: it seems this has the wrong effect when run multiple times, the following seems more robust:

    let working_dir = Path::new(env!("CARGO_MANIFEST_DIR")).parent().unwrap();
    std::env::set_current_dir(working_dir).unwrap();

after which the test passes. However, I think this behavior is not self-consistent.

I would suggest either the search path begins from the local crate root rather than the workspace, or the working directory is changed within the test body in the generated code. I suppose this would be a breaking change, unfortunately.

[not-my-profile]

I also just ran into this problem. Just to add some context here:

• A PR to resolve this issue was opened more than a year ago Support cargo workspaces #14 but this repository appears to be unmaintained (see Maintenance status? #13).
• test_case is another crate that provides a proc macro for parametrized tests, however it currently doesn't support parametrization over files (see Generate a test per file in a directory frondeus/test-case#78).

Searching crates.io for parametrized tests glob did however yield the rstest crate, which does actually support both parametrization over files as well as Cargo workspaces.

With test-generator the usage looked like:

#[test_generator::test_resources("res/*/input.txt")]
fn for_each_file(path: &str) { 
   assert!(std::path::Path::new(path).exists()); 
}

With rstest it becomes:

use std::path::PathBuf;

#[rstest::rstest]
fn for_each_file(#[files("res/*/input.txt")] path: PathBuf) {
    assert!(path.exists())
}

[not-my-profile]

(small addendum): I think the modern approach is to just write your own test harness with something like libtest-mimic. This way you don't need any proc macros.