If the goal is (as I think it should be) to (also) have minimal Schematron files to exercise the testable statements in the ISO standard, how should the tests be named/test directories be structured?
Naming tests after clause number -- e.g., 5.15.11-1.sch, 5.5.11-2.sch -- is potentially fragile since clause numbers have changed between ISO versions (as @AndrewSales points out). It's also opaque. (Who, other than @AndrewSales, knows that Clause 5.5.11 is the role attribute?)
Should the tests be named after what they test? E.g., role-1.sch, role-2.sch?
A future ISO version could invalidate an existing test (once such tests exist). However, putting 2016 in every test name as it's written seems like just adding noise.
I suggest:
- Put all tests in an
iso-schematron directory.
- Name tests after what they test, e.g.,
role-01.sch
- Use two digits when numbering tests so that they sort well on operating systems that don't pretend to be able to recognise files with a numeric sequence. (Can we realistically ever expect more than 99 tests for a single concept?)
- If/when a new ISO version invalidates an existing test, make
iso-schematron-2016 and, say, iso-schematron-2018 directories then move the existing test to iso-schematron-2016 and make a new test in iso-schematron-2018. That way, an implementation can say that it passes all of the common and version-specific tests rather than never being able to pass 100% of tests if they are all lumped together.
- Maybe the structure should go straight to
iso-schematron/common, iso-schematron/2011, and iso-schematron/2016 directories instead?
- Test names should be unique across all test directories so there's no problems with test frameworks that flatten directories. E.g., if
role-01.sch is moved to iso-schematron-2016 (or similar), the 2018(?)-specific test would be named role-02.sch rather than making another role-01.sch test.
Comments welcome but, realistically, the first person making an organised effort at producing tests is going to have the biggest effect on the test naming convention.
If the goal is (as I think it should be) to (also) have minimal Schematron files to exercise the testable statements in the ISO standard, how should the tests be named/test directories be structured?
Naming tests after clause number -- e.g.,
5.15.11-1.sch,5.5.11-2.sch-- is potentially fragile since clause numbers have changed between ISO versions (as @AndrewSales points out). It's also opaque. (Who, other than @AndrewSales, knows that Clause 5.5.11 is theroleattribute?)Should the tests be named after what they test? E.g.,
role-1.sch,role-2.sch?A future ISO version could invalidate an existing test (once such tests exist). However, putting
2016in every test name as it's written seems like just adding noise.I suggest:
iso-schematrondirectory.role-01.schiso-schematron-2016and, say,iso-schematron-2018directories then move the existing test toiso-schematron-2016and make a new test iniso-schematron-2018. That way, an implementation can say that it passes all of the common and version-specific tests rather than never being able to pass 100% of tests if they are all lumped together.iso-schematron/common,iso-schematron/2011, andiso-schematron/2016directories instead?role-01.schis moved toiso-schematron-2016(or similar), the 2018(?)-specific test would be namedrole-02.schrather than making anotherrole-01.schtest.Comments welcome but, realistically, the first person making an organised effort at producing tests is going to have the biggest effect on the test naming convention.