Using Plan Executor with TestScripts

The plan_executor library is the part of Crucible that executes tests against HL7 FHIR servers.

Crucible has two varieties of tests: suites and TestScript

Suites

Suites are the built-in tests authored in the Ruby programming language. These tests predate the existence of TestScript and are located in lib/tests/suites.

TestScript

The TestScript is a FHIR resource that defines standard test procedures that a "TestScript Execution Engine" can programmatically execute against a FHIR Server.

The plan_executor is an open-source reference implementation of a TestScript Execution Engine.

The default TestScripts are located in lib/tests/testscripts/scripts.

Loading New TestScripts

Copy into lib/tests/testscripts/scripts/[folder] your TestScript file (as XML) and related fixtures.

Listing Available TestScripts

Assuming a properly functioning plan_executor (see the README), run

bundle exec rake crucible:list_testscripts

This will display a list of TestScripts that were properly loaded and ready to be executed by name using the following instructions.

Executing Single TestScript

To execute a single TestScript, use the execute task. For example:

bundle exec rake crucible:execute[http://fhirtest.uhn.ca/baseDstu3,testscript-example-readtest]

Generating a TestReport

To generate a TestReport for a single TestScript, use the testreport task. For example:

bundle exec rake crucible:testreport[http://fhirtest.uhn.ca/baseDstu3,testscript-example-readtest,reportfile.json]

The last argument (e.g. reportfile.json) is optional and will save the results to the named file.

Executing All TestScripts

To execute all of the TestScripts available to the plan_executor, use the execute_all_testscripts task. For example:

bundle exec rake crucible:execute_all_testscripts[http://fhirtest.uhn.ca/baseDstu3]