Add test for AM

[andreleblanc11]

With more AM stuff being added in the source code, their should be a flow test dedicated to AM.

This test would:

• Test pulling AM data from known sources (or from AM sender perhaps)
• Test the AM renamer. Make sure it does its job correctly.
  • We could confirm this by comparing operational data?
• Test sr3 ingestors of AM data. Validate they are able to download.

[andreleblanc11]

I was able to verify a proof of concept where the AM sender would get files from a watch/sarra pair then send to an AM receiver.

I think it would be best to utilize a similar flow for the flow test.

The flow
watch -> (post locally) -> sarra -> (post locally) -> sender/am_send -> (AM socket) -> flow/amserver -> download locally

We already have a good variability of bulletins under samples/data/20200105/WXO-DD/bulletins/alphanumeric so it would be good to utilize these bulletins for the test.

I prefer having a local flow for the test because:

1. If we pull live data, the flow isn't big enough to cover most use cases. Having a local flow we can encounter every use case if we want (which we likely will).
2. It doesn't make sense to use AMTCP writers on the regionals for flow tests. We would need to run the tests from one machine only and we couldn't automate it through GitHub actions

[petersilva]

Sounds good, and you could even have the local download be the same directory that is watched, so it makes an infinite loop. Then in the log, duplicate suppression should stop it at some point.

[andreleblanc11]

I've been able to get flow_setup.sh and flow_cleanup.sh to setup the flow and cleanup the flow effectively.

Next steps:

• I don't think we need to have a flow_limit.sh in place. The test goes by and finishes in a couple of seconds.
• Have flow_check.sh correctly verify the output of the file transfers, the messages sent/received, and possibly checksums?