Fix tests to support parallel execution with pytest-xdist

[cuu508]

What do these changes do?

This is a follow-up to #524 where @webknjaz noted:

We'll need however to find places with the hardcoded port — they are most likely to happen on Windows, but things like pytest-xdist would induce these races too.

This PR updates tests (and has some supporting changes in Controller too) to allow tests to complete with pytest-xdist. Hopefully it fixes the flaky tests in Windows too.

The test suite uses a hard-coded port for listening sockets: 8025. When running tests in parallel this is a problem, as tests will race to bind to the same port. One solution would be to hardcode a different port in each test case. This would require test writers to keep track which ports have already been "used", and could create annoying issues when copy-pasting code and forgetting to change the port number. An IMHO nicer alternative is to use a special port "0" which tells asyncio.create_server to pick a random unused port. This is what this PR does.

There were a few complications.

If asyncio.create_server picks the port, there needs some mechanism for the testcase to figure out what port was chosen, so it can connect to it as a client. I changed the InetMixin class to:

• store the hostname and port constructor arguments in the requested_hostname and requested_port instance attributes
• change the hostname and port instance attributes to properties that read the hostname and port from self.server.sockets.

This way, clients and test cases can pass port=0 to the controller constructor, and then read the chosen port from controller.port property and its corresponding hostname from the controller.hostname property. The testing code accesses controller.port in several places, and these parts don't need to change.

---

The other complication was with tests in test_main.py. Here, if we pass -l :0 arguments to main() then it will pick a random port, but there's no convenient way to communicate the port back to the testing code. What I chose to do instead is to generate a random port in the test code and pass it to main using the -l argument. There's a small chance of multiple test cases randomly choosing the same port number, but there are only 3 test cases that need to do this, and each picks from ~16K ports so the probability of collision is tiny.

---

There's one change that is related to making pytest-xdist work, but not related to hardcoded ports. It is in test_smtp.py, test_bad_encodings test case. The test was blowing up because pytest-xdist was trying to serialize logs which contain badly encoded strings. The least intrusive fix I could think of was to, temporarily for this test case, reduce the log level to WARNING, so the bad strings don't end up in logs.

Are there changes in behavior for the user?

Yes: the behavior of Controller.port and Controller.hostname has changed:

• If user specifies port "0", Controller.port will contain the randomly chosen port, not value 0
• After calling controller.end(), both Controller.port and Controller.hostname will return None (because the listening sockets are closed and we cannot consult them any more)
• There are new Controller.requested_port and Controller.requested_hostname instance attributes, containing the values that were passed to the constructor

Related issue number

Checklist

• I think the code is well written
• Unit tests for the changes exist
• tox testenvs have been executed in the following environments:
  • I have not run tox, tut GHA ci-cd workflow passes
• Documentation reflects the changes
• Add a news fragment into the NEWS.rst file

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.86%. Comparing base (dc78ef6) to head (b7e509b).
⚠️ Report is 12 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #593      +/-   ##
==========================================
+ Coverage   97.75%   97.86%   +0.10%     
==========================================
  Files          23       23              
  Lines        5709     5748      +39     
  Branches      351      353       +2     
==========================================
+ Hits         5581     5625      +44     
+ Misses         79       73       -6     
- Partials       49       50       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[webknjaz]

We may want to mention the keyword “ephemeral port”, which is what the standards are calling it.

[webknjaz]

@cuu508 missed this?

Suggested change

	# port 0 means "pick a random unused port"
	# port 0 means "pick a random unused port"
	# it is also known as an ephemeral port

[webknjaz]

Is there any way to make use of an ephemeral port here too? What are the challenges?

[cuu508]

The test case runs aiosmtpd.main.main() in a separate process (using multiprocessing). We can pass port=0 to it, and it will run on an ephemeral port, but there is no convenient way to communicate the chosen port back to the test case.

[webknjaz]

Alright, let's postpone this then.

[webknjaz]

@cuu508 looks like this also needs a rebase with conflict resolution.