Need more exhaustive check to determine if an HSTS header will be ignored

[jsf9k]

🐛 Bug Report

Consider the function hsts_check() in pshtt/pshtt.py. It's not just a bad hostname that should cause the HSTS header to be ignored. It is also ignored if the certificate is untrusted, for example. (See item 2 in the RFC.) Therefore I think that this if clause should be fleshed out a little more.

We should be a little careful when making this change, since as @echudow pointed out in a separate pull request:

Currently we don't check that the certificate is trusted even on the initial response, so that change could have more negative effects on the results.

[mcdonnnj]

Should this check just get blanketed under endpoint.https_valid? That would check endpoint.https_expired_cert, endpoint.https_self_signed_cert, endpoint.https_bad_chain, and endpoint.https_bad_hostname. My only concern is that there are allowances for self-signed certs in section 11, but I'm not sure if we currently encounter those environments. Since we already fail valid HTTPS based on a self-signed cert it doesn't seem to conflict much.

[echudow]

@konklone said in a different thread:

We can't know whether or not a publicly trusted chain is appropriate across the government ... we can't be in the position of flagging non-public certs when they're actually legitimate.

It is entirely possible that some sites will use non-public certs with HSTS and expect all internal clients to trust the non-public certs, so it is ok for them. If all the sites that are scanned will always be public-facing sites that the public should be accessing, then they should use publicly trusted certs, but I don't know that we can make that assumption. In that case, we probably can't not give credit for HSTS based on it being a self-signed cert or having a bad chain. In addition to the current check for a bad hostname, we could probably also not give HSTS credit if there is an expired certificate though.