Add support for disabling and restoring access to private projects on AWS

[Chrystinne]

Part of #2348

Depends on #2627 (#2627 should be merged before this PR)

This PR adds support for temporarily disabling and restoring access to private projects on AWS S3 through the console, without deleting the project files.

A "Disable access" button deletes the S3 access points for the project:

A "Restore access" button recreates them:

Buttons are mutually exclusive and only visible for private projects when sent_files is True.

Note: for public projects, per-project access cannot be disabled without deleting the files, since the bucket policy applies to the entire bucket.

[bemoody]

It looks like this merely hides the button from the published project page. It doesn't prevent people from submitting the enable_aws_access form or joining by signing DUA (unless those actions are somehow prevented by having zero access points - if that's the case and that's desirable, that needs to be documented and tested.)

Overall I'm not a fan of this approach. I think that "access disabled" should be represented by an explicit flag in the database, not by having zero access points.

[Chrystinne]

It looks like this merely hides the button from the published project page. It doesn't prevent people from submitting the enable_aws_access form or joining by signing DUA (unless those actions are somehow prevented by having zero access points - if that's the case and that's desirable, that needs to be documented and tested.)

Overall I'm not a fan of this approach. I think that "access disabled" should be represented by an explicit flag in the database, not by having zero access points.

Thanks, @bemoody. That's a good catch.

Just to clarify what happens at the S3-level: deleting all access points for a private project does effectively block access to the files. The private bucket policy (DelegateToAccessPoints) only allows access through access points, so
without them, there is no way to reach the files directly on S3.

The issue you raised at the PhysioNet level is correct, though. I agree that would be a better approach to add an explicit access_disabled flag to the AWS model. We should check that flag before adding users to access points after they sign the DUA, and use it to control the display of the enable_aws_access button.

[Chrystinne]

@bemoody After some back and forth with this, I've decided to go for the addition of the "access disabled" flag you suggested, and I think the result is good. It didn't overcomplicate the code, and I believe it's a good idea to decouple the AWS utility functions anyway for some reasons: the deprecation functionality is not quite ready to work with the undoing operation, and we still don't have a straightforward way to check whether files have been removed or not when deprecating.

Next, we can do a bit of improvement on the deprecation as well so we can integrate these operations more smoothly.

Code is now ready to review. Could you please have another look?

[bemoody]

This looks like it should work (though should check again after merging #2627, which this depends on.)

If access is disabled then the "configure your AWS credentials" link should probably also be hidden. It might be better to add project.aws.access_disabled to the outer conditional in published_project.html:

{% if has_s3_credentials and project.aws.sent_files and project.aws.access_disabled %}

(but keep in mind that in the case of open projects, the access_disabled flag still won't actually do anything.)

[Chrystinne]

Chrystinne

Hiding the aws configuration makes sense. Thanks, @bemoody

[Chrystinne]

@bemoody I made the changes and resolved the conflicts with dev. Could you please take another look?

[tompollard]

@Chrystinne it looks like this is ready to merge? If so, please could you rebase, and delete and regenerate migrations?