fix(init): replace GetBucketLocation with HeadBucket for cross-account S3 support

[bensi94]

Description

parseS3ARNAndGetBucketInfo uses s3:GetBucketLocation to discover the region of an S3 bucket during the Fluent Bit init process. This API requires the caller to be the bucket owner, which causes AccessDenied errors in cross-account scenarios (e.g., an ECS task pulling Fluent Bit config from an S3 bucket owned by a different AWS account).

This PR adds HeadBucket as the primary region discovery method with a fallback to GetBucketLocation for backward compatibility. This avoids breaking users who only have s3:GetBucketLocation in their IAM policies.

Behavior

1. Try HeadBucket first — works cross-account, returns canonical region strings directly
2. If HeadBucket fails (e.g., missing s3:ListBucket permission), fall back to GetBucketLocation with the existing LocationConstraint mapping (empty → us-east-1, "EU" → eu-west-1)

Changes

• Add HeadBucket to the S3Client interface alongside existing GetBucketLocation
• Update parseS3ARNAndGetBucketInfo to try HeadBucket first, fall back to GetBucketLocation
• Remove the TODO comment about migrating to HeadBucket
• Update MockS3Client to support both methods with configurable HeadBucketError for testing the fallback path
• Add TestParseS3ARNAndGetBucketInfo_Fallback covering the GetBucketLocation fallback path
• Add tests for ARN parsing, region retrieval across standard regions, and cross-account scenarios

Testing

All existing tests pass alongside the new tests. No regressions.

References

• HeadBucket API docs
• GetBucketLocation API docs

[ShelbyZ]

The hard part here is that if we immediately deprecate s3:GetBucketLocation we may break users that do not have the correct IAM permissions set. We likely need to make this optional or as a fallback.

[bensi94]

The hard part here is that if we immediately deprecate s3:GetBucketLocation we may break users that do not have the correct IAM permissions set. We likely need to make this optional or as a fallback.

@ShelbyZ Yes that's a very good point. I've added a fallback now. Is that suitable?

[ShelbyZ]

@bensi94 - had some time to test out both APIs via CLI and there is no strong requirement to use s3:HeadBucket when dealing with cross-account buckets. The bucket needs to enable calling s3:GetBucketLocation (and likely upload operations) to allow cross-account access:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PolicyForAllowUploadWithACL",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::$MY_ACCOUNT:root"
            },
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::shelbyz-test-bucket12345"
        }
    ]
}

head-bucket

aws s3api head-bucket --bucket shelbyz-test-bucket12345 --region us-west-2 --debug
{
    "BucketRegion": "us-west-2",
    "AccessPointAlias": false
}

get-bucket-location

aws s3api get-bucket-location --bucket shelbyz-test-bucket12345 --region us-west-2 --debug
{
    "LocationConstraint": "us-west-2"
}

Let me know if there is something more to the current setup that is preventing the cross-account access

[bensi94]

Thank you @ShelbyZ, that is correct if they are in the same region.

But if they're in different regions GetBucketLocation will not work.

See this one from the AWS Docs:

In a bucket's home Region, calls to the GetBucketLocation operation are governed by the bucket's policy. In other Regions, the bucket policy doesn't apply, which means that cross-account access won't be authorized.