Authentication fails on get requests

[nkeating-mutualofenumclaw]

When trying to perform elastic search queries with the latest version 9.0.0, the IAM authentication fails with the message "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."

After troubleshooting, I found that if I used version 8.1.3 the authentication with IAM works successfully. Somewhere between 8.1.3 and 9.0.0 the authentication broke for IAM

[nkeating-mutualofenumclaw]

The problem appears to be in
AmazonConnection.js line 17: req.headers.host = req.hostname

It appears to work if changed to: req.headers.Host = req.hostname

[jpadhye]

Encountered same issue.

    "name": "ResponseError",
    "meta": {
        "body": {
            "message": "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details."
        },
        "statusCode": 403,
        "headers": {
            "date": "Tue, 07 Jul 2020 18:55:13 GMT",
            "content-type": "application/json",
            "content-length": "192",
            "connection": "keep-alive",
            "x-amzn-requestid": "a6581234-36d9-4a57-b81d-ea2db63f0350",
            "access-control-allow-origin": "*"
        },

[compwright]

@nkeating-mutualofenumclaw @elastic/elasticsearch changed how headers work internally in v7.7.1. Which version are you using? Does v9 work if you upgrade to the latest version of @elastic/elasticsearch?

[compwright]

Also FYI this dependency is noted in package.json:

"peerDependencies": {
    "@elastic/elasticsearch": ">=7.8.0",
    "aws-sdk": "^2.709.0"
}

So you should get a warning from NPM if you're running @elastic/elasticsearch < 7.8.

[fimbulvetr]

I am getting the same error. We are running

  "dependencies": {
    "@elastic/elasticsearch": "^6.8.7",
    "aws-elasticsearch-connector": "^8.2.0"
  }

As we are running AWS ES @ 6.8, the documentation for the elasticsearch client says we should run 6.8 client version if we are using 6.8 server version, so we cannot upgrade to client version 7.8 ( See https://github.com/elastic/elasticsearch-js#compatibility ); Therefore, the solution of using the latest @elastic/elasticsearch is not an option for us.

Thanks

[dobrynin]

@compwright perhaps you should deprecate v8.3.0 as it was a breaking change.

[jimmone]

I am getting authorization error aswell even when using AdministratorAccess policy: User: <arn> is not authorized to perform: es:ESHttpPost

I am using the following versions:

"@elastic/elasticsearch": "^7.9.1"
"aws-elasticsearch-connector": "^9.0.0"
"aws-sdk": "^2.773.0"