AWS SDK consumes too much cpu cycles

[MrPresent-Han]

version: 1.11.352
problem: consuming too much cpu usage while downloading

hi, I'm using aws-sdk-cpp to download files, which are mostly 16MB in size
the concurrency of downloading varies between 2~10,
and I found that the cpu usage is very high and a big portion is in the memcopy introduced by AWS::EventLoop

in my code, I set up Download Request as follow:

Aws::S3Crt::Model::GetObjectRequest request;
    request.WithBucket(bucket_name).WithKey(object_name);
    request.SetResponseStreamFactory([buf, size]() {
    // For macOs, pubsetbuf interface not implemented
#ifdef __linux__
        std::unique_ptr<Aws::StringStream> stream(
            Aws::New<Aws::StringStream>(""));
        stream->rdbuf()->pubsetbuf(static_cast<char*>(buf), size);
#else
        std::unique_ptr<Aws::IOStream> stream(Aws::New<AwsResponseStream>(
            "AwsResponseStream", static_cast<char*>(buf), size));
#endif
        return stream.release();
    });
    auto start = std::chrono::system_clock::now();
    auto outcome = crt_client_->GetObject(request);

As I've set up a customized buf area for aws request, why there can be so much memmove stacks

[SergeyRyabinin]

Hello! Thank you for reporting this!
Could you please specify what buffer size are you using?
Thanks!

[MrPresent-Han]

Hi, SergeyRyabinin
So nice to meet you again!
The buffer size I use is always the target file size in the Minio, which is exactly equal to the logical downloaded data size
is this enough or should I extend the size?

[MrPresent-Han]

Hi, SergeyRyabing, this issue might be caused by my incorrect usage.
Can you provide me with a few possible troubleshooting directions?
Maybe there are some configuration items in the S3CrtClientConfig that can greatly influence cpu usage and I do not know

[MrPresent-Han]

Also, another confusing question is that:
when testing serial downloading 500 * 16MB files locally, I found that the s3CrtClient consume more cpu cycles(30%35) than S3Client(2025%)
And the AWS::MemRelease consumes 39.24% cpu usage in the flame graph

I count on CrtClient to reduce cpu usage, but current problems block my progress
Any help will be appreciated, thanks