out_azure_blob: added buffering to azure blob

[tanmaya-panda1]

This pull request introduces significant enhancements to the Azure Blob plugin, focusing on adding buffering capabilities and improving URI handling.

Buffering Enhancements

• Added new configuration options and fields to support buffering, such as buffering_enabled, buffer_dir, store_dir_limit_size, and current_buffer_size in the flb_azure_blob struct. These changes enable the plugin to buffer data before uploading to Azure Blob Storage.
• Introduced validation to ensure buffering is not used with the appendblob blob type, as it is incompatible. An error is logged if this configuration is detected.

URI Handling Improvements

• Updated the azb_block_blob_uri and azb_block_blob_uri_commit functions to include a new random_str parameter for generating more unique URIs for block blobs. This ensures better handling of concurrent uploads.

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• Run local packaging test showing all targets (including any new ones) build.
• Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• Documentation required for this feature

Backporting

• Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[tanmaya-panda1]

docs PR # fluent/fluent-bit-docs#1632

[uristernik]

We have been running this version in many clusters for the past 24 hours and it's working great

[edsiper]

hi @tanmaya-panda1

the number of commits needs to be reduced and just grouped to what is needed, keeping a clean history is very important for long term maintenance:

you can use the guidelines as a reference: https://github.com/fluent/fluent-bit/blob/master/CONTRIBUTING.md#commit-changes

[tanmaya-panda1]

@edsiper as per your comments, cleaned up the commit history.

[edsiper]

thank you!