Add data replication support across multiple nodes

[harrison314]

Currently, ObjeX appears to operate as a single-node or non-replicated system. For production and high-availability use cases, it would be very valuable to support replication of data.

My use case is 3-node cluster setup and there is no solution under a non-permission license that would implement it.

[MeritonAliu]

@harrison314 Thanks for the first issue in my open source scene ever!

Replication was originally planned under a v4 milestone, and we're currently at v1.1.0. Since there's clearly demand for it, I'll be looking into this sooner than originally planned.

Feel free to share more about your setup and requirements in the meantime. Stay tuned!

[harrison314]

I'm working on a system that processes documents, mainly PDFs. And I need a temporary file storage (5kB-200MB per file) that can handle replication so that it can work in HA mode. I'm also looking for other solutions (Redisa is not certified, and other solutions either don't have the appropriate license or are huge juggernauts, such as Ceph).

[MeritonAliu]

Thanks for the details! Replication is a big one, it touches a lot of the architecture (cluster sync, consistency model, conflict handling), so it'll take some time to get right.

One thing I'm exploring is async replication as a first step. Instances would sync objects between each other via S3, eventually consistent. Would that cover your use case, or do you need read-after-write consistency?

[harrison314]

I understand that it won't be quick. So start with what will be easier for you.