Discussion: Protocol Codec Architecture: Logical vs. Physical Separation in N+M IR Conversion

[a-rookie-of-C-language]

I have been following the TiyGate project closely, and our team is currently developing a project with a very similar vision called ferryllm.

It is exciting to see that TiyGate also leverages an Intermediate Representation (IR) approach, successfully reducing the complexity of multi-protocol translation from $N \times M$ to $N + M$. This aligns perfectly with the core design philosophy behind ferryllm.

However, we have taken slightly different paths when it comes to the concrete implementation of the codebase:

• TiyGate uses Logical Separation: A single EndpointCodec (one comprehensive trait) encapsulates the entire inbound and outbound encoding/decoding logic.
• ferryllm uses Physical Separation: It completely decouples entry (Client-to-IR) and adapter (IR-to-Provider) into distinct modules and single-direction traits.

To explore which model is better suited for long-term system evolution, or if there is a sweet spot between the two, I would love to open up a discussion here.

Comprehensive Comparison

Dimension	Physical Separation (ferryllm approach)	Logical Separation (TiyGate current)
File Count	2N (N protocols, independent entry/adapter)	N (N protocols, single highly-cohesive file)
Module Boundaries	Clear (strict separation of ingress/egress concerns)	Blurred (both directions mixed within one type)
Trait Count	2 single-direction traits	1 bidirectional mega-trait
Registration Complexity	Requires registering both entry and adapter	Only requires a single codec registration
Code Reuse	Needs explicit shared utility modules	Natural reuse (shared type context)
Testing Granularity	Fine-grained (unidirectional flows can be isolated)	Coarse (tests must account for bidirectional logic)
Readability	High (lower cognitive load, unidirectional focus)	Lower (files grow significantly as protocols expand)
Maintainability	Better when protocols differ drastically	Better when protocols are highly similar
Extensibility	More flexible (supports mix-and-match or single-direction)	More compact (high protocol suite integrity)
Boilerplate	High (more ceremony and setup code)	Low (minimal overhead)

Discussion & Potential Hybrid Approach

While physical separation provides robust compile-time boundaries and isolated testability, it introduces boilerplate and registration overhead. On the flip side, logical separation keeps things compact but couples inbound and outbound concerns too tightly.

Could there be a third way—a Hybrid Approach?

For example:

1. Physical Separation at the Trait Level: Split the logic into two separate traits—IngressCodec and EgressCodec—to guarantee compile-time boundaries and single-direction testing.
2. Logical Cohesion at the Implementation Level: Allow the same struct (e.g., OpenAiCodec) to implement both traits within a single file. These can then be aggregated under a blanket EndpointCodec marker trait, preserving TiyGate's current elegant single-plugin registration mechanism (via inventory).

I would love to get your thoughts on the following:

1. Has the current logical separation caused any maintenance pain points as protocols become more complex?
2. Do we foresee scenarios where a protocol suite might only need a single direction (e.g., acting strictly as an ingress gateway without bridging to an underlying provider)?
3. How do you view the evolution of this architecture going forward? Looking forward to the discussion!

[jorben]

非常感谢你带来这样一个不一样的视角，也感谢你对 TiyGate 的关注和深入思考。

我非常认可你提出的观点。无论是物理分离、逻辑分离，还是你提到的 Hybrid Approach，本质上都在讨论同一个核心问题：如何让 IR 驱动的协议转换架构在长期演进中保持稳健、可扩展且易维护。这也正是 TiyGate 设计理念中非常重视的方向。

目前 TiyGate 还处在项目创建的早期阶段，未来确实还有很多优化和成长空间。就当下而言，我会更优先关注它是否能够完成从 0 到 1 的可用性建设，包括基础能力是否完整、协议转换是否稳定、网关链路是否可靠，以及它是否能真正解决用户在 AI Gateway 场景中的实际问题。

在更长远的未来，我也会更加关注 TiyGate 是否能持续为用户带来价值，并提供更友好的开发体验和使用体验。

你提出的 codec 架构边界问题，确实是一个值得认真思考的方向，尤其是在协议数量增加、单个协议复杂度上升之后，当前设计是否还能保持清晰和可维护，是需要持续评估的。

再次感谢这次非常有价值的观点碰撞。我会把这个设计思考列入后续计划中，也许在不久的未来，TiyGate 会围绕这个方向做出一些改变。

[a-rookie-of-C-language]

In FerryLLM, I originally intended to build a general AI gateway module to interface with OneAPI, NewAPI, etc. However, after coming across your project, I've decided to pivot our project's scope. I will now integrate your project and reference tiygate-core Perhaps I'll be submitting some PRs to your repository in the near future—who knows?

[jorben]

感谢，这一切都是欢迎。