Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,16 @@

## [Unreleased]

### 新增
- **内置指标(Meter `StreamFrame`,`System.Diagnostics.Metrics`,零外部依赖)**:帧/字节收发计数(`streamframe.frames_sent/received`、`bytes_sent/received`)、重连次数(`reconnects`)、会话时长(`session_duration` 直方图)、发送队列水位(`send_queue_length` 直方图,入队采样);单一 `endpoint` 标签,订阅方式与仪器清单见 README"内置指标"。netstandard2.0 目标经 `System.Diagnostics.DiagnosticSource` 包获得同款 API(netfx 可用)。
- **浸泡/混沌测试套件(默认不运行)**:`STREAMFRAME_SOAK_SECONDS=<秒> dotnet test --filter FullyQualifiedName~Soak` 手动启用——长时间消息流完整性(顺序/无丢失/无重复)与随机故障混沌(对端 FIN/RST 死亡 + 半帧注入 + 绑定/普通发送突发,终局核对无悬挂、普通消息跨会话续发送达、绑定消息无重复投递)。已在本机通过 20s/60s 两档验证。

### 变更
- **`StxEtxFramer` 切帧向量化**:net8+ 用 `SearchValues` 一次跳到最近的 STX/ETX 候选(不含候选的整段字节直接跳过),实测 ≈1.1 µs/帧 → ≈50 ns/帧(约 22 倍);netstandard2.0 目标保留逐字节实现,语义逐字节等价(两套独立实现,现有 StxEtx/FrameDecoder 测试全绿)。README/bench 性能数据以 2026-08-28 重跑刷新(含内置指标的真实路径)。
- 会话纪元(epoch)改为在 Connected 发布点分配(与会话编号同点),彻底闭合"发布后、任务未创建"窗口内旧纪元幽灵故障卷入新生会话的瞬态误杀(v2.3.0 评审附带发现);发布窗口注入幽灵的回归测试锁定该行为。
- demo 新增场景 5:会话感知收发(编号生命周期、整帧写完才完成、旧会话发送失效不重放、新会话照常收发)。

### 修复
- 暂无

## [2.3.1] - 2026-08-28
Expand Down
26 changes: 24 additions & 2 deletions README.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ public interface ICodec<TMessage>
- **Robustness**: payload decode failures, incomplete-frame overflows/timeouts, send failures, and receive idle timeouts all invalidate the session and rebuild it automatically (no more "connection looks alive while messages silently vanish")
- **Liveness detection (optional)**: TCP KeepAlive and receive idle timeout to catch half-open connections (peer power loss / unplugged cable)
- **Events**: `ConnectionChanged` state changes, `FrameError` frame-level diagnostics, `RawBytesReceived/Sent` raw bytes (HEX debugging)
- **Built-in metrics**: `System.Diagnostics.Metrics` (meter `StreamFrame`) — frame/byte counters, reconnects, session duration, send-queue watermark; see the "Built-in metrics" section
- **Backpressure**: bounded send queue — `SendAsync` waits when full; the receive side buffers without limit by default, or set `ReceiveQueueCapacity` — decoding pauses when the consumer lags, propagating TCP backpressure to the peer and preventing unbounded memory growth
- **Session-aware messaging (optional, advanced)**: `CurrentSessionId` / `SendInSessionAsync` (completes only after the whole frame is written to the socket; fails on session loss and never replays across sessions) / `GetSessionMessages` (messages carry their session id) — for protocols with strict session boundaries such as HSMS; see the dedicated section below

Expand Down Expand Up @@ -168,6 +169,27 @@ A peer that announces a huge frame but never completes it (or a stream with STX

The incomplete-frame timeout only counts while a frame is actually in progress: an idle connection with an empty buffer never trips it (each received byte resets it; it clears once a whole frame is cut). It complements `ReceiveIdleTimeoutMs` (see below) — the latter also times total silence, which suits protocols with periodic traffic; for protocols that may stay idle for a long time but must not tolerate a stalled half-frame (e.g. HSMS T8), use the incomplete-frame timeout. Note its scope is "waiting for further bytes from the network": when `ReceiveQueueCapacity` is set and the consumer stalls completely, the decode loop blocks on the message-channel write and the timeout does not tick during that period (the memory guard `MaxIncompleteFrameBufferBytes` still applies).

### Built-in metrics

Connections ship [System.Diagnostics.Metrics](https://learn.microsoft.com/dotnet/core/diagnostics/metrics-instrumentation) instruments (meter name `StreamFrame`, tag `endpoint`) with zero external dependencies — subscribe with a `MeterListener` or OpenTelemetry in production; unsubscribed, each record costs nanoseconds:

| Instrument | Kind | Meaning |
|---|---|---|
| `streamframe.frames_sent` / `frames_received` | Counter | Business frames sent/received |
| `streamframe.bytes_sent` / `bytes_received` | Counter | Bytes sent/received (framing bytes/noise included) |
| `streamframe.reconnects` | Counter | Reconnect count |
| `streamframe.session_duration` | Histogram | Lifetime of one TCP session (seconds) |
| `streamframe.send_queue_length` | Histogram | Send-queue watermark (sampled per enqueue) |

```csharp
// Minimal subscription (example): OTel's MeterProvider.AddMeter("StreamFrame") hooks the full ecosystem
using var listener = new MeterListener { InstrumentPublished = (i, l) => l.EnableMeasurementEvents(i) };
listener.SetMeasurementEventCallback<long>((instrument, value, _, _) => { /* report */ });
listener.Start();
```

> The netstandard2.0 target gets the same API via the `System.Diagnostics.DiagnosticSource` package (works on .NET Framework).

### Logging (optional)

Pass an `ILogger` when constructing the connection to route internal events (connect retries, session faults, user-callback exceptions) to your logs — no more silence in production:
Expand Down Expand Up @@ -206,8 +228,8 @@ Choosing between the two receive timeouts: `ReceiveIdleTimeoutMs` demands **peri
Measured with BenchmarkDotNet (details and how to reproduce in [bench/README.md](bench/README.md)):

- **Streaming encode**: halves per-frame heap allocation (single vs. double buffer); 25–35% faster for small payloads, on par for large ones;
- **Frame decoding**: `LengthPrefixFramer` ≈10 ns/frame vs `StxEtxFramer` ≈1.1 µs/frame (byte-by-byte scanning) — prefer length-prefix for high-throughput scenarios;
- **End-to-end** (real TCP loopback): one-way throughput ≈246k msgs/s (1KB messages, LengthPrefix), round-trip latency ≈63 µs; the XML codec costs 2–16 µs per message (400B–4KB) — serialization dominates, not framing.
- **Frame decoding**: `LengthPrefixFramer` ≈8.3 ns/frame, `StxEtxFramer` ≈50 ns/frame (≈22× faster after SearchValues vectorization on net8+) — both handle tens of millions of frames per second;
- **End-to-end** (real TCP loopback, re-measured 2026-08-28 with built-in metrics on): one-way throughput ≈160k msgs/s (1KB messages, LengthPrefix ≈6.2 µs/msg), round-trip latency ≈51 µs; the XML codec costs 2–16 µs per message (400B–4KB) — serialization dominates, not framing. Absolute values vary by machine; see [bench/README.md](bench/README.md) to reproduce.

## Supported frameworks

Expand Down
26 changes: 24 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ public interface ICodec<TMessage>
- **健壮性**:帧内容解码失败、未完成帧超限/超时、发送失败、接收空闲超时都会判定会话失效并自动重建(不再产生"连接看似存活、消息静默消失"的假活)
- **活性探测(可选)**:TCP KeepAlive 与接收空闲超时,兜底半开连接(对端断电/拔线)
- **事件**:`ConnectionChanged` 状态变化、`FrameError` 帧层诊断、`RawBytesReceived/Sent` 原始字节(HEX 调试)
- **内置指标**:`System.Diagnostics.Metrics`(Meter `StreamFrame`)——帧/字节收发计数、重连次数、会话时长、发送队列水位,详见下文"内置指标"
- **发送背压**:有界发送队列,队列满时 `SendAsync` 自动等待;接收侧默认无上限缓冲,可用 `ReceiveQueueCapacity` 设上限——消费慢时解码暂停、TCP 背压自然传导到对端,防内存无限增长
- **会话感知收发(可选高级)**:`CurrentSessionId` / `SendInSessionAsync`(整帧写入 socket 才完成、会话失效即失败、绝不跨会话重放)/ `GetSessionMessages`(消息带会话编号)——为有严格会话边界的协议(如 HSMS)准备,详见下文

Expand Down Expand Up @@ -166,6 +167,27 @@ socket 层全量输出(含被丢弃的噪声字节),发送侧按实际写

未完成帧超时只计"帧已开头、迟迟收不齐"的时间:缓冲为空的静默连接不计时(收到字节即重置,整帧切尽后归零)。它与 `ReceiveIdleTimeoutMs`(见下节)互补——后者在完全没流量时也计时,适合有周期报文的协议;对允许长时间空闲、但半帧卡死必须判死的协议(如 HSMS T8),用未完成帧超时。注意其作用域是"等待网络后续字节"期间:若设置了 `ReceiveQueueCapacity` 且消费端完全停滞,解码循环阻塞在消息通道写入上,此期间不计时(内存防线仍由 `MaxIncompleteFrameBufferBytes` 兜底)。

### 内置指标(Metrics)

连接自带 [System.Diagnostics.Metrics](https://learn.microsoft.com/dotnet/core/diagnostics/metrics-instrumentation) 指标(Meter 名 `StreamFrame`,标签 `endpoint`),零外部依赖——生产部署用 `MeterListener` 或 OpenTelemetry 订阅即可观测,不订阅则开销为每次记录纳秒级:

| 指标 | 类型 | 含义 |
|---|---|---|
| `streamframe.frames_sent` / `frames_received` | Counter | 业务帧收发计数 |
| `streamframe.bytes_sent` / `bytes_received` | Counter | 字节收发计数(含帧定界字节/噪声) |
| `streamframe.reconnects` | Counter | 进入重连的次数 |
| `streamframe.session_duration` | Histogram | 单次 TCP 会话存活时长(秒) |
| `streamframe.send_queue_length` | Histogram | 发送队列水位(每次入队采样) |

```csharp
// 最简订阅(示例):OTel 的 MeterProvider.AddMeter("StreamFrame") 一行即可接入完整体系
using var listener = new MeterListener { InstrumentPublished = (i, l) => l.EnableMeasurementEvents(i) };
listener.SetMeasurementEventCallback<long>((instrument, value, _, _) => { /* 上报 */ });
listener.Start();
```

> netstandard2.0 目标经 `System.Diagnostics.DiagnosticSource` 包提供同款 API(netfx 运行时可用)。

### 日志(可选)

构造连接时传入 `ILogger`,内部事件(连接重试、会话故障、用户回调异常等)输出到日志,生产环境不再静默:
Expand Down Expand Up @@ -203,8 +225,8 @@ _ = Task.Run(async () =>
BenchmarkDotNet 实测(详见 [bench/README.md](bench/README.md),可本地复现):

- **流式编码**:每帧堆分配减半(单缓冲 vs 双缓冲),小负载耗时快 25–35%,大负载持平;
- **切帧吞吐**:`LengthPrefixFramer` ≈10 ns/帧,`StxEtxFramer` ≈1.1 µs/帧(逐字节扫描)——高吞吐场景优先长度前缀;
- **端到端**(真实 TCP 回环):单向吞吐 ≈24 万条/秒(1KB 消息,LengthPrefix),往返延迟 ≈63 µs;XML codec 每条报文 2–16 µs(400B–4KB)——序列化开销远大于定界层。
- **切帧吞吐**:`LengthPrefixFramer` ≈8.3 ns/帧,`StxEtxFramer` ≈50 ns/帧(net8+ SearchValues 向量化后较逐字节版提速约 22 倍)——两者均可达每秒千万帧级;
- **端到端**(真实 TCP 回环,2026-08-28 重测,含内置指标):单向吞吐 ≈16 万条/秒(1KB 消息,LengthPrefix ≈6.2 µs/条),往返延迟 ≈51 µs;XML codec 每条报文 2–16 µs(400B–4KB)——序列化开销远大于定界层。绝对值随机器浮动,复现方式见 [bench/README.md](bench/README.md)。

## 支持框架

Expand Down
36 changes: 18 additions & 18 deletions bench/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,24 +18,24 @@ dotnet run -c Release --project bench/StreamFrame.Benchmarks -- --filter "*EndTo
- **Codec 基准**(`CodecBenchmarks`):官方 XML 驱动 `XmlDocumentCodec` 对典型设备报文(~400B 单值上报 / ~4KB 批量明细)的编解码开销。
- **端到端基准**(`EndToEndBenchmarks`):真实 TCP 回环上的完整连接(收发队列 + Pipe + 定界 + codec + socket)——单向吞吐(连发 1 万条 1KB 消息,双 framer)与往返延迟(逐条乒乓 2000 次)。

## 结果
## 结果(2026-08-28 重跑:StxEtx 向量化 + 内置指标已接入;Codec 表为历史数据)

| 方法 | 负载 | 耗时 | 分配 |
|---|---|---:|---:|
| LengthPrefix_Plain | 64B | 36.3 ns | 64 B |
| LengthPrefix_Streaming | 64B | **27.3 ns(-25%)** | **32 B(-50%)** |
| StxEtx_Plain | 64B | 36.7 ns | 32 B |
| StxEtx_Streaming | 64B | **24.4 ns(-34%)** | 32 B |
| LengthPrefix_Plain | 1KB | 57.6 ns | 64 B |
| LengthPrefix_Streaming | 1KB | 67.3 ns(+17%)¹ | **32 B(-50%)** |
| StxEtx_Plain | 1KB | 77.0 ns | 32 B |
| StxEtx_Streaming | 1KB | 82.6 ns(+7%)¹ | 32 B |
| LengthPrefix_Plain | 64KB | 4.11 µs | 64 B |
| LengthPrefix_Streaming | 64KB | 4.11 µs(持平) | **32 B(-50%)** |
| StxEtx_Plain | 64KB | 6.38 µs | 32 B |
| StxEtx_Streaming | 64KB | 6.04 µs(-5%) | 32 B |
| LengthPrefix 切帧 100×1KB | — | **1.0 µs**(≈10 ns/帧) | 0 |
| StxEtx 切帧 100×1KB | — | **≈110 µs**(≈1.1 µs/帧) | 0 |
| LengthPrefix_Plain | 64B | 28.1 ns | 64 B |
| LengthPrefix_Streaming | 64B | **21.4 ns(-24%)** | **32 B(-50%)** |
| StxEtx_Plain | 64B | 28.6 ns | 32 B |
| StxEtx_Streaming | 64B | **19.1 ns(-33%)** | 32 B |
| LengthPrefix_Plain | 1KB | 45.5 ns | 64 B |
| LengthPrefix_Streaming | 1KB | 56.3 ns(+24%)¹ | **32 B(-50%)** |
| StxEtx_Plain | 1KB | 60.9 ns | 32 B |
| StxEtx_Streaming | 1KB | 68.3 ns(+12%)¹ | 32 B |
| LengthPrefix_Plain | 64KB | 3.44 µs | 64 B |
| LengthPrefix_Streaming | 64KB | 3.47 µs(持平) | **32 B(-50%)** |
| StxEtx_Plain | 64KB | 4.51 µs | 32 B |
| StxEtx_Streaming | 64KB | 4.64 µs(+3%) | 32 B |
| LengthPrefix 切帧 100×1KB | — | **0.82 µs**(≈8.3 ns/帧) | 0 |
| StxEtx 切帧 100×1KB | — | **≈5.0 µs**(≈50 ns/帧,向量化后提速约 22 倍) | 0 |

¹ 恰好压在 `EncodeBufferInitialSize`(默认 1024)的缓冲增长边界上:流式路径触发一次扩容租借,
抵消了省下的那次拷贝。负载远大于初始缓冲后两条路径的拷贝量趋同(几何扩容摊销)。
Expand All @@ -53,8 +53,8 @@ dotnet run -c Release --project bench/StreamFrame.Benchmarks -- --filter "*EndTo

| 指标 | LengthPrefix | StxEtx |
|---|---:|---:|
| 单向吞吐(每消息折算) | 4.06 µs ≈ **24.6 万条/秒** | 4.65 µs ≈ 21.5 万条/秒 |
| 往返延迟(乒乓串行) | 62.8 µs | 71.1 µs |
| 单向吞吐(每消息折算) | 6.21 µs ≈ **16.1 万条/秒** | 6.05 µs ≈ 16.5 万条/秒 |
| 往返延迟(乒乓串行) | 50.8 µs | 52.4 µs |
| 每消息堆分配(吞吐模式) | 6.5 KB | 6.9 KB |

测试环境:Windows(本地开发机),.NET 8,BenchmarkDotNet 0.15.8,SimpleJob(warmup 3 / iteration 15;e2e 为 2/10)。
Expand All @@ -65,4 +65,4 @@ dotnet run -c Release --project bench/StreamFrame.Benchmarks -- --filter "*EndTo
- **流式编码的实测收益主要是"每帧堆分配减半"(一个缓冲对象 vs 两个)**,小负载(≤ 百字节)耗时也快 25–35%;
大负载下省掉的那次 memcpy 被缓冲增长摊销,耗时基本持平。"零拷贝"指的是帧路径不再整体搬运负载,不是字面意义的零成本。
- **切帧吞吐相差约两个数量级**:`LengthPrefixFramer` 读长度头直接跳转(≈10 ns/帧);
`StxEtxFramer` 逐字节扫描定位边界(≈1.1 µs/帧)。高吞吐/大帧场景优先长度前缀,STX/ETX 留给既有协议兼容。
`StxEtxFramer` 在 net8+ 用 `SearchValues` 向量化定位边界(≈50 ns/帧;netstandard2.0 目标回退逐字节实现)。长度前缀在超大帧上仍占优,STX/ETX 适合既有协议兼容。
Loading
Loading