diff --git a/CHANGELOG.md b/CHANGELOG.md index ddca178..e797849 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,15 @@ ### 新增 +- 增加独立 SecsFrame.Trace 程序集和版本化 + SecsFrame-Trace/1 信封，为已解码数据消息提供确定性导出与 + 严格读取，并保留 UTC 时间、方向和可选 HSMS 诊断标识。 +- 增加按 S/F 与 Item List 路径执行的结构化替换脱敏；路径漂移和重叠规则 + 明确失败，避免敏感值因字符串替换或 Schema 变化而静默泄漏。 +- 增加显式 allowlist 的受控重放；只重放 Sent 记录，忽略旧 Session ID、 + System Bytes、时间和回复令牌，通过公共发送 API 创建新事务。 +- 增加 Trace 黄金向量、畸形信封、资源限制、脱敏泄漏、预验证和真实 TCP + 新事务测试。 - 增加独立 SecsFrame.Sml 程序集，为动态消息和全部 Item 类型 提供确定性 SML 调试文本写出与严格读取。 - 增加 ASCII 可逆转义、JIS-8 原始十六进制字节、InvariantCulture 数字、 diff --git a/README.md b/README.md index 3f0c492..926e42f 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,7 @@ | `SecsFrame` | SECS-II 数据模型、HSMS-SS 会话与事务基础 | | `SecsFrame.Gem` | GEM 通用状态模型与能力服务 | | `SecsFrame.Sml` | 动态消息的确定性 SML 调试读写 | +| `SecsFrame.Trace` | 已解码消息的导出、结构化脱敏与受控重放 | | `SecsFrame.Smn` | E173 SMN 日志、文档与消息表示 | | `SecsFrame.Sedd` | E172 SEDD 设备接口数据字典 | @@ -40,6 +41,9 @@ Equipment 应用发送策略的 Collection Event、报警目录查询、单报 独立 SecsFrame.Sml 为相同动态消息模型提供确定性文本写出、 严格读取、全部 Item 类型往返和显式资源边界；它是非规范性调试 profile， 不进入线上通讯路径。 +SecsFrame.Trace 在该文本层上提供版本化的数据消息 Trace 信封、 +严格的 Item 路径脱敏和显式 allowlist 重放；它不会接管连接事件消费者， +也不会复用原事务标识。 Item 用法与边界见 [docs/SECS-II-ITEMS.md](docs/SECS-II-ITEMS.md)，消息集成见 [docs/SECS-MESSAGES.md](docs/SECS-MESSAGES.md)，传输适配边界见 @@ -59,6 +63,8 @@ Host/Equipment 端点见 [docs/GEM-FOUNDATION.md](docs/GEM-FOUNDATION.md)， SML 调试文本见 [docs/SML.md](docs/SML.md)， +Trace 工具边界见 +[docs/TRACE.md](docs/TRACE.md)， 官方 secs4net 跨实现证据见 [docs/SECS4NET-INTEROP.md](docs/SECS4NET-INTEROP.md)， 整体路线图见 diff --git a/SecsFrame.slnx b/SecsFrame.slnx index 0e90820..273bb9e 100644 --- a/SecsFrame.slnx +++ b/SecsFrame.slnx @@ -2,6 +2,7 @@ + diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 40e1cbd..4f26f72 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -13,6 +13,8 @@ StreamFrame TCP、Pipe、帧边界、连接生命周期 E172 SEDD 与 E173 SMN 是可选元数据/表示层，不进入核心通讯路径。 SecsFrame.Sml 同样是只依赖核心动态消息模型的可选调试表示层； 核心包不反向依赖它。 +SecsFrame.Trace 依赖 SML 表示层和核心公共发送 API，只处理已经 +解码的数据消息；它不订阅内部 transport 或创建第二个公共事件消费者。 ## 核心原则 @@ -160,5 +162,7 @@ HSMS 头 Session ID 与 System Bytes。没有 W-Bit 的出站消息在写出后 - `SecsFrame` 可以依赖 `StreamFrame`。 - `SecsFrame.Gem` 依赖 `SecsFrame`。 - `SecsFrame.Sml` 依赖 `SecsFrame`，不得依赖 GEM 或线上传输实现。 +- `SecsFrame.Trace` 依赖 `SecsFrame.Sml` 与 `SecsFrame`，重放必须回到 + 公共发送 API，不得直接构造或写出保留旧事务标识的线上帧。 - 核心包不得依赖 GEM、SML/SMN、依赖注入容器或具体日志实现。 - 状态机计时使用可替换的时间抽象，测试不得依赖真实长时间等待。 diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index feba74b..af9d459 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -95,15 +95,21 @@ Collection Event 发送策略沿用相同的默认不注册模式；拒绝发生 E30-0526 材料。下一可执行优先级已转入阶段 3；SML 调试读写已经基于许可 清晰的公开实现完成非规范性 profile，为现有动态 SecsMessage 和全部 SecsItem 类型提供确定性文本写出、严格读取、资源边界与 -自主协议向量，不宣称标准 SML 合规。下一切片在该稳定文本表示上增加 Trace -导出、结构化脱敏与受控重放。 +自主协议向量，不宣称标准 SML 合规。已在该稳定文本表示上增加已解码数据 +消息的版本化 Trace 导出、结构化脱敏与显式 allowlist 受控重放；重放通过 +公共发送 API 创建新事务，不复用旧会话或 System Bytes。 ## 阶段 3：元数据与工具 - [x] SML 调试读写。 - E173 SMN 日志与消息文档。 - E172 SEDD 设备字典导入、校验与发现。 -- Trace 重放、脱敏和互操作诊断工具。 +- [x] 已解码数据消息的 Trace 导出、结构化脱敏和受控重放基线。 +- 控制消息、结构化诊断、故障原始样本和重放时序等互操作诊断扩展。 + +当前 Trace 基线不接管连接事件流，也不把原始标识写回线上。后续控制与故障 +诊断扩展必须先定义原始字节的数据分级、默认不导出策略和可验证脱敏边界。 +E173 SMN 与 E172 SEDD 仍需先核对授权标准及允许分发的 Schema/表示形式。 ## 发布门槛 diff --git a/docs/TRACE.md b/docs/TRACE.md new file mode 100644 index 0000000..f8bc555 --- /dev/null +++ b/docs/TRACE.md @@ -0,0 +1,114 @@ +# Trace 导出、脱敏与重放 + +## 范围 + +SecsFrame.Trace 为已经解码的 SECS-II Data Message 提供： + +- 确定性的 SecsFrame-Trace/1 文本导出和严格读取； +- 按 Stream、Function 与 Item List 路径执行的结构化替换脱敏； +- 只选择显式允许的本地发送记录，并通过正常公共发送 API 串行重放。 + +当前切片不是原始网络抓包器，不记录 TCP 字节、transport Session generation、 +控制帧或异常对象，也不会接管 HsmsConnection 的单消费者事件流。 +应用应在自己的事件循环中从 HsmsIncomingDataMessage 创建接收记录， +并在调用发送 API 前显式创建发送记录。 + +## 记录与导出 + +SecsTraceRecord 保存 UTC 时间、本地方向、动态 +SecsMessage，以及可选的协议 Session ID 和 System Bytes。 +发送前通常尚不知道新事务的 System Bytes，因此这些 HSMS 字段允许缺省： + +~~~csharp +using SecsFrame.Trace; + +var sent = SecsTraceRecord.CreateSent( + DateTimeOffset.UtcNow, + primary, + sessionId: connection.Options.SessionId); + +var received = SecsTraceRecord.CreateReceived( + DateTimeOffset.UtcNow, + connectionEvent.IncomingMessage!); + +var codec = new SecsTraceCodec(); +var text = codec.Encode(new[] { sent, received }); +var records = codec.Decode(text); +~~~ + +文件信封固定使用 LF，并在每条记录前写出随后 SML 块的字符长度： + +~~~text +SecsFrame-Trace/1 +Record 2026-08-29T05:00:00.0000000Z Sent 10 - 10 +'S1F1'W +. +~~~ + +字符长度使 ASCII 转义、句点和换行不会被误识别成记录边界。Trace codec +限制总字符数与记录数；每个消息继续受 SmlMessageCodec 的深度、 +Item、值和文本长度限制。解析错误使用包含记录索引与字符偏移的 +SecsTraceParseException。 + +该格式是 SecsFrame 的版本化诊断信封，不是 SEMI 标准、PCAP 或原始 HSMS +wire trace。协议标识只用于问题关联，不能作为新连接上的重放标识。 + +## 结构化脱敏 + +脱敏发生在不可变 Item 树上，而不是对 SML 文本做字符串替换： + +~~~csharp +var redactor = new SecsTraceRedactor(new[] +{ + new SecsTraceRedactionRule( + stream: 6, + function: 11, + itemPath: new[] { 0 }, + replacement: SecsItem.Ascii("REDACTED")), +}); + +var safeRecord = redactor.Redact(received); +var safeText = codec.Encode(new[] { safeRecord }); +~~~ + +空路径选择根 Item，后续整数逐层选择 List 子项。匹配消息缺少根 Item、路径 +进入非 List 或索引越界时立即失败，避免消息 Schema 漂移后静默漏脱敏。 +同一 S/F 上相同或祖先/后代重叠的规则在构造时拒绝；互不重叠的兄弟路径 +可以一起替换。原记录和原 Item 树不会被修改。 + +SecsTraceCodec 本身不会自动脱敏。写入磁盘、日志、Issue 或测试 +附件前，应用必须根据自身数据分级先执行规则，并对导出结果使用常规访问控制 +和保留策略。 + +## 受控重放 + +SecsTraceReplayer 要求调用方提供显式 allowlist predicate： + +~~~csharp +var results = await new SecsTraceReplayer().ReplayAsync( + records, + connection, + record => record.Message.Stream == 1 && + record.Message.Function == 1, + cancellationToken); +~~~ + +重放具有以下固定边界： + +- 先完整枚举、验证记录数量并计算 allowlist，再开始第一次发送； +- 只考虑 Sent 记录，Received 永远跳过； +- 按源顺序逐条等待正常 SendAsync，W-Bit 继续使用现有 T3； +- 忽略原时间、Session ID、System Bytes 和回复令牌，由活连接创建新事务； +- 不等待连接进入 Selected，不自动重试，也不按原时间间隔暂停； +- 返回每条已发送记录及新事务获得的可选 Secondary。 + +allowlist 是业务安全边界。库不会依据 Function 奇偶、消息名称或未知设备状态 +推断 Primary/Secondary、幂等性或命令权限；应用只能允许已确认可在目标设备 +与当前状态执行的消息。 + +## 验证边界 + +测试覆盖确定性信封、多记录往返、畸形头与长度、SML 嵌入错误、记录和文本 +资源限制、路径漂移、规则重叠、敏感值不进入导出文本、预验证零发送，以及 +真实 TCP 上重新分配 Session ID/System Bytes 并获得新 Secondary。它证明 +当前工具组合边界，不代表设备命令安全性或 SEMI 一致性认证。 diff --git a/src/SecsFrame.Trace/SecsFrame.Trace.csproj b/src/SecsFrame.Trace/SecsFrame.Trace.csproj new file mode 100644 index 0000000..b33455e --- /dev/null +++ b/src/SecsFrame.Trace/SecsFrame.Trace.csproj @@ -0,0 +1,24 @@ + + + netstandard2.0;net8.0;net10.0 + Deterministic trace export, redaction, and controlled replay for SecsFrame. + SecsFrame.Trace + MIT + README.md + secs;hsms;trace;redaction;replay;diagnostics;secsframe + 0.1.0-alpha.1 + true + false + true + snupkg + + + + + + + + + + + diff --git a/src/SecsFrame.Trace/SecsTraceCodec.cs b/src/SecsFrame.Trace/SecsTraceCodec.cs new file mode 100644 index 0000000..9e560f7 --- /dev/null +++ b/src/SecsFrame.Trace/SecsTraceCodec.cs @@ -0,0 +1,250 @@ +using System.Collections.ObjectModel; +using System.Globalization; +using System.Text; +using SecsFrame.Sml; + +namespace SecsFrame.Trace; + +/// Encodes and decodes deterministic decoded-message trace files. +public sealed class SecsTraceCodec +{ + /// The exact trace format identifier. + public const string FormatIdentifier = "SecsFrame-Trace/1"; + + /// The default maximum number of records in one trace. + public const int DefaultMaxRecordCount = 100_000; + + /// The default maximum trace text length. + public const int DefaultMaxTextLength = 64 * 1024 * 1024; + + /// Creates a trace codec with explicit resource limits. + public SecsTraceCodec( + SmlMessageCodec? messageCodec = null, + int maxRecordCount = DefaultMaxRecordCount, + int maxTextLength = DefaultMaxTextLength) + { + if (maxRecordCount <= 0) + throw new ArgumentOutOfRangeException(nameof(maxRecordCount), maxRecordCount, "The maximum record count must be positive."); + if (maxTextLength <= 0) + throw new ArgumentOutOfRangeException(nameof(maxTextLength), maxTextLength, "The maximum text length must be positive."); + + MessageCodec = messageCodec ?? new SmlMessageCodec(); + MaxRecordCount = maxRecordCount; + MaxTextLength = maxTextLength; + } + + /// Gets the SML codec used for each decoded data message. + public SmlMessageCodec MessageCodec { get; } + + /// Gets the maximum number of records in one trace. + public int MaxRecordCount { get; } + + /// Gets the maximum accepted or produced trace text length. + public int MaxTextLength { get; } + + /// Encodes records in enumeration order using LF line endings. + public string Encode(IEnumerable records) + { + if (records is null) + throw new ArgumentNullException(nameof(records)); + + var text = new StringBuilder(FormatIdentifier); + text.Append('\n'); + var recordCount = 0; + foreach (var record in records) + { + if (record is null) + throw new ArgumentException("The trace sequence contains a null record.", nameof(records)); + if (++recordCount > MaxRecordCount) + throw new InvalidOperationException($"The trace record count exceeds the configured maximum {MaxRecordCount}."); + + var messageText = MessageCodec.Encode(record.Message); + AppendRecordHeader(text, record, messageText.Length); + text.Append(messageText); + if (text.Length > MaxTextLength) + throw new InvalidOperationException($"The trace text length exceeds the configured maximum {MaxTextLength}."); + } + + return text.ToString(); + } + + /// Strictly decodes one complete trace file. + public IReadOnlyList Decode(string text) + { + if (text is null) + throw new ArgumentNullException(nameof(text)); + if (text.Length > MaxTextLength) + throw new ArgumentOutOfRangeException(nameof(text), text.Length, $"The trace text length cannot exceed {MaxTextLength} characters."); + + return new SecsTraceParser(text, this).Parse(); + } + + private static void AppendRecordHeader(StringBuilder text, SecsTraceRecord record, int messageLength) + { + text.Append("Record "); + text.Append(record.Timestamp.UtcDateTime.ToString("O", CultureInfo.InvariantCulture)); + text.Append(' '); + text.Append(record.Direction == SecsTraceDirection.Sent ? "Sent" : "Received"); + text.Append(' '); + text.Append(record.SessionId.HasValue + ? record.SessionId.Value.ToString(CultureInfo.InvariantCulture) + : "-"); + text.Append(' '); + text.Append(record.SystemBytes.HasValue + ? "0x" + record.SystemBytes.Value.ToString("X8", CultureInfo.InvariantCulture) + : "-"); + text.Append(' '); + text.Append(messageLength.ToString(CultureInfo.InvariantCulture)); + text.Append('\n'); + } + + private sealed class SecsTraceParser + { + private readonly string _text; + private readonly SecsTraceCodec _options; + private int _index; + private int _recordIndex; + + public SecsTraceParser(string text, SecsTraceCodec options) + { + _text = text; + _options = options; + } + + public IReadOnlyList Parse() + { + var headerOffset = _index; + var formatIdentifier = ReadLine(); + if (!string.Equals(formatIdentifier, SecsTraceCodec.FormatIdentifier, StringComparison.Ordinal)) + throw Error("The trace format identifier is missing or unsupported", recordIndex: -1, headerOffset); + + var records = new List(); + while (!IsEnd) + { + if (_recordIndex >= _options.MaxRecordCount) + throw Error($"The trace record count exceeds the configured maximum {_options.MaxRecordCount}", _recordIndex, _index); + + records.Add(ParseRecord()); + _recordIndex++; + } + + return new ReadOnlyCollection(records); + } + + private SecsTraceRecord ParseRecord() + { + var headerOffset = _index; + var header = ReadLine(); + var fields = header.Split(new[] { ' ' }, StringSplitOptions.RemoveEmptyEntries); + if (fields.Length != 6 || !string.Equals(fields[0], "Record", StringComparison.Ordinal)) + throw Error("A record header must contain exactly six fields", _recordIndex, headerOffset); + + var timestamp = ParseTimestamp(fields[1], headerOffset); + var direction = ParseDirection(fields[2], headerOffset); + var sessionId = ParseSessionId(fields[3], headerOffset); + var systemBytes = ParseSystemBytes(fields[4], headerOffset); + var messageLength = ParseMessageLength(fields[5], headerOffset); + if (messageLength > _text.Length - _index) + throw Error("The SML block is shorter than its declared character length", _recordIndex, _index); + + var messageOffset = _index; + var messageText = _text.Substring(_index, messageLength); + _index += messageLength; + SecsMessage message; + try + { + message = _options.MessageCodec.Decode(messageText); + } + catch (SmlParseException ex) + { + throw Error("The record contains invalid SML", _recordIndex, messageOffset + ex.Offset, ex); + } + + return new SecsTraceRecord(timestamp, direction, message, sessionId, systemBytes); + } + + private DateTimeOffset ParseTimestamp(string value, int offset) + { + if (!DateTimeOffset.TryParseExact( + value, + "O", + CultureInfo.InvariantCulture, + DateTimeStyles.RoundtripKind, + out var timestamp)) + { + throw Error("The record timestamp must use the round-trip format", _recordIndex, offset); + } + + return timestamp; + } + + private SecsTraceDirection ParseDirection(string value, int offset) + => value switch + { + "Sent" => SecsTraceDirection.Sent, + "Received" => SecsTraceDirection.Received, + _ => throw Error("The record direction must be Sent or Received", _recordIndex, offset), + }; + + private ushort? ParseSessionId(string value, int offset) + { + if (string.Equals(value, "-", StringComparison.Ordinal)) + return null; + if (!ushort.TryParse(value, NumberStyles.None, CultureInfo.InvariantCulture, out var sessionId)) + throw Error("The protocol Session ID must be '-' or an unsigned 16-bit integer", _recordIndex, offset); + return sessionId; + } + + private uint? ParseSystemBytes(string value, int offset) + { + if (string.Equals(value, "-", StringComparison.Ordinal)) + return null; + if (value.Length != 10 || value[0] != '0' || value[1] != 'x') + throw Error("System Bytes must be '-' or use the exact form 0xHHHHHHHH", _recordIndex, offset); + + for (var index = 2; index < value.Length; index++) + { + var character = value[index]; + if (!((character >= '0' && character <= '9') || (character >= 'A' && character <= 'F'))) + throw Error("System Bytes contain an invalid uppercase hexadecimal digit", _recordIndex, offset); + } + + if (!uint.TryParse(value.Substring(2), NumberStyles.AllowHexSpecifier, CultureInfo.InvariantCulture, out var systemBytes)) + throw Error("System Bytes are outside the unsigned 32-bit range", _recordIndex, offset); + return systemBytes; + } + + private int ParseMessageLength(string value, int offset) + { + if (!int.TryParse(value, NumberStyles.None, CultureInfo.InvariantCulture, out var length) || length <= 0) + throw Error("The SML character length must be a positive integer", _recordIndex, offset); + if (length > _options.MessageCodec.MaxTextLength) + throw Error($"The SML character length exceeds the configured maximum {_options.MessageCodec.MaxTextLength}", _recordIndex, offset); + return length; + } + + private string ReadLine() + { + var start = _index; + while (!IsEnd && _text[_index] != '\n') + _index++; + + var length = _index - start; + if (length > 0 && _text[start + length - 1] == '\r') + length--; + var line = _text.Substring(start, length); + if (!IsEnd) + _index++; + return line; + } + + private bool IsEnd => _index >= _text.Length; + + private static SecsTraceParseException Error( + string message, + int recordIndex, + int offset, + Exception? innerException = null) + => new(message, recordIndex, offset, innerException); + } +} diff --git a/src/SecsFrame.Trace/SecsTraceDirection.cs b/src/SecsFrame.Trace/SecsTraceDirection.cs new file mode 100644 index 0000000..7fd9e6d --- /dev/null +++ b/src/SecsFrame.Trace/SecsTraceDirection.cs @@ -0,0 +1,11 @@ +namespace SecsFrame.Trace; + +/// Identifies the local direction of a decoded data message. +public enum SecsTraceDirection +{ + /// The local endpoint sent the message. + Sent, + + /// The local endpoint received the message. + Received, +} diff --git a/src/SecsFrame.Trace/SecsTraceParseException.cs b/src/SecsFrame.Trace/SecsTraceParseException.cs new file mode 100644 index 0000000..4347e1e --- /dev/null +++ b/src/SecsFrame.Trace/SecsTraceParseException.cs @@ -0,0 +1,18 @@ +namespace SecsFrame.Trace; + +/// Reports a strict trace envelope parsing failure. +public sealed class SecsTraceParseException : FormatException +{ + internal SecsTraceParseException(string message, int recordIndex, int offset, Exception? innerException = null) + : base($"{message} (record {recordIndex}, offset {offset}).", innerException) + { + RecordIndex = recordIndex; + Offset = offset; + } + + /// Gets the zero-based record index, or -1 for the file header. + public int RecordIndex { get; } + + /// Gets the zero-based character offset. + public int Offset { get; } +} diff --git a/src/SecsFrame.Trace/SecsTraceRecord.cs b/src/SecsFrame.Trace/SecsTraceRecord.cs new file mode 100644 index 0000000..43c177f --- /dev/null +++ b/src/SecsFrame.Trace/SecsTraceRecord.cs @@ -0,0 +1,65 @@ +namespace SecsFrame.Trace; + +/// +/// Describes one decoded SECS-II data message with optional HSMS diagnostic +/// identifiers. +/// +public sealed class SecsTraceRecord +{ + /// Creates an immutable decoded-message trace record. + public SecsTraceRecord( + DateTimeOffset timestamp, + SecsTraceDirection direction, + SecsMessage message, + ushort? sessionId = null, + uint? systemBytes = null) + { + if (direction != SecsTraceDirection.Sent && direction != SecsTraceDirection.Received) + throw new ArgumentOutOfRangeException(nameof(direction), direction, "Unknown trace direction."); + + Timestamp = timestamp.ToUniversalTime(); + Direction = direction; + Message = message ?? throw new ArgumentNullException(nameof(message)); + SessionId = sessionId; + SystemBytes = systemBytes; + } + + /// Gets the UTC timestamp associated with the local observation. + public DateTimeOffset Timestamp { get; } + + /// Gets the local message direction. + public SecsTraceDirection Direction { get; } + + /// Gets the optional HSMS protocol Session ID. + public ushort? SessionId { get; } + + /// Gets the optional original System Bytes value. + public uint? SystemBytes { get; } + + /// Gets the decoded dynamic SECS-II message. + public SecsMessage Message { get; } + + /// Creates a sent record before a public connection send. + public static SecsTraceRecord CreateSent( + DateTimeOffset timestamp, + SecsMessage message, + ushort? sessionId = null) + => new(timestamp, SecsTraceDirection.Sent, message, sessionId); + + /// Creates a received record from a decoded incoming message. + public static SecsTraceRecord CreateReceived( + DateTimeOffset timestamp, + HsmsIncomingDataMessage incoming) + { + if (incoming is null) + throw new ArgumentNullException(nameof(incoming)); + + var dataMessage = incoming.DataMessage; + return new SecsTraceRecord( + timestamp, + SecsTraceDirection.Received, + dataMessage.Message, + dataMessage.SessionId, + dataMessage.SystemBytes); + } +} diff --git a/src/SecsFrame.Trace/SecsTraceRedactionRule.cs b/src/SecsFrame.Trace/SecsTraceRedactionRule.cs new file mode 100644 index 0000000..021a59d --- /dev/null +++ b/src/SecsFrame.Trace/SecsTraceRedactionRule.cs @@ -0,0 +1,52 @@ +using System.Collections.ObjectModel; + +namespace SecsFrame.Trace; + +/// +/// Replaces one Item selected by Stream, Function, and zero-based List path. +/// +public sealed class SecsTraceRedactionRule +{ + private readonly ReadOnlyCollection _itemPath; + + /// Creates one exact structural redaction rule. + /// The matching message stream. + /// The matching message function. + /// List indexes from the root Item; empty selects the root. + /// The replacement Item written into the exported copy. + public SecsTraceRedactionRule( + byte stream, + byte function, + IEnumerable itemPath, + SecsItem replacement) + { + if (stream > 0x7F) + throw new ArgumentOutOfRangeException(nameof(stream), stream, "The stream number must be between 0 and 127."); + if (itemPath is null) + throw new ArgumentNullException(nameof(itemPath)); + + var path = itemPath.ToArray(); + for (var index = 0; index < path.Length; index++) + { + if (path[index] < 0) + throw new ArgumentOutOfRangeException(nameof(itemPath), path[index], "Item path indexes cannot be negative."); + } + + Stream = stream; + Function = function; + _itemPath = Array.AsReadOnly(path); + Replacement = replacement ?? throw new ArgumentNullException(nameof(replacement)); + } + + /// Gets the matching message stream. + public byte Stream { get; } + + /// Gets the matching message function. + public byte Function { get; } + + /// Gets the zero-based List path from the root Item. + public IReadOnlyList ItemPath => _itemPath; + + /// Gets the immutable replacement Item. + public SecsItem Replacement { get; } +} diff --git a/src/SecsFrame.Trace/SecsTraceRedactor.cs b/src/SecsFrame.Trace/SecsTraceRedactor.cs new file mode 100644 index 0000000..bdb3b43 --- /dev/null +++ b/src/SecsFrame.Trace/SecsTraceRedactor.cs @@ -0,0 +1,100 @@ +namespace SecsFrame.Trace; + +/// Applies exact structural replacement rules before trace export. +public sealed class SecsTraceRedactor +{ + private readonly SecsTraceRedactionRule[] _rules; + + /// Creates a redactor and rejects ambiguous overlapping rules. + public SecsTraceRedactor(IEnumerable rules) + { + if (rules is null) + throw new ArgumentNullException(nameof(rules)); + + _rules = rules.ToArray(); + for (var index = 0; index < _rules.Length; index++) + { + if (_rules[index] is null) + throw new ArgumentException("The redaction rule sequence contains a null rule.", nameof(rules)); + for (var otherIndex = index + 1; otherIndex < _rules.Length; otherIndex++) + { + if (_rules[otherIndex] is null) + throw new ArgumentException("The redaction rule sequence contains a null rule.", nameof(rules)); + if (RulesOverlap(_rules[index], _rules[otherIndex])) + throw new ArgumentException("Redaction rules for the same message cannot target identical or ancestor paths.", nameof(rules)); + } + } + } + + /// + /// Returns the original record when no rule matches, otherwise a copy with + /// replacements applied. + /// + public SecsTraceRecord Redact(SecsTraceRecord record) + { + if (record is null) + throw new ArgumentNullException(nameof(record)); + + var rootItem = record.Message.RootItem; + var changed = false; + foreach (var rule in _rules) + { + if (rule.Stream != record.Message.Stream || rule.Function != record.Message.Function) + continue; + if (rootItem is null) + throw new InvalidOperationException("A matching redaction rule cannot resolve because the message has no root Item."); + + rootItem = ReplaceAtPath(rootItem, rule.ItemPath, pathIndex: 0, rule.Replacement); + changed = true; + } + + if (!changed) + return record; + + var message = new SecsMessage( + record.Message.Stream, + record.Message.Function, + record.Message.ReplyExpected, + rootItem); + return new SecsTraceRecord( + record.Timestamp, + record.Direction, + message, + record.SessionId, + record.SystemBytes); + } + + private static SecsItem ReplaceAtPath( + SecsItem item, + IReadOnlyList path, + int pathIndex, + SecsItem replacement) + { + if (pathIndex == path.Count) + return replacement; + if (item.Format != SecsItemFormat.List) + throw new InvalidOperationException($"The redaction path enters a non-List Item at depth {pathIndex}."); + + var childIndex = path[pathIndex]; + if (childIndex >= item.Items.Count) + throw new InvalidOperationException($"The redaction path index {childIndex} is outside the List at depth {pathIndex}."); + + var items = item.Items.ToArray(); + items[childIndex] = ReplaceAtPath(items[childIndex], path, pathIndex + 1, replacement); + return SecsItem.List(items); + } + + private static bool RulesOverlap(SecsTraceRedactionRule left, SecsTraceRedactionRule right) + { + if (left.Stream != right.Stream || left.Function != right.Function) + return false; + + var sharedLength = Math.Min(left.ItemPath.Count, right.ItemPath.Count); + for (var index = 0; index < sharedLength; index++) + { + if (left.ItemPath[index] != right.ItemPath[index]) + return false; + } + return true; + } +} diff --git a/src/SecsFrame.Trace/SecsTraceReplayResult.cs b/src/SecsFrame.Trace/SecsTraceReplayResult.cs new file mode 100644 index 0000000..2e35c27 --- /dev/null +++ b/src/SecsFrame.Trace/SecsTraceReplayResult.cs @@ -0,0 +1,17 @@ +namespace SecsFrame.Trace; + +/// Describes one sent trace record and its optional new Secondary. +public sealed class SecsTraceReplayResult +{ + internal SecsTraceReplayResult(SecsTraceRecord record, HsmsDataMessage? secondary) + { + Record = record; + Secondary = secondary; + } + + /// Gets the source trace record. + public SecsTraceRecord Record { get; } + + /// Gets the Secondary produced by the new transaction, when requested. + public HsmsDataMessage? Secondary { get; } +} diff --git a/src/SecsFrame.Trace/SecsTraceReplayer.cs b/src/SecsFrame.Trace/SecsTraceReplayer.cs new file mode 100644 index 0000000..fa29005 --- /dev/null +++ b/src/SecsFrame.Trace/SecsTraceReplayer.cs @@ -0,0 +1,81 @@ +using System.Collections.ObjectModel; + +namespace SecsFrame.Trace; + +/// Replays explicitly allowed sent records through a normal send API. +public sealed class SecsTraceReplayer +{ + /// The default maximum number of source records in one replay request. + public const int DefaultMaxRecordCount = SecsTraceCodec.DefaultMaxRecordCount; + + /// Creates a controlled replayer with an explicit source-record limit. + public SecsTraceReplayer(int maxRecordCount = DefaultMaxRecordCount) + { + if (maxRecordCount <= 0) + throw new ArgumentOutOfRangeException(nameof(maxRecordCount), maxRecordCount, "The maximum record count must be positive."); + MaxRecordCount = maxRecordCount; + } + + /// Gets the maximum number of source records in one replay request. + public int MaxRecordCount { get; } + + /// + /// Replays allowed sent records through an existing selected connection. + /// Original Session ID, System Bytes, timestamps, and reply tokens are ignored. + /// + public Task> ReplayAsync( + IEnumerable records, + HsmsConnection connection, + Func isAllowed, + CancellationToken cancellationToken = default) + { + if (connection is null) + throw new ArgumentNullException(nameof(connection)); + + return ReplayAsync(records, connection.SendAsync, isAllowed, cancellationToken); + } + + /// + /// Replays allowed sent records through a caller-supplied normal send API. + /// This overload can target a role endpoint or a test boundary. + /// + public async Task> ReplayAsync( + IEnumerable records, + Func> sendAsync, + Func isAllowed, + CancellationToken cancellationToken = default) + { + if (records is null) + throw new ArgumentNullException(nameof(records)); + if (sendAsync is null) + throw new ArgumentNullException(nameof(sendAsync)); + if (isAllowed is null) + throw new ArgumentNullException(nameof(isAllowed)); + + var selected = new List(); + var sourceCount = 0; + foreach (var record in records) + { + cancellationToken.ThrowIfCancellationRequested(); + if (record is null) + throw new ArgumentException("The replay sequence contains a null record.", nameof(records)); + if (++sourceCount > MaxRecordCount) + throw new InvalidOperationException($"The replay source record count exceeds the configured maximum {MaxRecordCount}."); + if (record.Direction == SecsTraceDirection.Sent && isAllowed(record)) + selected.Add(record); + } + + var results = new List(selected.Count); + foreach (var record in selected) + { + cancellationToken.ThrowIfCancellationRequested(); + var operation = sendAsync(record.Message, cancellationToken); + if (operation is null) + throw new InvalidOperationException("The replay send delegate returned a null task."); + var secondary = await operation.ConfigureAwait(false); + results.Add(new SecsTraceReplayResult(record, secondary)); + } + + return new ReadOnlyCollection(results); + } +} diff --git a/test/SecsFrame.Tests/SecsFrame.Tests.csproj b/test/SecsFrame.Tests/SecsFrame.Tests.csproj index 2a9e300..118c13a 100644 --- a/test/SecsFrame.Tests/SecsFrame.Tests.csproj +++ b/test/SecsFrame.Tests/SecsFrame.Tests.csproj @@ -16,6 +16,7 @@ + diff --git a/test/SecsFrame.Tests/SecsTraceTests.cs b/test/SecsFrame.Tests/SecsTraceTests.cs new file mode 100644 index 0000000..47ba50b --- /dev/null +++ b/test/SecsFrame.Tests/SecsTraceTests.cs @@ -0,0 +1,319 @@ +using System.Net; +using SecsFrame.Trace; + +namespace SecsFrame.Tests; + +public sealed class SecsTraceTests +{ + [Fact] + public void Codec_produces_a_deterministic_trace_vector() + { + var timestamp = new DateTimeOffset(2026, 8, 29, 13, 0, 0, TimeSpan.FromHours(8)); + var record = SecsTraceRecord.CreateSent(timestamp, new SecsMessage(1, 1, true)); + var codec = new SecsTraceCodec(); + + var text = codec.Encode(new[] { record }); + + Assert.Equal( + "SecsFrame-Trace/1\n" + + "Record 2026-08-29T05:00:00.0000000Z Sent - - 10\n" + + "'S1F1'W\n" + + ".\n", + text); + var decoded = Assert.Single(codec.Decode(text)); + Assert.Equal(timestamp.ToUniversalTime(), decoded.Timestamp); + Assert.Equal(SecsTraceDirection.Sent, decoded.Direction); + Assert.Null(decoded.SessionId); + Assert.Null(decoded.SystemBytes); + AssertMessageEqual(record.Message, decoded.Message); + } + + [Fact] + public void Codec_round_trips_multiple_records_and_optional_hsms_identifiers() + { + var records = new[] + { + new SecsTraceRecord( + DateTimeOffset.Parse("2026-08-29T05:00:00Z", System.Globalization.CultureInfo.InvariantCulture), + SecsTraceDirection.Received, + new SecsMessage(6, 11, true, SecsItem.List(SecsItem.Ascii("LOT-1"))), + sessionId: 10, + systemBytes: 0x01020304), + SecsTraceRecord.CreateSent( + DateTimeOffset.Parse("2026-08-29T05:00:01Z", System.Globalization.CultureInfo.InvariantCulture), + new SecsMessage(6, 12, rootItem: SecsItem.Binary(0x00))), + }; + var codec = new SecsTraceCodec(); + + var decoded = codec.Decode(codec.Encode(records)); + + Assert.Equal(2, decoded.Count); + Assert.Equal(SecsTraceDirection.Received, decoded[0].Direction); + Assert.Equal((ushort)10, decoded[0].SessionId); + Assert.Equal(0x01020304u, decoded[0].SystemBytes); + AssertMessageEqual(records[0].Message, decoded[0].Message); + AssertMessageEqual(records[1].Message, decoded[1].Message); + } + + [Fact] + public void Received_factory_preserves_decoded_protocol_identifiers() + { + var dataMessage = new HsmsDataMessage(42, 0x89ABCDEF, new SecsMessage(5, 1)); + var incoming = new HsmsIncomingDataMessage(new object(), new HsmsTransportSessionId(7), dataMessage); + + var record = SecsTraceRecord.CreateReceived(DateTimeOffset.UtcNow, incoming); + + Assert.Equal(SecsTraceDirection.Received, record.Direction); + Assert.Equal((ushort)42, record.SessionId); + Assert.Equal(0x89ABCDEFu, record.SystemBytes); + Assert.Same(dataMessage.Message, record.Message); + } + + [Fact] + public void Codec_rejects_malformed_headers_lengths_and_embedded_sml() + { + var codec = new SecsTraceCodec(); + var valid = codec.Encode(new[] + { + new SecsTraceRecord( + Epoch, + SecsTraceDirection.Received, + new SecsMessage(1, 1), + 1, + 0x01020304), + }); + + Assert.Throws(() => codec.Decode(string.Empty)); + Assert.Throws(() => codec.Decode(valid.Replace("Trace/1", "Trace/2"))); + Assert.Throws(() => codec.Decode(valid.Replace("Received", "Sideways"))); + Assert.Throws(() => codec.Decode(valid.Replace("0x01020304", "0x0102030a"))); + Assert.Throws(() => codec.Decode(valid.Replace(" 9\n'S1F1'", " 99\n'S1F1'"))); + Assert.Throws(() => codec.Decode( + "SecsFrame-Trace/1\nRecord 1970-01-01T00:00:00.0000000Z Sent - - 4\nnope")); + } + + [Fact] + public void Codec_enforces_record_and_text_limits() + { + var record = SecsTraceRecord.CreateSent(Epoch, new SecsMessage(1, 1)); + var twoRecords = new[] { record, record }; + var encoded = new SecsTraceCodec().Encode(twoRecords); + + Assert.Throws(() => new SecsTraceCodec(maxRecordCount: 1).Encode(twoRecords)); + Assert.Throws(() => new SecsTraceCodec(maxRecordCount: 1).Decode(encoded)); + Assert.Throws(() => new SecsTraceCodec(maxTextLength: 20).Encode(new[] { record })); + Assert.Throws(() => new SecsTraceCodec(maxTextLength: 20).Decode(encoded)); + Assert.Throws(() => new SecsTraceCodec(maxRecordCount: 0)); + Assert.Throws(() => new SecsTraceCodec(maxTextLength: 0)); + } + + [Fact] + public void Structural_redaction_removes_sensitive_values_before_export() + { + var original = new SecsTraceRecord( + Epoch, + SecsTraceDirection.Received, + new SecsMessage( + 6, + 11, + true, + SecsItem.List( + SecsItem.Ascii("SECRET-LOT"), + SecsItem.List(SecsItem.U4(42), SecsItem.Ascii("SECRET-USER")))), + 10, + 0x01020304); + var redactor = new SecsTraceRedactor(new[] + { + new SecsTraceRedactionRule(6, 11, new[] { 0 }, SecsItem.Ascii("REDACTED")), + new SecsTraceRedactionRule(6, 11, new[] { 1, 1 }, SecsItem.Ascii("REDACTED")), + }); + + var redacted = redactor.Redact(original); + var exported = new SecsTraceCodec().Encode(new[] { redacted }); + + Assert.Equal("SECRET-LOT", original.Message.RootItem![0].GetString()); + Assert.Equal("REDACTED", redacted.Message.RootItem![0].GetString()); + Assert.Equal("REDACTED", redacted.Message.RootItem[1][1].GetString()); + Assert.Equal(original.Timestamp, redacted.Timestamp); + Assert.Equal(original.SessionId, redacted.SessionId); + Assert.Equal(original.SystemBytes, redacted.SystemBytes); + Assert.DoesNotContain("SECRET", exported, StringComparison.Ordinal); + Assert.Contains("REDACTED", exported, StringComparison.Ordinal); + } + + [Fact] + public void Redaction_is_strict_about_paths_and_ambiguous_rules() + { + var record = SecsTraceRecord.CreateSent( + Epoch, + new SecsMessage(1, 1, rootItem: SecsItem.List(SecsItem.Ascii("value")))); + var missing = new SecsTraceRedactor(new[] + { + new SecsTraceRedactionRule(1, 1, new[] { 1 }, SecsItem.Ascii("x")), + }); + + Assert.Throws(() => missing.Redact(record)); + Assert.Throws( + () => new SecsTraceRedactionRule(1, 1, new[] { -1 }, SecsItem.Ascii("x"))); + Assert.Throws(() => new SecsTraceRedactor(new[] + { + new SecsTraceRedactionRule(1, 1, new[] { 0 }, SecsItem.Ascii("x")), + new SecsTraceRedactionRule(1, 1, new[] { 0, 1 }, SecsItem.Ascii("y")), + })); + } + + [Fact] + public async Task Replay_prevalidates_filters_and_sends_in_source_order() + { + var received = new SecsTraceRecord(Epoch, SecsTraceDirection.Received, new SecsMessage(1, 9)); + var first = SecsTraceRecord.CreateSent(Epoch, new SecsMessage(1, 1, true)); + var denied = SecsTraceRecord.CreateSent(Epoch, new SecsMessage(2, 1)); + var second = SecsTraceRecord.CreateSent(Epoch, new SecsMessage(1, 3)); + var sent = new List(); + + var results = await new SecsTraceReplayer().ReplayAsync( + new[] { received, first, denied, second }, + (message, _) => + { + sent.Add(message); + HsmsDataMessage? secondary = message.ReplyExpected + ? new HsmsDataMessage(10, 99, new SecsMessage(message.Stream, (byte)(message.Function + 1))) + : null; + return Task.FromResult(secondary); + }, + record => record.Message.Stream == 1).ConfigureAwait(true); + + Assert.Equal(new byte[] { 1, 3 }, sent.Select(message => message.Function)); + Assert.Equal(2, results.Count); + Assert.Same(first, results[0].Record); + Assert.NotNull(results[0].Secondary); + Assert.Same(second, results[1].Record); + Assert.Null(results[1].Secondary); + } + + [Fact] + public async Task Replay_validation_failure_occurs_before_any_send() + { + var record = SecsTraceRecord.CreateSent(Epoch, new SecsMessage(1, 1)); + var sends = 0; + var replayer = new SecsTraceReplayer(); + + await Assert.ThrowsAsync(() => replayer.ReplayAsync( + new SecsTraceRecord[] { record, null! }, + (_, _) => + { + sends++; + return Task.FromResult(null); + }, + _ => true)).ConfigureAwait(true); + + Assert.Equal(0, sends); + + await Assert.ThrowsAsync(() => new SecsTraceReplayer(1).ReplayAsync( + new[] { record, record }, + (_, _) => + { + sends++; + return Task.FromResult(null); + }, + _ => true)).ConfigureAwait(true); + Assert.Equal(0, sends); + + using var cancellation = new CancellationTokenSource(); + cancellation.Cancel(); + await Assert.ThrowsAnyAsync(() => replayer.ReplayAsync( + new[] { record }, + (_, _) => + { + sends++; + return Task.FromResult(null); + }, + _ => true, + cancellation.Token)).ConfigureAwait(true); + Assert.Equal(0, sends); + Assert.Throws(() => new SecsTraceReplayer(0)); + } + + [Fact] + public async Task Replay_uses_a_new_real_tcp_transaction_and_ignores_received_records() + { + var port = GetFreePort(); + await using var passive = new HsmsConnection(CreateOptions(port, HsmsConnectionMode.Passive)); + await using var active = new HsmsConnection(CreateOptions(port, HsmsConnectionMode.Active)); + using var cancellation = new CancellationTokenSource(TimeSpan.FromSeconds(15)); + passive.Start(); + active.Start(); + await using var events = passive.GetEventsAsync(cancellation.Token).GetAsyncEnumerator(); + await Task.WhenAll( + passive.WaitUntilSelectedAsync(cancellation.Token), + active.WaitUntilSelectedAsync(cancellation.Token)).ConfigureAwait(true); + + var primary = new SecsMessage(1, 1, true, SecsItem.Ascii("REPLAY")); + var source = new[] + { + new SecsTraceRecord(Epoch, SecsTraceDirection.Received, new SecsMessage(9, 9)), + new SecsTraceRecord(Epoch, SecsTraceDirection.Sent, primary, 999, 0xDEADBEEF), + }; + var replay = new SecsTraceReplayer().ReplayAsync(source, active, _ => true, cancellation.Token); + var received = await NextDataMessageAsync(events).ConfigureAwait(true); + + Assert.Equal((ushort)10, received.DataMessage.SessionId); + Assert.NotEqual(0xDEADBEEFu, received.DataMessage.SystemBytes); + AssertMessageEqual(primary, received.DataMessage.Message); + await passive.ReplyAsync( + received, + new SecsMessage(1, 2, rootItem: SecsItem.Boolean(true)), + cancellation.Token).ConfigureAwait(true); + var result = Assert.Single(await replay.ConfigureAwait(true)); + + Assert.NotNull(result.Secondary); + Assert.Equal((ushort)10, result.Secondary.SessionId); + Assert.NotEqual(0xDEADBEEFu, result.Secondary.SystemBytes); + Assert.Equal(2, result.Secondary.Message.Function); + } + + private static HsmsConnectionOptions CreateOptions(int port, HsmsConnectionMode mode) + => new( + IPAddress.Loopback, + port, + mode, + sessionId: 10, + t3: TimeSpan.FromSeconds(5), + t5: TimeSpan.FromMilliseconds(10), + t6: TimeSpan.FromSeconds(5), + t7: TimeSpan.FromSeconds(10), + t8: TimeSpan.FromSeconds(5)); + + private static DateTimeOffset Epoch + => new(1970, 1, 1, 0, 0, 0, TimeSpan.Zero); + + private static int GetFreePort() + { + var listener = new System.Net.Sockets.TcpListener(IPAddress.Loopback, 0); + listener.Start(); + var port = ((IPEndPoint)listener.LocalEndpoint).Port; + listener.Stop(); + return port; + } + + private static async Task NextDataMessageAsync( + IAsyncEnumerator events) + { + while (await events.MoveNextAsync().ConfigureAwait(true)) + { + if (events.Current.Kind == HsmsConnectionEventKind.DataMessageReceived) + return events.Current.IncomingMessage!; + } + + Assert.Fail("The replayed data message was not received."); + return null!; + } + + private static void AssertMessageEqual(SecsMessage expected, SecsMessage actual) + { + Assert.Equal(expected.Stream, actual.Stream); + Assert.Equal(expected.Function, actual.Function); + Assert.Equal(expected.ReplyExpected, actual.ReplyExpected); + Assert.Equal(expected.RootItem, actual.RootItem); + } +}