[WIP] Integrate C++ DataStream concept into C# codebase

[Copilot]

• Inspect current MetaStream/MetaStreamReader/MetaStreamWriter architecture and identify Stream/BinaryReader/BinaryWriter coupling points
• Run baseline build/tests to capture pre-existing status before changes
• Add new internal Serialization.DataStream abstraction with XML docs (DataStream base + DataStreamMemory, DataStreamFile, DataStreamSubStream)
• Refactor MetaStream and section storage to use DataStream while preserving existing public API usage
• Refactor MetaStreamReader/MetaStreamWriter internals to operate on DataStream instead of direct BinaryReader/BinaryWriter
• Build and run targeted validation for changed codepaths
• Run final solution build/tests and run parallel validation

Original prompt

Overview

Integrate the C++ DataStream concept into the C# codebase to provide a flexible, polymorphic stream abstraction layer. This refactoring will decouple MetaStream from direct System.IO.Stream usage and enable support for multiple stream types (file, memory, compressed, encrypted, sub-streams).

Current State

• MetaStream (abstract base class) directly uses System.IO.Stream in UnderlyingStream and Sections[].Stream
• MetaStreamReader and MetaStreamWriter hardcoded to work with .NET's BinaryReader/BinaryWriter
• No support for custom stream backends (compressed streams, encrypted streams, sub-streams, etc.)
• Limited flexibility for stream composition and transformation

Desired Architecture

Create a DataStream abstraction layer inspired by the C++ reference provided:

1. Create DataStream Interface/Base Class

Create src/TelltaleToolKit/Serialization/DataStream/DataStream.cs:

• Abstract base class that wraps around System.IO.Stream
• Methods:
  • Serialize(byte[] buffer, int offset, int count, bool write) - unified read/write
  • GetSize() - get total stream size
  • GetPosition() - get current offset
  • SetPosition(long offset, SeekOrigin) - set position
  • Truncate(long newSize) - truncate stream
  • GetSubStream(long offset, long size) - create read-only sub-stream view
  • Copy(DataStream dst, long dstOffset, long srcOffset, long size) - copy between streams

2. Create Concrete DataStream Implementations

DataStreamFile (src/TelltaleToolKit/Serialization/DataStream/DataStreamFile.cs)

• Wraps System.IO.FileStream
• Direct file I/O operations

DataStreamMemory (src/TelltaleToolKit/Serialization/DataStream/DataStreamMemory.cs)

• Wraps System.IO.MemoryStream
• With optional growth factor for efficient expansion

DataStreamSubStream (src/TelltaleToolKit/Serialization/DataStream/DataStreamSubStream.cs)

• Read-only view into a base stream section
• No truncation/modification allowed

DataStreamContainer (src/TelltaleToolKit/Serialization/DataStream/DataStreamContainer.cs)

• Optional: For compressed/encrypted streams
• Decompresses/decrypts on-the-fly

3. Refactor MetaStream to Use DataStream

• Replace Stream UnderlyingStream with DataStream UnderlyingStream
• Replace Sections[].Stream (currently System.IO.Stream) with DataStream
• Update MetaStreamReader and MetaStreamWriter to work with DataStream instead of BinaryReader/BinaryWriter
• Maintain backward compatibility by keeping the current public API unchanged

4. Benefits

• Flexible stream composition (e.g., sub-streams, compressed containers)
• Better abstraction matching the engine's C++ design
• Easier to add new stream types (e.g., encrypted streams)
• More control over stream operations (size queries, truncation, copying)
• C# idiomatic implementation

Implementation Notes

• Do NOT break existing public API in MetaStream, MetaStreamReader, or MetaStreamWriter
• Maintain compatibility with existing serializers and usage patterns
• Add comprehensive XML documentation to all new classes
• Follow the existing C# code style (PascalCase, region organization, null-safety patterns)
• Keep the DataStream classes internal or in a separate Serialization.DataStream namespace

Priority

Start with the base DataStream class and DataStreamMemory implementation. These are the most critical for supporting the core functionality.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: ```// This file was written by Lucas Saragosa. Im the author of this interpretation of
// the engine and require that if you use this code or library, you give credit to me and
// the amazing Telltale Games.

#pragma once

#include "../LibraryConfig.h"
#include "../Compression.h"
#include
#include
#include
#include

#ifndef _DATASTREAM
#define _DATASTREAM

#define DEFAULT_GROWTH_FACTOR 0x1000

#define READ DataStreamMode::eMode_Read
#define WRITE DataStreamMode::eMode_Write

//Returns a new instance as an object directly
#define OpenDataStreamFromDisc(file_path, mode) DataStreamFileDisc(
PlatformSpecOpenFile(file_path,
mode),
mode)

//Returns a new instance as a pointer which needs to be deleted. Most classes
//will delete it when done with it (consumers)
#define _OpenDataStreamFromDisc(file_path, mode)
new OpenDataStreamFromDisc(file_path, mode)

//With line terminator
#define OpenDataStreamFromDisc(file_path, mode)
_OpenDataStreamFromDisc(file_path, mode);

enum class DataStreamSeekType : unsigned char {
eSeekType_Begin = 0,
eSeekType_Current = 1,
eSeekType_End = 2
};

class DataStreamSubStream;

/*

• A data stream. Abstract class which represents a stream of data (bytes) being read or written.

• Class is not copyable but is moveable.
  */
  class DataStream {
  public:
  DataStreamMode mMode;
  int mSubStreams;

  virtual bool Copy(DataStream* pDst, unsigned __i...

This pull request was created from Copilot chat.