Skip to content

CHANGELOG.md

Latest commit

 

History

History
351 lines (256 loc) · 18.5 KB

CHANGELOG.md

File metadata and controls

351 lines (256 loc) · 18.5 KB
Raw

Changelog

All notable changes to databento-dotnet will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[5.3.5] - 2026-07-24

Fixed

  • Fix ConsolidatedBidAskPair.AskPublisher always 0 — deserialization read from reserved padding at offset 26 instead of the actual ask_pb field at offset 28 per the DBN spec. Affects all record types carrying consolidated levels: Cmbp1, Cbbo, and Tcbbo (Issue #36)

[5.3.4] - 2026-06-28

Added

  • stype_in and limit parameters for metadata APIs — GetCostAsync, GetRecordCountAsync, GetBillableSizeAsync, and GetBillingInfoAsync now accept SType stypeIn and ulong limit parameters, enabling queries with Continuous, Parent, Smart, and other symbol types (Issue #35)

Changed

  • Native wrapper updated — dbento_metadata_get_cost, dbento_metadata_get_record_count, dbento_metadata_get_billable_size, and dbento_metadata_get_billing_info now pass stype_in and limit through to databento-cpp's full overloads

[5.3.3] - 2026-05-29

Fixed

  • Add NOMINMAX guards to native wrappers — prevents <windows.h> min/max macro conflicts that could cause compilation failures with certain build configurations (Issue #24)

Changed

  • Thread safety documentation — added class-level and method-level thread safety remarks to ILiveClient and ILiveBlockingClient documenting the databento-cpp calling convention: complete all Subscribe calls before StartAsync, await each call sequentially
  • README thread safety callout — added thread safety guidance between LiveClient and LiveBlockingClient API reference sections
  • Coding agent guide — updated to v5.3.3, added pitfall #6 documenting concurrent Subscribe call risks with correct/incorrect code examples

[5.3.2] - 2026-05-26

Fixed

  • Pin databento-cpp to v0.58.0 to prevent builds from pulling broken intermediate states from main branch (Issue #24)
  • Fix FormUrlEncodedContent disposal in retry loops — content was created outside the retry lambda, causing ObjectDisposedException on retry attempts in AdjustmentFactorsApi, CorporateActionsApi, and SecurityMasterApi (PR #28)
  • Fix XML doc comments — escape S&P as S&amp;P, qualify BatchSubmitJobAsync cref to resolve to correct overload (PR #25)
  • Fix CA1826 — replace FirstOrDefault() with direct indexer after count check in FileDataSource.CreateSymbolMappingMessage (PR #27)

Changed

  • Deprecate caching API — WithMemoryCache(), WithDiskCache(), CachePolicy, IRecordCache, DiskRecordCache, and MemoryRecordCache are now marked [Obsolete] with guidance to use WithFileSource() instead (Issue #29)
  • PlaybackController implements IDisposable — disposes internal SemaphoreSlim; called automatically by FileDataSource and HistoricalDataSource during disposal (Issue #29)
  • StreamAsync XML documentation — detailed docs explaining live mode (indefinite until StopAsync/cancellation) vs backtest mode (ends when data exhausted) (Issue #30)
  • Add ConfigureAwait(false) across Reference API methods for library best practices (PR #28)
  • Simplify reconnect lambda — async ct => await PerformReconnectAsync(ct) → PerformReconnectAsync in LiveClient constructor (PR #28)
  • Remove CA1805 default initializations — remove explicit = 0 and = false on fields across 10 files (PR #26)
  • Remove dead _apiKey field from LiveClient and ReferenceClient (PR #31)

[4.1.0] - 2025-11-28

Added

  • .NET 9 Support: Now targets both .NET 8 and .NET 9
  • Resilience Features: Added comprehensive connection resilience capabilities (non-breaking, opt-in)
    • RetryPolicy class with exponential backoff and jitter
    • ResilienceOptions for configuring auto-reconnect behavior
    • ConnectionHealthMonitor for detecting stale connections
    • New builder methods:
      • WithAutoReconnect(bool) - Enable automatic reconnection on failure
      • WithRetryPolicy(RetryPolicy) - Configure retry behavior
      • WithHeartbeatTimeout(TimeSpan) - Configure stale connection detection
      • WithResilienceOptions(ResilienceOptions) - Full resilience configuration
    • Pre-built policies: RetryPolicy.Default, RetryPolicy.Aggressive, ResilienceOptions.HighAvailability
    • Health monitoring events: OnReconnecting, OnReconnected, OnReconnectFailed

Fixed

  • CRITICAL: Fixed SEHException (STATUS_STACK_BUFFER_OVERRUN) crash during LiveClient disposal
    • Root Cause: databento-cpp's LiveThreaded destructor only signals internal thread to stop but doesn't wait for it to terminate, causing race condition where callbacks execute during/after resource cleanup
    • Solution: Implemented proper thread synchronization using databento-cpp's BlockForStop() method
    • Changes:
      • Added new native API dbento_live_stop_and_wait() that synchronously waits for thread termination
      • Updated StopAsync() to use new API with 10-second timeout
      • Updated dbento_live_destroy() to call BlockForStop() before resource cleanup
      • Removed SEHException catch workaround from DisposeAsync() (no longer needed)
    • Impact: await using pattern now works reliably without crashes
    • Files Changed:
      • src/Databento.Native/include/databento_native.h
      • src/Databento.Native/src/live_client_wrapper.cpp
      • src/Databento.Interop/Native/NativeMethods.cs
      • src/Databento.Client/Live/LiveClient.cs

[4.0.1-beta] - 2025-11-25

Fixed

  • CRITICAL: Fixed AccessViolationException crash in Historical, Live, and LiveBlocking clients
    • Root Cause: v4.0.0-beta removed logger from native client wrappers, passing nullptr to databento-cpp
    • Symptoms: Fatal crashes when databento-cpp tried to log messages (particularly on empty query results or warnings)
    • Impact: Affected ALL Historical API queries, Live subscriptions, and LiveBlocking operations in v4.0.0-beta
    • Fix: Restored StderrLogReceiver logger in all three client wrapper implementations
    • User Reports: Multiple users reported crashes immediately after upgrading to v4.0.0-beta
    • Recommendation: All users on v4.0.0-beta should upgrade to v4.0.1-beta immediately

[4.0.0-beta] - 2025-11-25

Changed (BREAKING)

  • InstrumentDefMessage.RawInstrumentId: Changed from uint to ulong to support venues with 64-bit instrument IDs
    • Some venues like Eurex (XEUR.EOBI) encode venue/type information in the upper 32 bits for complex instruments (spreads)
    • Example: Eurex spread IDs like 0x010002B100000060 (72,060,553,270,394,976) exceed uint.MaxValue (4,294,967,295)
    • Migration: Change any explicit uint declarations to ulong. Code using var or implicit conversions should work unchanged.
    • Impact: May affect serialization, database schemas (INT → BIGINT), or API contracts expecting 32-bit values
    • See: MIGRATION_GUIDE_v4.md for detailed migration instructions

Added

  • New Examples:
    • IntradayReplay2.Example - Demonstrates LiveClient streaming with replay mode
    • Get_Most_Recent_Market_Open.Example - Market open time calculation and querying
    • List_Available_Schemas.Example - Schema discovery via MetadataListSchemas

Fixed

  • InstrumentDefinitionDecoder.Example - Renamed to "OHLCV Bar Decoder" and rewritten to use OHLCV-1S schema (Definition schema has no time-series data)
  • TestsScratchpad.Internal - Fixed stream lifetime management (added StreamAsync loop to keep program alive)
  • Improved documentation distinguishing LiveClient (streaming/push) vs LiveBlockingClient (pull)

[3.0.28-beta] - 2025-11-23

Changed

  • Native Library: Rebuilt with clean, unmodified databento-cpp v0.44.0
    • Previous v3.0.27-beta inadvertently included debug-modified databento-cpp dependency
    • All functionality remains identical to v3.0.27-beta
    • No code changes, only dependency cleanup

Notes

This is a maintenance release to ensure production packages use official, unmodified upstream dependencies. All fixes from v3.0.27-beta are included and verified:

  • Issue #1: AccessViolationException with future dates - ✅ Working
  • Issue #4: InstrumentDefMessage.InstrumentClass always 0 - ✅ Working

[3.0.27-beta] - 2025-11-23

Fixed

  • CRITICAL: Fixed AccessViolationException crash in Historical and Batch APIs when server returns warning headers (#1)

    • Historical API now correctly handles server warnings for future dates and degraded data quality
    • Batch API now safely handles submission errors without crashing
    • Native wrapper now properly passes ILogReceiver to databento-cpp instead of NULL pointer
    • Server warnings now visible on stderr instead of causing silent crashes

  • CRITICAL: Fixed InstrumentDefMessage deserialization to match DBN v2 specification (#4)

    • InstrumentClass now correctly populated (was always returning 0/Unknown)
    • StrikePrice now reading from correct offset 104 (was reading from offset 320)
    • All string fields now reading from correct offsets with correct lengths

Added

  • 13 new fields for multi-leg strategy instruments in InstrumentDefMessage:
    • LegPrice, LegDelta, LegInstrumentId, LegRatioPriceNumerator, LegRatioPriceDenominator
    • LegRatioQtyNumerator, LegRatioQtyDenominator, LegUnderlyingId, LegCount, LegIndex
    • StrikePriceCurrency, LegRawSymbol, LegInstrumentClass, LegSide

Changed

  • RawInstrumentId now reads from correct offset 112 (was reading from incorrect offset)
    • ⚠️ Note: Property type changed to ulong in later version (see Unreleased section above)

[4.0.0-beta] - 2025-11-22

🚨 BREAKING CHANGES

This is a major version release with breaking changes to InstrumentDefMessage deserialization. All applications using instrument definition data from the Historical API must be reviewed and potentially updated.

Fixed

  • CRITICAL: Fixed InstrumentDefMessage deserialization to match DBN v2 specification (#4)
    • InstrumentClass now correctly populated (was always returning 0/Unknown)
    • StrikePrice now reading from correct offset 104 (was reading from offset 320)
    • All string fields now reading from correct offsets with correct lengths:
      • RawSymbol: Now 71 bytes at offset 238 (was 22 bytes at offset 194)
      • Asset: Now 11 bytes at offset 335 (was 7 bytes at offset 242)
      • Currency: Now at offset 224 (was at offset 178)
      • SettlCurrency: Now at offset 228 (was at offset 183)
      • SecSubType: Now at offset 232 (was at offset 188)
      • Group: Now at offset 309 (was at offset 216)
      • Exchange: Now at offset 330 (was at offset 237)
      • Cfi: Now at offset 346 (was at offset 249)
      • SecurityType: Now at offset 353 (was at offset 256)
      • UnitOfMeasure: Now at offset 360 (was at offset 263)
      • Underlying: Now at offset 391 (was at offset 294)

Added

  • 13 new fields for multi-leg strategy instruments (spreads, combos):
    • LegPrice (int64): Leg price for multi-leg strategies
    • LegDelta (int64): Leg delta for multi-leg strategies
    • LegInstrumentId (uint32): Leg instrument ID
    • LegRatioPriceNumerator (int32): Leg price ratio numerator
    • LegRatioPriceDenominator (int32): Leg price ratio denominator
    • LegRatioQtyNumerator (int32): Leg quantity ratio numerator
    • LegRatioQtyDenominator (int32): Leg quantity ratio denominator
    • LegUnderlyingId (uint32): Leg underlying instrument ID
    • LegCount (ushort): Number of legs in multi-leg strategies
    • LegIndex (ushort): Leg index (0-based)
    • StrikePriceCurrency (string): Strike price currency
    • LegRawSymbol (string): Raw symbol for leg instrument
    • LegInstrumentClass (InstrumentClass): Instrument class for leg
    • LegSide (Side): Side for leg instrument
  • Added InstrumentClass.Unknown = 0 enum value as safety net for undefined values
  • Added helper methods for reading integers: ReadInt16, ReadUInt16, ReadInt32, ReadUInt32, ReadInt64, ReadUInt64

Changed

  • BREAKING: RawInstrumentId type changed from uint to ulong (correct per DBN spec)
  • BREAKING: All existing InstrumentDefMessage field values will be different (correct values per DBN spec)
  • BREAKING: Removed obsolete TradingReferencePrice field (not in DBN v2 specification)
  • BREAKING: Removed obsolete TradingReferenceDate field (not in DBN v2 specification)

Impact Assessment

Who is affected?

  • Applications querying schema=Definition on any dataset
  • Applications using InstrumentDefMessage fields for filtering or analysis
  • Applications that cache or persist instrument definition data

What breaks?

  • Code filtering by InstrumentClass == 0 will break (previously all instruments returned 0, now returns correct values)
  • Code comparing RawInstrumentId values may need casting from uint to ulong
  • Code using TradingReferencePrice or TradingReferenceDate will fail to compile
  • Cached/persisted instrument data will have mismatched values compared to new version

Migration required:

  • Review all code using InstrumentDefMessage fields
  • Update comparisons for RawInstrumentId to use ulong
  • Remove references to TradingReferencePrice and TradingReferenceDate
  • Invalidate cached instrument definition data
  • See MIGRATION_GUIDE_v4.0.0.md for detailed upgrade instructions

Technical Details

  • Completely rewrote DeserializeInstrumentDefMsg in src/Databento.Client/Models/Record.cs (lines 416-513)
  • Updated InstrumentDefMessage.cs to add 13 new properties for multi-leg strategies
  • All byte offsets now verified against databento-cpp record.hpp specification
  • Added comprehensive inline documentation of correct DBN v2 offsets
  • Zero changes to native DLL (784KB unchanged)
  • Zero changes to other record types (MBO, MBP, OHLCV, etc.)

Verification

  • ✅ Code compiles successfully
  • ✅ All byte offsets verified against DBN v2 specification
  • ✅ InstrumentClass enum now includes Unknown = 0 value
  • ✅ All 13 multi-leg fields added with correct types
  • ⏳ Awaiting verification with real GLBX.MDP3 data

References

[3.0.24-beta] - 2025-11-20

Fixed

  • CRITICAL: Fixed AccessViolationException crash in Historical and Batch APIs when the Databento server returns warning headers (e.g., querying future dates with degraded data quality). The crash was caused by passing NULL pointer to the ILogReceiver parameter in the native C++ wrapper. (#1)
    • Historical API now correctly handles server warnings and continues processing records
    • Batch API now safely handles submission errors without crashing
    • Server warnings now visible on stderr instead of causing silent crashes
    • Example: Querying OHLCV data from 2025-05-01 to 2025-11-15 now receives all 172 records with warning visible, instead of immediate crash

Changed

  • Native Logging: Improved logging consistency across all client types (Historical, Batch, LiveBlocking, LiveThreaded)
    • Log destination changed from stdout to stderr
    • Log format changed from LEVEL: [Component] Message to [Databento LEVEL] [Component] Message
    • All four client wrappers now use consistent StderrLogReceiver implementation
    • DEBUG-level logs now visible for enhanced diagnostics
    • Impact: ~90% of users unaffected (console output unchanged); ~10% may need to update log redirection scripts (see Migration Guide)

Technical Details

  • Added StderrLogReceiver class in src/Databento.Native/src/common_helpers.hpp
  • Updated historical_client_wrapper.cpp: Added log_receiver field, passed to Historical constructor
  • Updated batch_wrapper.cpp: Added log_receiver field (Batch uses Historical client internally)
  • Updated live_blocking_wrapper.cpp: Added log_receiver field, passed to LiveBlocking::Builder
  • Updated live_client_wrapper.cpp: Added log_receiver field, passed to LiveThreaded::Builder
  • Native DLL size unchanged: 784KB
  • Zero API surface changes: Fully backward compatible

Testing

  • All 33 examples pass (100% success rate, up from 32/33)
  • Comprehensive testing completed:
    • Historical API with future dates: ✅ Fixed (172 records received, no crash)
    • Historical API with past dates: ✅ Working (regression test passed)
    • Batch API with invalid symbols: ✅ Fixed (proper exception instead of crash)
    • Live authentication: ✅ Working (new log format visible)
    • Live replay mode: ✅ Working (new log format visible)
    • LiveThreaded streaming: ✅ Working (event callbacks functioning correctly)
    • All metadata and symbology APIs: ✅ Working
  • Zero functionality regressions detected
  • Zero performance regressions detected

Documentation

  • Updated API_REFERENCE.md: Removed crash warnings from GetRangeAsync methods
  • Created RELEASE_NOTES_v3.0.24-beta.md: Comprehensive release documentation
  • Created LOG_FORMAT_VERIFICATION.md: Log format change documentation and migration guide
  • Created ALL_EXAMPLES_REPORT.md: Full test execution report (33/33 examples passing)
  • Created TEST_RESULTS_v3.0.24-beta.md: Detailed test results and verification

[3.0.23-beta] - 2025-11 (Previous Release)

Added

  • Bundle Visual C++ runtime DLLs to fix DllNotFoundException (#2)
  • Add crash warnings to API_REFERENCE.md for Historical GetRange methods

Fixed

  • Fixed DllNotFoundException on systems without Visual C++ redistributables installed

Versioning Scheme

This project uses Semantic Versioning:

  • MAJOR version for incompatible API changes
  • MINOR version for new functionality in a backward compatible manner
  • PATCH version for backward compatible bug fixes

Links