Skip to content

fix: replace hardcoded 120s step percentiles with real FileStepHistory timing - #128

Closed
Akash29g wants to merge 3 commits into
mainfrom
fix/step-processing-percentile
Closed

fix: replace hardcoded 120s step percentiles with real FileStepHistory timing#128
Akash29g wants to merge 3 commits into
mainfrom
fix/step-processing-percentile

Conversation

@Akash29g

Copy link
Copy Markdown
Owner

What

  • AnalyticsService.GetStepPercentilesAsync was returning hardcoded 120s for all
    steps — a leftover from seeded data where timestamps were artificially 120s apart
  • ExtractionWorker only recorded one FileStepHistory entry (Extract) covering
    the entire process with no per-phase timing

Changes

DocAnalytics.Service/Analytics/AnalyticsService.cs

  • Replaced hardcoded StepPercentileDto list with a real DB query
  • Navigates through FileRecord (tenant/site global query filter preserved)
    FileStepHistory where Status == "Success" and both timestamps present
  • Computes P50/P90/P99 via linear interpolation in-memory
  • Returns empty list when no completed steps exist (shows "No completed steps yet")
  • Fixed return type from IReadOnlyList<>List<> to match IAnalyticsService

DocAnalytics.Api/BackgroundServices/ExtractionWorker.cs

  • Added timestamp captures at 3 phase boundaries: validateStart, extractStart,
    loadStart
  • Replaced single Extract step history entry with 3 separate timed entries:
    • Validate — GuardDuty poll + magic-byte check duration
    • Extract — Bedrock ExtractAsync call duration
    • Load — DB persistence duration
  • Existing Extract / Processing marker (for realtime SignalR updates) unchanged

Why

Processing Time by Step table was showing 120s for all steps even after seeded
data was cleared. Real uploads now populate with actual measured durations.

Testing

  • Cleared seeded file_step_history rows from RDS
  • Uploaded real PDFs → Validate / Extract / Load now show real timings
  • Empty state ("No completed steps yet") shows correctly before first upload

@Akash29g
Akash29g requested a review from sumitgupta-cse July 24, 2026 07:04
@Akash29g
Akash29g requested a review from g9shubh as a code owner July 24, 2026 07:04
@Akash29g Akash29g closed this Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant