Converter robustness: 7 findings from exploratory edge-case testing

[radupana]

Context

Ran exploratory testing against the Strong and Hevy converters using 22 synthetic edge-case CSV files covering: empty files, header-only, single rows, extreme values, unicode/emoji, garbage data, missing fields, whitespace, BOM, CRLF, quoted commas, newlines in notes, extra columns, all set types, bodyweight/distance exercises, many exercises, various date/duration formats, lbs-only Hevy exports, reversed timestamps, format auto-detection, and format mismatches.

Good news: the converters are solid overall. 18 of 22 tests passed cleanly. Unicode, BOM, CRLF, whitespace, quoted fields, newlines in notes, extra columns, missing fields, format detection, and format mismatches all work correctly.

7 findings below, 3 of which are bugs that cause data loss.

Test fixtures are in tools/converters/src/__fixtures__/exploratory/.

---

Finding #1 (Enhancement): Strong set types are not extracted from Notes

Severity: Low — data loss of set type metadata

Strong exports encode set type in the Notes column (e.g., "warm up", "drop set", "failure"). The converter treats these as plain text notes and does not produce type: "warmup", type: "dropset", or toFailure: true on the set.

Repro: strong-all-set-types.csv — all 4 rows produce sets with no type field.

Expected: Sets with notes matching known set-type strings should have the corresponding type field set.

---

Finding #2 (Minor): Different date string formats create separate workouts

Severity: Low — only affects contrived test data

When rows in the same file use different date format strings that parse to the same instant (e.g., 2024-01-15 10:30:00 vs 2024-01-15T10:30:00Z), they produce separate workouts because workout grouping uses the parsed date string, not a normalized representation.

Repro: strong-date-formats.csv — 8 rows that should arguably be 1 workout become 6 workouts.

Expected: Dates should be normalized before grouping.

---

Finding #3 (Bug): Bodyweight + distance exercises fail schema validation — entire workout lost

Severity: High — causes complete data loss for workouts containing cardio

When a Strong export has exercises with distance > 0 (e.g., Running with 5000m), the schema requires a distanceUnit field. The Strong converter never sets distanceUnit because Strong CSV has no distance unit column. This causes schema validation to fail, and the entire workout is rejected — even if it contains other valid strength exercises.

Repro: strong-bodyweight-distance.csv — workout with Running + Pull Ups + Planks produces 0 workouts.

Error: must have required property 'distanceUnit'; must match "then" schema

Fix: When distance > 0, infer a default distanceUnit (likely m for Strong, since Strong uses meters).

---

Finding #4 (Bug): One invalid set kills the entire workout

Severity: High — causes disproportionate data loss

When a single row has an invalid value (e.g., negative weight, Infinity string), schema validation rejects the entire workout, not just that set. A workout with 20 good sets and 1 garbage row loses all 20 sets.

Repro: strong-extreme-values.csv — 5 rows including weight: -5 and weight: Infinity → 0 workouts produced. The 3 valid rows are lost.

Error: /exercises/0/sets/2/weight: must be number (for Infinity string), /exercises/1/sets/1: must have required property 'distanceUnit'

Fix: Filter out invalid sets before validation rather than rejecting the whole workout. Produce warnings for each dropped set.

---

Finding #5 (Minor): Scientific notation and Infinity handling

Severity: Low — cosmetic / theoretical

• Infinity as a weight string becomes "Infinity" (string, not number) → correctly fails validation
• 1e10 (scientific notation) is parsed as 10000000000 — a valid number that passes through
• -5 (negative weight) passes parsing but fails schema validation

Expected: Numeric parsing could reject Infinity, -Infinity, NaN, and optionally warn on implausible values (e.g., weight > 1000kg).

---

Finding #6 (Minor): Empty sets {} pass validation

Severity: Low — spec question

When Hevy has reps=0, weight_kg=0, weight_lbs=0, the converter produces an empty set object {}. This passes schema validation since set objects apparently have no required fields.

Repro: hevy-extreme-values.csv — first set produces {}.

Question: Should the schema require at least one of reps, weight, duration, or distance on a set? Or should the converter filter out empty sets?

---

Finding #7 (Bug): Reversed start/end time produces no duration and no warning

Severity: Medium — silent data issue

When a Hevy export has start_time > end_time (reversed timestamps), the converter silently drops the duration field with no warning. Same when start_time == end_time (0 duration).

Repro: hevy-reversed-time.csv:

• Reversed (start=12:00, end=10:00) → no durationSeconds, no warning
• Same time (start=10:00, end=10:00) → no durationSeconds, no warning

Expected: Negative duration should produce a warning. Zero duration could be valid (quick workout) but might also warrant a warning.

---

Implementation Plan

Phase 1: Critical data-loss bugs (Findings #3, #4)

These should be fixed first as they cause complete workout loss.

1a. Set-level validation with graceful degradation (Finding #4)

File: tools/converters/src/transform/transformer.ts

• Before passing the workout to validateWorkoutLog(), validate each set individually
• Filter out sets that have invalid values (non-numeric weight, negative weight, Infinity, NaN)
• If an exercise has 0 valid sets remaining after filtering, drop the exercise (with warning)
• If a workout has 0 valid exercises remaining after filtering, drop the workout (with warning)
• Add a sanitizeSet() function that:
  • Rejects negative weight/reps/distance/duration
  • Rejects non-finite numbers (Infinity, NaN)
  • Optionally warns on implausible values (weight > 1000, reps > 500)

1b. Infer distanceUnit for Strong (Finding #3)

File: tools/converters/src/parsers/strong.ts

• When distance > 0, set distanceUnit: "m" (Strong uses meters)
• Add this to the Strong parser's row transformation
• This is a one-line fix in the Strong parser

Phase 2: Silent data issues (Findings #7, #2)

2a. Warn on reversed/zero duration (Finding #7)

File: tools/converters/src/transform/duration.ts

• When computing duration from start_time|end_time pipe format:
  • If end < start: produce a parse warning with "start_time is after end_time", set duration to undefined
  • If end == start: produce a parse warning with "start_time equals end_time (zero duration)", set duration to 0 (or undefined)

2b. Normalize dates before workout grouping (Finding #2)

File: tools/converters/src/transform/transformer.ts

• After parsing dates, normalize all dates to ISO 8601 with Z timezone before using them as grouping keys
• This ensures 2024-01-15 10:30:00 and 2024-01-15T10:30:00Z group into the same workout

Phase 3: Enhancements (Findings #1, #5, #6)

3a. Extract set types from Strong Notes column (Finding #1)

File: tools/converters/src/parsers/strong.ts

• After parsing a Strong row, check if the exerciseNotes field matches a known set-type pattern
• Patterns: warm up, warm-up, warmup → type: "warmup"; drop set, dropset → type: "dropset"; failure, to failure → toFailure: true
• If matched, set the type on the set and optionally clear the note (or keep it)
• Need to verify: does Strong actually put set type in the Notes column, or in a separate field that we're not seeing?

3b. Filter empty sets (Finding #6)

File: tools/converters/src/transform/transformer.ts

• After building a set object, check if it has any meaningful data (reps, weight, duration, distance)
• If all are absent/zero, either drop the set or keep it with a warning
• This is related to the sanitizeSet() function from Phase 1

3c. Sanitize implausible numeric values (Finding #5)

File: tools/converters/src/transform/transformer.ts (in the sanitizeSet() function)

• Reject Infinity, -Infinity, NaN at parse time (before they become strings)
• Optionally warn on values that are technically valid but implausible

---

Test Coverage

The exploratory test fixtures in tools/converters/src/__fixtures__/exploratory/ can be promoted to proper unit tests for regression coverage. Key files:

• strong-bodyweight-distance.csv → regression test for Finding Write example workout log files #3
• strong-extreme-values.csv → regression test for Finding Set up TypeScript SDK package #4
• strong-garbage-values.csv → regression test for Finding Design program and template data model #5
• hevy-extreme-values.csv → regression test for Finding Create JSON Schema for programs and templates #6
• hevy-reversed-time.csv → regression test for Finding Create Kotlin SDK for Featherweight integration #7
• strong-all-set-types.csv → regression test for Finding Design core workout log data model #1
• strong-date-formats.csv → regression test for Finding Create JSON Schema for workout logs #2