Skip to content

fix(imports): report every invalid cell instead of aborting the row or silently guessing - #206

Merged
ManukMinasyan merged 6 commits into
3.xfrom
fix/import-validation-contract
Aug 28, 2026
Merged

fix(imports): report every invalid cell instead of aborting the row or silently guessing#206
ManukMinasyan merged 6 commits into
3.xfrom
fix/import-validation-contract

Conversation

@ManukMinasyan

@ManukMinasyan ManukMinasyan commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator

The bug

Import casts were doing validation's job.

Filament runs remapData → castData → validateData → fillRecord. Importer::castData() is a bare per-column loop with no error aggregation, so a cast has exactly two ways to signal a bad cell:

  1. Throw. The row dies mid-loop. One message. No other column is ever evaluated.
  2. Return something. The value is now indistinguishable from real data.

ImportColumnConfigurator used both. Choices and lookups threw. Toggle, checkbox, date, date-time, currency and number returned a guess.

Neither is correct, and at that layer there is no third option.

What that looked like

A row with a valid select and garbage in a toggle, a currency field and a date:

reported errors: []
row imported?    YES

  stored toggle   = 'No'          <- 'Q/A' became a definite No
  stored currency = '0'           <- money field silently zero
  stored date     = '2027-09-13'  <- from '13/45/2024'

Zero errors. Row succeeds. Three fields wrong.

And when a choice column did fail, it threw RowImportFailedException, which ImportCsv catches on a separate branch that can only ever carry one message. Every other bad cell in that row stayed hidden until the next upload.

Dates were worse, because the input was valid

3/4/2024  ->  2024-04-03   (day-first regex branch)
3/4/24    ->  2024-03-04   (Carbon::parse fallback, month-first)

The same column flipped day-first and month-first based on whether the year had two digits or four. Not garbage input. Ordinary dates, silently transposed, invisible whenever both numbers are 12 or under.

The fix

A cast is now total and silent. Every input maps to some output and it never throws. A value it cannot honestly produce becomes an UnresolvedValue, and a bail-prefixed RejectsUnresolvedValue rule reports it.

Because rejection now happens in validateData(), Laravel's single validation pass names every bad column at once, merged with the host application's own column errors:

Invalid choice 'Q/A' for Housing Program. Valid choices: Rapid Rehousing, Permanent Supportive
'Q/A' is not a valid value for Ra Eligible. Use true or false (accepted: true, false, 1, 0, yes, no, on, off).
'Q/A' is not a valid amount for Monthly Stipend. Expected format: 1,234.56.

One upload, every problem.

bail sits first so an unresolvable cell reports its own reason once, rather than also tripping the type rule behind it with a message describing the cast's fallback instead of the user's mistake.

Dates and numbers are declared, not guessed

ImportDateFormat (iso, european, american) and ImportNumberFormat (point, comma), both configurable under custom-fields.imports.

The date default is iso. A column declared ISO means the Y-m-d family and nothing else. Day-first, month-first and textual month forms belong to the convention that disambiguates them, so choosing one is a deliberate act rather than a silent guess. Every convention is a superset of ISO, so picking european or american only widens what is read, never changes how a Y-m-d cell is interpreted.

Parsing uses createFromFormat with an explicit format list and no Carbon::parse fallback. Overflow is rejected via DateTimeImmutable::getLastErrors():

31/02/2024  ->  rejected   (used to be 2 March)
2024-02-31  ->  rejected   (used to be 2 March)
2024-13-01  ->  rejected   (used to be 1 Jan 2025)
1/1/24      ->  rejected   (used to be 1 January, year 24)

Worth recording for anyone who hits this later: the ! format prefix does not prevent overflow (!Y-m-d still turns 2024-02-31 into 2 March), and Carbon::hasFormat() does not detect it either (returns true for 2024-02-31). getLastErrors() is the only reliable gate.

Host field types cannot break the contract

configureViaFieldType() wraps every registered importTransformer, so a throw from a field type this package does not own becomes an UnresolvedValue instead of killing the row. The wrapper also prefixes the field name when the transformer's message lacks it, because a field type does not know which field it is configuring and ImportCsv flattens the error bag to message text, dropping attribute names.

CurrencyFieldType is why this is a wrapper and not a convention. It shipped a transformer that ran preg_replace('/[^0-9.-]/','') and turned Q/A into 0.0, and nothing caught it.

Behavior change

Rows carrying values the importer previously swallowed now fail. That is the intent. The failure names the field, preserves the row in failed_import_rows, and is fixed by correcting the cell. A recoverable error replaces silent wrong data.

What does not change:

  • The number default. It stays point, matching how this package has always behaved.
  • Blank cells. Every cast keeps its blank -> null guard.
  • Rows that already failed. Choices and lookups only ever affected failing rows; they now report every error instead of one.

The one visible difference for choices is the exception class reaching ImportCsv: ValidationException rather than RowImportFailedException. Both are caught there. Code catching RowImportFailedException around its own import loop should widen it.

Second behavior change: the date default

This package previously read dates through a day-first regex with a Carbon::parse fallback, so 15/01/2024 parsed and 3/4/2024 meant 3 April. Under the iso default neither is accepted, and the cell is reported rather than guessed at.

If your users are not on Y-m-d, set the convention explicitly:

// config/custom-fields.php
'imports' => ['date_format' => 'european'],   // or 'american'

or CUSTOM_FIELDS_IMPORT_DATE_FORMAT=european.

That is the upgrade step for this release. It is one line, and it replaces an ambiguity that silently transposed day and month on every two-digit-year cell.

Tests

ImportValidationContractTest runs a real Importer over real rows through a fixture importer wired the way the docs tell people to wire one, and asserts on failed_import_rows and stored values rather than on cast internals. It covers multi-error reporting, mixed custom-field and ordinary column errors, message quality for the single-error case, each silent-swallow class, a clean row storing every value, blank optional cells still importing, partial failure leaving good rows imported, and the failed row being preserved for re-upload.

ImportContractConformanceTest iterates every registered field type and asserts the contract holds: never throws, never silently coerces, always names the field, blank still becomes null. That is what stops the next field type reintroducing this, which is how the currency transformer got here.

Two assertions in ImportArchitectureTest changed, both of which encoded the bug: 'invalid-date' becoming null, and an invalid option throwing. Every other assertion in that file passes untouched, including '15/01/2024' -> '2024-01-15' and 'January 15, 2024' -> '2024-01-15', which is the evidence that existing behavior is preserved.

Full suite: 906 passed. PHPStan clean. Pint and Rector clean. Type coverage is unchanged at 99.4%, which is the pre-existing value on 3.x.

Copilot AI lite review requested due to automatic review settings August 28, 2026 10:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@ManukMinasyan
ManukMinasyan merged commit be43d73 into 3.x Aug 28, 2026
5 checks passed
@ManukMinasyan
ManukMinasyan deleted the fix/import-validation-contract branch August 28, 2026 11:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants