Skip to content

Get-BulkRowsCopiedCount - Read the rows copied counter as Int64 - #10677

Open
andreasjordan wants to merge 1 commit into
developmentfrom
fix-bulkrowscopied-int64
Open

Get-BulkRowsCopiedCount - Read the rows copied counter as Int64#10677
andreasjordan wants to merge 1 commit into
developmentfrom
fix-bulkrowscopied-int64

Conversation

@andreasjordan

Copy link
Copy Markdown
Collaborator

Fixes #10675

Problem

The report: the verbose rows-copied total of Copy-DbaDbTableData can over-report against the destination's real row count on the final batch, while the copied data itself is correct.

What reproduced and what did not

On the current stack (dbatools.library with Microsoft.Data.SqlClient 6.1.5, SQL Server 2022 -> 2025) a multi-batch copy shows correct live counters and a correct final total, with the batch size aligned to the row count and not - the reported symptom does not reproduce below [int32]::MaxValue rows. @niphlod is right that the wrapping 4-byte counter itself is history: SqlBulkCopy._rowsCopied is an Int64 in current Microsoft.Data.SqlClient, so the counter never wraps any more.

But the machinery the issue points at does over-report the final total, exactly once the copy exceeds [int32]::MaxValue rows:

  1. Get-BulkRowsCopiedCount reads _rowsCopied by reflection and casts it to [int]. Above 2,147,483,647 rows that cast throws.
  2. The catch returns -1, the helper's failure sentinel.
  3. The three callers feed that sentinel into Get-AdjustedTotalRowsCopied, whose integer-wrap branch takes a negative value for a wrapped counter and adds ~1.3 billion phantom rows.

Demonstrated by setting the field via reflection: a 3,000,000,000-row copy reports 4,294,967,295 rows copied while the destination holds the correct 3,000,000,000. The data is right, the reported total is not - the shape the issue describes. Ironically the over-report comes from the workaround for the old 4-byte counter, on a library whose counter no longer wraps.

What changed

  • Get-BulkRowsCopiedCount returns the counter as [long] ([OutputType([long])]); the value passes through unharmed on both the Int64 field of current SqlClient and the Int32 field of the legacy library.
  • The three callers with the final-adjustment pattern - Copy-DbaDbTableData, Import-DbaCsv, Import-DbaParquet - skip the final adjustment when the helper signals failure with -1, keeping the running total from the copy notifications instead of adding a bogus wrap correction.

What deliberately did not change

  • Get-AdjustedTotalRowsCopied keeps its wrap logic untouched: it is still correct for genuinely wrapped (negative) values that an old library delivers through the SqlRowsCopied event.
  • Write-DbaDbTableData uses only the event-side adjustment and never calls Get-BulkRowsCopiedCount, so it has no sentinel path and stays as it is.

Tests

New tests/Get-BulkRowsCopiedCount.Tests.ps1 (unit, InModuleScope, no instance needed): the helper returns the counter below and above [int32]::MaxValue, and the callers' final-adjustment pattern stays correct above it. Red on development (Expected 3000000000, but got -1.), green with the fix.

Full runs of the four affected test files through the testing-dbatools harness (SQL 2022 -> SQL 2025), both editions: 79/79 green on pwsh and on 5.1.

Copying two billion real rows is not something CI or the lab can afford, which is why the regression test sets the counter field by reflection instead - the same access path the helper itself uses.

created by Claude and reviewed by Andreas Jordan

馃 Generated with Claude Code

The _rowsCopied field of SqlBulkCopy is an Int64 in current
Microsoft.Data.SqlClient (the 4 byte counter of the legacy library, which
Get-AdjustedTotalRowsCopied works around, was fixed upstream). The cast to
[int] therefore throws above [int32]::MaxValue rows, the catch returned the
-1 failure sentinel, and the callers fed that sentinel into the wrap
adjustment as if it were a wrapped counter: a copy of 3 billion rows
reported 4294967295 rows copied while the destination held the correct
count. The callers now skip the final adjustment when the helper signals
failure, so the running total from the notifications stands instead of a
number that is off by billions. The wrap adjustment itself stays, it is
still correct for genuinely wrapped values from an old library.

Fixes #10675

(do Copy-DbaDbTableData, Import-DbaCsv, Import-DbaParquet, Get-BulkRowsCopiedCount)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Copy-DbaDbTableData verbose progress counter over-reports rows copied on final batch

1 participant