Skip to content

docs: correct Nil processor description, errors.As guidance, and file structure #68

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
MasterOfBinary wants to merge 3 commits into
base: master
Choose a base branch
from
Open

docs: correct Nil processor description, errors.As guidance, and file structure #68

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
05d73f6
docs: correct Nil processor description and errors.As guidance
MasterOfBinary May 29, 2026
a13e76b
docs: clean up doc.go example comments
MasterOfBinary May 29, 2026
849789b
docs: reconcile doc corrections with single-use API
MasterOfBinary May 29, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,11 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo
- Include practical examples in documentation

## File Structure
- `/batch`: Core batch processing functionality, includes the main Batch type, configuration, errors, and helper functions
- `/processor`: Different processors for data manipulation (Transform, Filter, Channel, Error, Nil)
- `/source`: Data sources for batch processing (Channel, Error, Nil)
- `/doc.go`: Package-level documentation
- `/example_test.go`: Top-level usage examples
- `/batch`: Core batch processing package. Includes `batch.go` (the main `Batch` type and pipeline), `config.go` (Config interface, ConstantConfig, DynamicConfig, BufferConfig), `errors.go` (SourceError, ProcessorError), `constants.go`, `helpers.go` (IgnoreErrors, CollectErrors, RunBatchAndWait, ExecuteBatches), and `doc.go`. Accompanied by unit tests (`*_test.go`) and several runnable `example_*_test.go` files.
- `/processor`: Processor implementations — `transform.go`, `filter.go`, `channel.go`, `error.go`, `nil.go` — plus `doc.go` and tests.
- `/source`: Source implementations — `channel.go`, `error.go`, `nil.go` — plus `doc.go` and tests.
- `/doc.go`: Root package-level documentation.
- `/example_test.go`: Top-level usage examples.

## Key Concepts
- **Batch**: Main type that orchestrates the batch processing pipeline
Expand All @@ -57,7 +57,7 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo
- **Filter**: Filters items based on a predicate function
- **Channel**: Writes item data to an output channel
- **Error**: Simulates processor errors (for testing)
- **Nil**: Passes items through unchanged (for benchmarking)
- **Nil**: Sleeps for a configurable `Duration` without modifying item data (can mark items cancelled via `MarkCancelled`); useful for simulating slow processing and testing timing behavior

### Sources
- **Channel**: Uses Go channels as data sources
Expand All @@ -75,4 +75,4 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo
- Processors should respect context cancellation
- Items with errors are tracked individually through the Error field
- Batch processing continues despite individual item errors
- Use `errors.As` to check error types (SourceError, ProcessorError)
- Use `errors.As` to check error types (SourceError, ProcessorError). The engine wraps errors as pointers (`&SourceError{...}`, `&ProcessorError{...}`), so `errors.As` must target the pointer type — e.g. `var se *batch.SourceError; errors.As(err, &se)` (equivalently `errors.As(err, new(*batch.SourceError))`). A value-target such as `var se batch.SourceError; errors.As(err, &se)` will not match.

@gemini-code-assist gemini-code-assist Bot May 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The parenthetical (equivalently errors.As(err, new(*batch.SourceError))) is technically correct but highly non-idiomatic in Go.

In Go, the standard and idiomatic way to use errors.As with a pointer type is:

var se *batch.SourceError
errors.As(err, &se)

Using new(*batch.SourceError) is rarely seen and can confuse readers because it returns a double pointer (**batch.SourceError), requiring extra dereferencing to access the error fields.

Additionally, the phrase "The engine wraps errors as pointers" could be more precisely stated as "The engine returns errors as pointers" or "The engine instantiates errors as pointers", since "wrapping" in Go errors typically refers to the Unwrap chain rather than pointer vs. value types.

Consider simplifying and refining this sentence:

- Use `errors.As` to check error types (SourceError, ProcessorError). The engine returns errors as pointers (`*SourceError`, `*ProcessorError`), so `errors.As` must target the pointer type — e.g., `var se *batch.SourceError; errors.As(err, &se)`. A value-target such as `var se batch.SourceError; errors.As(err, &se)` will not match.

14 changes: 7 additions & 7 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,11 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo
- Include practical examples in documentation

## File Structure
- `/batch`: Core batch processing functionality, includes the main Batch type, configuration, errors, and helper functions
- `/processor`: Different processors for data manipulation (Transform, Filter, Channel, Error, Nil)
- `/source`: Data sources for batch processing (Channel, Error, Nil)
- `/doc.go`: Package-level documentation
- `/example_test.go`: Top-level usage examples
- `/batch`: Core batch processing package. Includes `batch.go` (the main `Batch` type and pipeline), `config.go` (Config interface, ConstantConfig, DynamicConfig, BufferConfig), `errors.go` (SourceError, ProcessorError), `constants.go`, `helpers.go` (IgnoreErrors, CollectErrors, RunBatchAndWait, ExecuteBatches), and `doc.go`. Accompanied by unit tests (`*_test.go`) and several runnable `example_*_test.go` files.
- `/processor`: Processor implementations — `transform.go`, `filter.go`, `channel.go`, `error.go`, `nil.go` — plus `doc.go` and tests.
- `/source`: Source implementations — `channel.go`, `error.go`, `nil.go` — plus `doc.go` and tests.
- `/doc.go`: Root package-level documentation.
- `/example_test.go`: Top-level usage examples.

## Key Concepts
- **Batch**: Main type that orchestrates the batch processing pipeline
Expand All @@ -57,7 +57,7 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo
- **Filter**: Filters items based on a predicate function
- **Channel**: Writes item data to an output channel
- **Error**: Simulates processor errors (for testing)
- **Nil**: Passes items through unchanged (for benchmarking)
- **Nil**: Sleeps for a configurable `Duration` without modifying item data (can mark items cancelled via `MarkCancelled`); useful for simulating slow processing and testing timing behavior

### Sources
- **Channel**: Uses Go channels as data sources
Expand All @@ -75,4 +75,4 @@ GoBatch is a Go library for batch data processing. It provides infrastructure fo
- Processors should respect context cancellation
- Items with errors are tracked individually through the Error field
- Batch processing continues despite individual item errors
- Use `errors.As` to check error types (SourceError, ProcessorError)
- Use `errors.As` to check error types (SourceError, ProcessorError). The engine wraps errors as pointers (`&SourceError{...}`, `&ProcessorError{...}`), so `errors.As` must target the pointer type — e.g. `var se *batch.SourceError; errors.As(err, &se)` (equivalently `errors.As(err, new(*batch.SourceError))`). A value-target such as `var se batch.SourceError; errors.As(err, &se)` will not match.

@gemini-code-assist gemini-code-assist Bot May 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The parenthetical (equivalently errors.As(err, new(*batch.SourceError))) is technically correct but highly non-idiomatic in Go.

In Go, the standard and idiomatic way to use errors.As with a pointer type is:

var se *batch.SourceError
errors.As(err, &se)

Using new(*batch.SourceError) is rarely seen and can confuse readers because it returns a double pointer (**batch.SourceError), requiring extra dereferencing to access the error fields.

Additionally, the phrase "The engine wraps errors as pointers" could be more precisely stated as "The engine returns errors as pointers" or "The engine instantiates errors as pointers", since "wrapping" in Go errors typically refers to the Unwrap chain rather than pointer vs. value types.

Consider simplifying and refining this sentence:

- Use `errors.As` to check error types (SourceError, ProcessorError). The engine returns errors as pointers (`*SourceError`, `*ProcessorError`), so `errors.As` must target the pointer type — e.g., `var se *batch.SourceError; errors.As(err, &se)`. A value-target such as `var se batch.SourceError; errors.As(err, &se)` will not match.

2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ go get github.com/MasterOfBinary/gobatch
- **Filter**: Filters items based on a predicate function.
- **Transform**: Transforms item data with a custom function.
- **Error**: Simulates processor errors for testing.
- **Nil**: Passes items through unchanged for benchmarking.
- **Nil**: Sleeps for a configurable `Duration` without modifying item data; useful for simulating slow processing and testing timing behavior.
- **Channel**: Writes item data to an output channel.

### Built-in Sources
Expand Down
4 changes: 1 addition & 3 deletions doc.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,7 @@
// batch.IgnoreErrors(errs)
// <-b.Done()
//
// Output:
//
// hello
// The Transform func above prints each item, so this pipeline writes "hello".
//
// See the README.md for an overview of how these pieces fit together.
package gobatch
12 changes: 7 additions & 5 deletions source/doc.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,15 @@
// src := &Channel[int]{Input: input}
// out, errs := src.Read(context.Background())
// for item := range out {
// fmt.Println(item)
// fmt.Println(item) // prints 1 then 2
// }
// for range errs {
// }
//
// Output:
//
// 1
// 2
// Caution: draining out fully and then errs sequentially, as shown above, is
// only safe because Channel emits no errors. In general, data and errors should
// be consumed concurrently — the Batch engine reads both channels at the same
// time — otherwise a source that emits an error while data is still pending can
// block. Prefer separate goroutines (or a select) when a source may report
// errors.
package source
Loading