Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,7 @@ The roadmap for TextBlaster is divided into 6 phases, with subtasks to complete.
3. **Standardized Logging and Tracing:** Fully integrate the `tracing` crate, replacing all `println!` calls. Structure logs as JSON and add `doc_id` to tracing spans to correlate all messages for a specific document. Allow log level configuration via CLI.
4. **Refined Error Handling & Messaging:** Improve the clarity of all user-facing error messages. Ensure that when a task fails in a worker, the propagated error clearly identifies the worker ID, the failing step, and the root cause.
5. **Comprehensive Documentation:** Create a `docs/` folder with Markdown guides on architecture, configuration, and tutorials. Add `CONTRIBUTING.md` for new developers and ensure all public functions and structs have detailed doc comments (`///`).
6. **Configuration Validation:** Implement a `--validate-config` flag and an automatic startup check to validate the `pipeline_config.yaml`. This check will catch syntax errors, schema violations, and logical issues (e.g., a filter running before its dependency).
6. **Configuration Validation:** Implement a `--validate-config` flag and an automatic startup check to validate the `pipeline_config.yaml`. This check will catch syntax errors, schema violations, and logical issues (e.g., a filter running before its dependency). - *Done*

### **Phase 2: Core Pipeline Enhancement & Usability**
**Theme:** Improve the user experience and expand the core capabilities of the processing pipeline, making it more flexible and powerful for common tasks.
Expand Down
28 changes: 28 additions & 0 deletions src/bin/worker.rs
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,10 @@ struct Args {
/// Optional: Port for the Prometheus metrics HTTP endpoint
#[arg(long)]
metrics_port: Option<u16>,

/// Validate the pipeline configuration and exit
#[arg(long)]
validate_config: bool,
}

// --- Prometheus Metrics (now imported from TextBlaster::utils::prometheus_metrics) ---
Expand Down Expand Up @@ -377,6 +381,30 @@ async fn process_tasks(
async fn main() -> Result<()> {
let args = Args::parse();

if args.validate_config {
match load_pipeline_config(&args.pipeline_config) {
Ok(_) => {
// Note: Tracing might not be initialized here.
// Consider simple println for this specific validation output.
println!(
"Configuration '{}' is valid.",
args.pipeline_config.display()
);
std::process::exit(0);
}
Err(e) => {
// Note: Tracing might not be initialized here.
// Consider simple eprintln for this specific validation output.
eprintln!(
"Configuration '{}' is invalid: {}",
args.pipeline_config.display(),
e
);
std::process::exit(1);
}
}
}

// Initialize tracing subscriber
let env_filter = EnvFilter::try_from_default_env().unwrap_or_else(|_| EnvFilter::new("info")); // Default to info if RUST_LOG is not set

Expand Down
Loading