Skip to content

Latest commit

 

History

History
515 lines (423 loc) · 22.7 KB

File metadata and controls

515 lines (423 loc) · 22.7 KB

Agent Log - Electronics Product Matching Challenge

Updated: 2026-03-07 | Duration: 8h | Strategy: 5-pass modular system (no LLM)


Challenge Overview

Objective: Match 91 source products to competitor products from 6 Austrian retailers
Scoring: 100 pts (50 visible + 50 scraped) - Recall (30) > Precision (10) > Coverage (10)
Web App: hackathon-production-49ca.up.railway.app

Data: TV & Audio (17 source, ~560 target → ~337 actual TVs)

  • Visible: Amazon AT | Hidden: Expert AT, Cyberport AT, electronic4you.at, E-Tec
  • EAN Coverage: 5/17 (29.4%) - requires fuzzy matching
  • Brands: Samsung, LG, TCL, Sharp, PEAQ, XIAOMI, CHIQ, Cello

Matching Strategy (5 Passes - No LLM)

Pass Method Criteria Confidence Expected
1 EAN Exact source.ean == target.ean + brand ≥0.6 95-100% 4-5
2 Model Number Substring/90% fuzzy + brand ≥0.6 80-95% 6-8
3 Strict Attr Brand≥0.8 + Size±2" + Display + Resolution 70-85% 3-5
4 Weighted Score 25%Brand + 20%Size + 15%Display + 15%Res + 10%Model + 10%Price + 5%Name ≥0.60 60-75% 1-2
5 Aggressive Brand≥0.6 + Size±2" (orphans) OR score≥0.5 40-60% 0-1
TOTAL Avg 70% 14-21

Match Policy: Keep all valid matches per source (maximize recall), allow target reuse across sources


Dynamic Attribute Extraction Strategy

Approach: Extract ALL specifications dynamically - work directly with German spec keys

Data Analysis (from actual dataset):

  • Source products: 152 unique spec keys, most common: Hersteller (14), WLAN (12), Displaytyp (11)
  • Target products: 534 unique spec keys, most common: Marke (289), Modellname (257), Konnektivitätstechnologie (247)
  • Only 62.6% of target products have specs (351/561)

Strategy:

  1. Extract critical attributes (brand, size, display, resolution, model, ean, price) with robust fallback logic
  2. Store ALL other specs as-is (no normalization/translation needed)
  3. Store raw specs for flexibility
  4. Build spec index showing coverage across products
  5. Allow Pass 4 weighted matcher to use ANY spec with configurable weights

Critical Attribute Extraction

Brand:

  • Priority: specs['Hersteller']specs['Marke'] → first word matching KNOWN_BRANDS → first word in name
  • Normalize: "Samsung Electronics GmbH" → "SAMSUNG"

Model Number:

  • Priority: specs['Hersteller Modellnummer']specs['Herstellernummer']specs['Modellnummer'] → regex from name
  • Clean: Remove .AEU, /XXN, FUXXN suffixes, uppercase
  • Example: 32LQ63806LC.AEU32LQ63806LC

Screen Size:

  • Check: specs['Bildschirmdiagonale in cm, Zoll']specs['Bildschirmgröße'] → name patterns
  • Patterns: 32 Zoll, 32", 81 cm (÷2.54)
  • Returns: int in inches

Display Type:

  • Check: specs['Displaytyp']specs['Produkttyp']specs['Bildschirmtechnologie'] → name
  • Normalize: LED, QLED, OLED, LCD, MINI_LED
  • Compatibility: QLED⊃LED, Mini LED⊃LED, OLED distinct

Resolution:

  • Check: specs['Bildqualität']specs['Auflösung']specs['Bildschirmauflösung'] → name
  • Normalize: UHD_4K (4K/3840x2160), FULL_HD (1920x1080), HD_READY (1366x768/720p)

EAN:

  • Priority: product['ean']specs['EAN-Code']specs['GTIN']

Price:

  • product['price_eur']

Validation

Note: No pre-filtering or post-validation initially - implement if needed based on results


Data Analysis Summary

Source Products (17):

  • EANs: 5/17 (29.4%) - low coverage requires fuzzy matching
  • Specs: 152 unique keys, all products have specs
  • Top keys: Hersteller (14), WLAN (12), Displaytyp (11), Energieeffizienzklasse (10)
  • Price range: €149-€669 | Sizes: 24"-65"
  • Brands: Samsung, Sharp, LG, XIAOMI, PEAQ, TCL, CHIQ

Target Products (561):

  • Specs: 534 unique keys, only 351/561 (62.6%) have specs
  • Top keys: Marke (289), Modellname (257), Konnektivitätstechnologie (247), Hersteller (228)
  • Mix: TVs + accessories (headphones, cables, adapters) - no pre-filtering
  • Retailer: Amazon AT (visible)
  • Data quality: Many missing EAN/specs/prices

Expected Results: 14-21 matches (82-100% coverage) at ~70% avg confidence


System Architecture (Modular - Matching Only)

matching_system/
├── config.py                  # Paths, thresholds, DEFAULT_WEIGHTS, known brands, display compatibility
├── utils.py                   # Fuzzy matching (rapidfuzz), text normalization
├── data_loader.py             # Load/validate JSONs, data summaries
├── attribute_extractor.py     # Dynamic extraction of ALL specs + critical attributes
├── matchers/
│   ├── base_matcher.py        # Abstract class: match() → [(source_ref, target_ref, conf, pass)]
│   ├── ean_matcher.py         # Pass 1: EAN exact
│   ├── model_matcher.py       # Pass 2: Model fuzzy/substring
│   ├── attribute_matcher.py   # Pass 3: Strict attributes
│   ├── weighted_matcher.py    # Pass 4: Weighted score ≥0.60
│   └── aggressive_matcher.py  # Pass 5: Fallback for orphans
├── pipeline.py                # Orchestrate passes, merge, dedupe, format
├── submission.py              # Generate submission JSON
└── main.py                    # CLI entry point

Note: No scrapers - scraping is handled separately by team

Pipeline Flow

Load → Extract Attrs → Pass 1-5 → Validate → Dedupe → Format → Submit

Weighted Score Formula (Pass 4 - Fully Configurable)

# Core attributes with default weights:
DEFAULT_WEIGHTS = {
    'brand': 0.25,
    'size': 0.20,
    'display_type': 0.15,
    'resolution': 0.15,
    'model_number': 0.10,
    'price': 0.10,
    'name': 0.05
    # Users can add ANY spec with custom weights via CLI
}

# Scoring logic:
score = sum(similarity(source[spec], target[spec]) * weight 
            for spec, weight in weights.items() 
            if spec in source and spec in target)

# Price scoring: ±50%=1.0, ±100%=0.5, >100%=0.0 (neutral if missing)
# Threshold: ≥0.60 for match (configurable via CLI)

Implementation Checklist

Phase 1: Foundation (DETAILED - 1-2h)

Step 1.1: config.py ✅ COMPLETE

# Contents:
- Paths: SOURCE_DATA, TARGET_DATA, OUTPUT_DIR
- Pass thresholds: EAN_BRAND_VALIDATION, MODEL_FUZZY_THRESHOLD, etc.
- DEFAULT_WEIGHTS dict (7 core, extensible for any spec via CLI)
- WEIGHTED_SCORE_THRESHOLD, AGGRESSIVE_SCORE_THRESHOLD
- KNOWN_BRANDS list (15 brands)
- DISPLAY_COMPATIBILITY mapping
- PRICE_RATIO_EXACT, PRICE_RATIO_ACCEPTABLE
- NAME_FUZZY_THRESHOLD

Status: ✅ Implemented and tested

  • Weights sum to 1.00
  • All paths verified
  • Focused on matching only (no scraping config)
  • Works with German spec keys directly (no translation)

Step 1.2: utils.py (NEXT)

# Functions (8 total - simplified from 10):
1. normalize_text(text) → str  # Uppercase, German chars (ä→AE, ö→OE, ü→UE), trim
2. clean_model_number(model) → str  # Remove .AEU, XXN, FUXXN suffixes
3. fuzzy_match_text(t1, t2) → float  # 0-1 using rapidfuzz.fuzz.ratio
4. fuzzy_match_brand(b1, b2) → float  # 0-1 using token_sort_ratio
5. substring_match(t1, t2) → bool
6. calculate_size_score(s1, s2, max_diff=10) → float  # Linear decay
7. calculate_price_score(p1, p2) → float or None  # ±50%/±100% thresholds
8. extract_size_from_text(text) → int or None  # Patterns: 32", 32 Zoll, 81cm

Note: Removed normalize_spec_key() - working with German keys directly
Note: Removed extract_model_from_text() - will be in attribute_extractor

Step 1.3: data_loader.py

# Functions (5 total):
1. load_json(filepath, encoding='utf-8') → list  # Error handling
2. load_source_products(filepath=None) → list  # Defaults to config.SOURCE_DATA
3. load_target_products(filepath=None) → list  # Validate required fields
4. validate_product(product, required_fields) → bool
5. get_data_summary(products) → dict  # Stats: count, with_ean, with_specs, etc.

Test:

sources = load_source_products()
targets = load_target_products()
print(f"Sources: {len(sources)}, Targets: {len(targets)}")
print(get_data_summary(sources))

Step 1.4: attribute_extractor.py (MOST COMPLEX)

# Functions (8 total - simplified):

# Critical attribute extractors (with fallback logic):
1. extract_brand(product) → str or None
2. extract_model_number(product) → str or None  # Includes regex extraction
3. extract_screen_size(product) → int or None
4. extract_display_type(product) → str or None
5. extract_resolution(product) → str or None

# Main extraction function:
6. extract_all_specs(product) → dict
   # Returns:
   {
       # Critical fields (extracted with fallbacks):
       'brand': str, 'model_number': str, 'ean': str,
       'size': int, 'display_type': str, 'resolution': str,
       'price': float, 'name': str,
       
       # ALL other specs stored as-is (German keys):
       # All keys from product['specifications'] copied directly
       
       # Metadata:
       '_raw_specs': dict,  # Original specs for debugging
       '_source_type': 'source' or 'target'
   }

7. extract_all_specs_batch(products) → list[dict]
   # Extract specs for all products

8. display_types_compatible(d1, d2) → bool
   # Check compatibility using config.DISPLAY_COMPATIBILITY

Note: No build_spec_index() needed initially - can add later if needed

Test Script:

from data_loader import *
from attribute_extractor import *

# Load data
sources = load_source_products()
targets = load_target_products()

print("=== Testing Extraction on 3 Source Products ===")
for i, product in enumerate(sources[:3], 1):
    specs = extract_all_specs(product)
    print(f"\nProduct {i}: {product['name'][:50]}")
    print(f"  Brand: {specs['brand']}")
    print(f"  Model: {specs['model_number']}")
    print(f"  Size: {specs['size']}\"")
    print(f"  Display: {specs['display_type']}")
    print(f"  Resolution: {specs['resolution']}")
    print(f"  EAN: {specs['ean']}")
    print(f"  Total specs: {len([k for k in specs if not k.startswith('_')])}")

print("\n=== Testing Display Compatibility ===")
print(f"LED vs QLED: {display_types_compatible('LED', 'QLED')}")
print(f"OLED vs LED: {display_types_compatible('OLED', 'LED')}")

print("\n=== Phase 1 Complete! ===")

Deliverables:

  • ✅ config.py: Complete data loading infrastructure
  • ⏳ utils.py: Text processing utilities
  • ⏳ data_loader.py: JSON loading and validation
  • ⏳ attribute_extractor.py: Dynamic extraction with German keys (no translation)
  • Result: Ready for matcher implementation

Phase 2: Core Matchers (2-3h)

  • matchers/base_matcher.py
  • matchers/ean_matcher.py (Pass 1)
  • matchers/model_matcher.py (Pass 2)
  • matchers/attribute_matcher.py (Pass 3)
  • matchers/weighted_matcher.py (Pass 4)
  • Test: Run each matcher independently

Phase 3: Pipeline (1h)

  • pipeline.py (orchestrate passes 1-4, merge, dedupe)
  • submission.py (format JSON per README:155-188)
  • main.py (CLI with full configurability - see below)
  • SUBMIT v1 for feedback

main.py CLI Interface:

python main.py [options]

# I/O
--source PATH              # Source JSON (default: config)
--target PATH              # Target JSON (default: config)
--output PATH              # Output JSON (default: output/submission.json)

# Pass selection
--passes 1,2,3,4,5         # Which passes to run (default: 1,2,3,4)

# Pass 4: Fully configurable weights (can add ANY spec)
--weight brand=0.25                    # Override default weight
--weight screen_size_inches=0.20
--weight display_type=0.15
--weight resolution=0.15
--weight model_number=0.10
--weight price=0.10
--weight name=0.05
--weight wifi=0.02                     # Add custom spec weight
--weight hdr=0.03                      # Add another custom weight

# Thresholds
--weighted-threshold 0.60              # Pass 4 cutoff (default: 0.60)
--aggressive-threshold 0.50            # Pass 5 cutoff (default: 0.50)

# Output control
--verbose                              # Print detailed match info
--stats                                # Print statistics
--save-specs PATH                      # Save extracted specs to JSON (debug)

Phase 4: Aggressive Matcher (30min)

  • matchers/aggressive_matcher.py (Pass 5 - orphans)
  • Integrate into pipeline
  • SUBMIT v2 (improved recall)

Note: Phase 5 (Web Scraping) is handled by another team member

Total Est: 4-6h (matching only - no scraping)


Key Decisions

  1. Architecture: Modular structure (not single script/notebook)
  2. LLM: Skip for now - focus on rule-based matching (5 passes instead of 6)
  3. Match Policy: Keep all valid matches per source, allow target reuse (maximize recall)
  4. Dynamic Attributes: Extract ALL specs, store with original German keys (no translation)
  5. Fully Configurable Weights: Pass 4 weights for ANY spec via CLI (not just 7 core attributes)
  6. No Pre-filtering/Post-validation: Test all targets initially, add if needed based on results
  7. Scope: Matching algorithm only - scraping handled by another team member

Resources

  • Web App: https://hackathon-production-49ca.up.railway.app/
  • Submission Format: README:155-188 (JSON with source_reference + competitors[])
  • Scoring: README:99-128 (Recall 30pts > Precision 10pts > Coverage 10pts)
  • Dependencies: rapidfuzz (for fuzzy matching)

Implementation Progress

Phase 1: Foundation ✅ COMPLETE

  • Step 1.1: config.py - Complete and tested
    • All paths verified
    • Weights sum to 1.00
    • Focused on matching only (no scraping)
    • Works with German spec keys directly
  • Step 1.2: utils.py - Complete and tested
    • 8 utility functions implemented: normalize_text, clean_model_number, fuzzy_match_text, fuzzy_match_brand, substring_match, calculate_size_score, calculate_price_score, extract_size_from_text
    • All tests passing (47/47 assertions passed)
    • German character handling (ä→AE, ö→OE, ü→UE, ß→SS)
    • Model number suffix removal (UXXN, .AEU, XXN, etc.)
    • Multi-pattern size extraction (Zoll, inch, cm, model numbers)
    • Fuzzy matching with rapidfuzz (ratio and token_sort_ratio)
  • Step 1.3: data_loader.py - Complete and tested
    • 5 functions implemented: load_json, load_source_products, load_target_products, validate_product, get_data_summary
    • All functions tested and working correctly
    • Proper error handling for missing files and invalid JSON
    • Data summary stats: Sources (17 total, 10 with EAN/GTIN 58.8%, 16 with specs 94.1%, 16 with price 94.1%)
    • Data summary stats: Targets (561 total, 86 with EAN/GTIN 15.3%, 351 with specs 62.6%, 184 with price 32.8%)
    • Handles both 'ean' field and 'GTIN' in specifications
    • Handles both 'price' and 'price_eur' fields
  • Step 1.4: attribute_extractor.py - Complete and tested
    • 8 functions implemented: extract_brand, extract_model_number, extract_screen_size, extract_display_type, extract_resolution, extract_all_specs, extract_all_specs_batch, display_types_compatible
    • All tests passing (41/41 assertions passed)
    • Intelligent multi-strategy extraction with fallbacks for all critical fields
    • Brand extraction: brand field → Hersteller → Marke → known brands in name → first word
    • Model extraction: specifications → regex patterns (Samsung/LG/Sony/Generic) → reference field
    • Size extraction: specifications → name → model number (supports Zoll, inch, cm, model codes)
    • Display type: specifications → name (normalized to LED/QLED/OLED/LCD/MINI_LED)
    • Resolution: specifications → name (normalized to UHD_4K/FULL_HD/HD_READY)
    • Preserves ALL additional specs with German keys
    • Extraction success rates: brand 100%, model 100%, size 94.1%, display 94.1%, resolution 94.1%

Phase 2: Matchers ✅ COMPLETE

  • matchers/base_matcher.py - Abstract base class

    • Abstract match() method - all matchers must implement
    • Helper methods: _get_brand_similarity(), _get_size_similarity(), _validate_brand(), _validate_size()
    • Statistics tracking: _update_stats(), get_stats(), print_stats()
    • Verbose logging: _log() method (prints if verbose=True)
    • All tests passing (test_base_matcher.py)
  • matchers/ean_matcher.py (Pass 1: EAN Exact Match)

    • Logic: Exact EAN match + brand validation ≥0.6
    • Confidence: 95-100% (0.95 + brand_sim * 0.05)
    • Uses EAN index for fast lookups
    • Test results: 3 matches found (Samsung F6000, PEAQ 32GF/43GQU)
    • Match rate: 30.0% of sources with EAN
    • All tests passing (test_ean_matcher.py)
  • matchers/model_matcher.py (Pass 2: Model Number Match)

    • Logic: Substring OR fuzzy ≥90% match + brand validation ≥0.6
    • Confidence: 85-95% for substring, 80-90% for fuzzy
    • Test results: 9 matches found (exceeded expected 6-8)
    • Match rate: 52.9% of sources
    • Note: Many matches use reference as model number fallback
    • All tests passing (test_model_matcher.py)
  • matchers/attribute_matcher.py (Pass 3: Strict Attribute Match)

    • Logic: Brand≥0.8 + Size±2" + Display compatible + Resolution matches
    • Confidence: 70-85% based on attribute alignment
    • Test results: 60 matches found (exceeded expected 3-5, good for recall!)
    • Match rate: 375.0% (multiple matches per source)
    • Note: Common attributes (32" Full HD TVs) lead to many compatible matches
    • All tests passing (test_attribute_matcher.py)
  • matchers/weighted_matcher.py (Pass 4: Weighted Score Match)

    • Logic: Weighted similarity score ≥0.60 with configurable weights
    • Default weights: brand (25%), size (20%), display (15%), resolution (15%), model (10%), price (10%), name (5%)
    • Confidence: 60-75% based on score above threshold
    • Supports custom weights for ANY attribute via constructor
    • Normalizes score by applied weights (missing attributes don't unfairly lower score)
    • All tests passing (test_weighted_matcher.py)
  • matchers/aggressive_matcher.py (Pass 5: Aggressive Match - Fallback for Orphans)

    • Logic: Two strategies for orphan sources:
      1. Brand≥0.6 AND Size±2" → confidence 50-60%
      2. Weighted score ≥0.50 → confidence 40-55%
    • Only processes orphan sources (unmatched in previous passes)
    • Confidence: 40-60% based on match quality
    • Designed to maximize recall for difficult cases
    • All tests passing (test_aggressive_matcher.py)

Phase 2 Summary:

  • All 5 matchers implemented and tested
  • Total test files: 6 (base + 5 matchers)
  • All matchers follow consistent interface via BaseMatcher
  • Confidence ranges properly distributed: Pass 1 (95-100%) → Pass 5 (40-60%)
  • Ready for pipeline integration

Phase 3: Pipeline ✅ COMPLETE

  • pipeline.py - Orchestrates the 5-pass matching system

    • Runs all matchers sequentially (Pass 1 → Pass 5)
    • Merges results from all passes
    • Deduplicates matches (keeps highest confidence per source-target pair)
    • Handles orphan detection for Pass 5 (only runs on unmatched sources)
    • Runtime configuration of thresholds (weighted_threshold, aggressive_threshold)
    • Statistics and logging (get_stats(), print_summary())
    • Test results with real data: 394 matches, 100% coverage, avg confidence 0.660
    • Pass breakdown: Pass 1 (3), Pass 2 (6), Pass 3 (52), Pass 4 (333), Pass 5 (0)
    • All tests passing (test_pipeline.py)
  • submission.py - Formats output to submission JSON structure

    • format_submission() - Converts matches to official format per README:155-188
    • save_submission() - Saves official submission JSON
    • save_submission_with_confidence() - Extended format with confidence/pass metadata for analysis
    • print_submission_summary() - Statistics display
    • Handles both required fields (source_reference, competitors[].reference) and optional metadata (retailer, name, url, price)
    • Groups matches by source, includes all competitors per source
    • All tests passing (test_submission.py)
  • main.py - CLI entry point with full configurability

    • Full CLI interface with argparse
    • I/O options: --source, --target, --output-dir, --output-name
    • Pass selection: --passes 1 2 3 4 5 (choose which passes to run)
    • Custom weights: --weight brand=0.30 --weight wifi=0.10 (configure ANY attribute)
    • Thresholds: --weighted-threshold, --aggressive-threshold
    • Output control: --verbose, --stats, --save-specs, --save-extended, --no-metadata
    • Integrates pipeline, submission formatter, and data loaders
    • Test results: All configurations working (default, custom weights, selective passes)
    • Produces official submission JSON ready for scoring

Phase 3 Summary:

  • Complete matching system end-to-end
  • 3 new files: pipeline.py, submission.py, main.py
  • 3 test files: test_pipeline.py, test_submission.py
  • CLI fully functional with comprehensive options
  • Real data results: 394 matches, 100% source coverage, 23.18 avg matches/source
  • Ready for submission and scoring

Session 2 (2026-03-07): Detailed pipeline design complete.
Session 3 (2026-03-07): Phase 1 detailed plan with dynamic attribute extraction strategy.
Session 4 (2026-03-07): Simplified config.py - matching only, no translation. Using agent.md for context. Session 5 (2026-03-07): Implemented and tested utils.py - all 8 functions working correctly. Session 5 (2026-03-07): Implemented and tested data_loader.py - all 5 functions working with accurate data insights. Session 5 (2026-03-07): Implemented and tested attribute_extractor.py - Phase 1 complete! All 8 functions with intelligent fallback logic working perfectly. Session 6 (2026-03-07): Implemented and tested base_matcher.py - Abstract base class complete. Starting Phase 2: Matchers. Session 6 (2026-03-07): Implemented and tested ean_matcher.py (Pass 1) - 3 matches found with real data. Session 6 (2026-03-07): Implemented and tested model_matcher.py (Pass 2) - 9 matches found (exceeded expected 6-8). Session 6 (2026-03-07): Implemented and tested attribute_matcher.py (Pass 3) - 60 matches found (exceeded expected 3-5, high recall). Session 6 (2026-03-07): Implemented and tested weighted_matcher.py (Pass 4) - Flexible weighted scoring with custom weight support. Session 6 (2026-03-07): Implemented and tested aggressive_matcher.py (Pass 5) - Orphan fallback with 2 strategies. Phase 2 COMPLETE! Session 7 (2026-03-07): Implemented and tested pipeline.py - Orchestrates all 5 passes with deduplication and stats. 394 matches, 100% coverage. Session 7 (2026-03-07): Implemented and tested submission.py - Formats matches to official JSON per README. Handles required and optional fields. Session 7 (2026-03-07): Implemented and tested main.py - Full CLI with custom weights, pass selection, thresholds. Phase 3 COMPLETE! System ready for submission.