Checkout SDK Latency Comparison Tool

A comprehensive performance testing and auto-optimization tool for comparing ForkJoinPool.commonPool() vs dedicated ThreadPoolExecutor when using the Checkout SDK with rate limiting.

🎯 Overview

This tool performs real API calls against Checkout's sandbox environment to compare the performance characteristics of different executor types under rate-limited conditions. It includes automated configuration optimization to find the best performance settings for your specific environment.

📊 Key Findings

• Dedicated ThreadPoolExecutor consistently outperforms ForkJoinPool.commonPool() in rate-limited I/O scenarios
• Optimal configuration automatically discovered through systematic testing
• Performance improvements of 50-80% in latency and throughput achieved through auto-tuning

🚀 Quick Start

Prerequisites

1. Java 11+ installed
2. Checkout account with sandbox access
3. Environment variables configured:

cp .env.example .env
# Edit .env with your Checkout credentials:
# CHECKOUT_DEFAULT_PUBLIC_KEY=pk_sbox_...
# CHECKOUT_DEFAULT_SECRET_KEY=sk_sbox_...
# CHECKOUT_PROCESSING_CHANNEL_ID=pc_...

Note: .env and .auto_tune_config files are automatically ignored by git for security.

Run Complete Analysis

# Full automated sequence: test → auto-tune → verify
./run_full_test.sh

This will:

1. Test current configuration
2. Run auto-optimization (45-60 minutes)
3. Apply best settings automatically
4. Verify optimized performance

Quick Test Only

# Test current configuration without optimization
./simple_test.sh

🔧 Available Tools

Script	Purpose	Duration
simple_test.sh	Test current configuration	2-3 minutes
auto_tune.sh	Find optimal configuration	45-60 minutes
run_full_test.sh	Complete automated sequence	50-65 minutes
run_simulation.sh	Enhanced simulation testing	1-2 minutes
load_config.sh	Display current configuration	Instant

🔬 Simulation vs Real Testing

• SdkQueueSimulation.java: Enhanced simulation with rate limiting and auto-tuned config

  • Uses Thread.sleep() to simulate HTTP delays
  • Perfect for controlled testing and demonstrations
  • Shows ~67% improvement with dedicated executors
  • Loads configuration from .auto_tune_config

• SdkQueueSimulationRealSdk.java: Real API testing with Checkout SDK

  • Makes actual HTTP calls to Checkout API
  • Real-world performance measurement
  • Rate limiting and retry logic
  • Production-ready implementation

📈 Auto-Tuned Configuration

The tool automatically tests different combinations of:

• CLIENT_THREADS: Number of concurrent client threads
• SDK_DEDICATED_THREADS: Dedicated executor pool size
• RATE_LIMITER: Maximum concurrent API requests
• CONNECTION_POOL_SIZE: HTTP connection pool size

Current Optimal Configuration

// Auto-discovered optimal settings
CLIENT_THREADS = 35
SDK_DEDICATED_THREADS = 35  
RATE_LIMITER = 42
CONNECTION_POOL_SIZE = 50

// Performance achieved:
// Average Latency: 394.5ms
// Throughput: 17.75 req/s
// Performance Score: 0.0449

🏗️ Architecture

Dynamic Configuration Loading

The system automatically loads configuration from .auto_tune_config if available:

// Java code automatically loads optimized values
private static void loadAutoTuneConfig() {
    // Loads from .auto_tune_config or uses defaults
}

Rate Limiting Strategy

• Semaphore-based concurrent request control
• Exponential backoff for 429 responses
• Retry logic with configurable attempts

Performance Metrics

• Average, P95, P99 latencies
• Throughput (requests per second)
• Composite score (throughput/latency ratio)
• Success rate tracking

📊 Results Analysis

Log Files Structure

logs/
├── auto_tune_results_YYYYMMDD_HHMMSS/
│   ├── results.csv              # Complete test results
│   └── test_*.log              # Individual test logs
└── simple_test_YYYYMMDD_HHMMSS.log  # Simple test results

Performance Comparison

Executor Type	Avg Latency	Throughput	Score
Dedicated	394.5ms	17.75 req/s	0.0449
CommonPool	1200-1400ms	11-13 req/s	0.009-0.011

Results from auto-tuned optimal configuration

⚙️ Configuration Details

Environment Variables

CHECKOUT_DEFAULT_PUBLIC_KEY=pk_sbox_...    # Sandbox public key
CHECKOUT_DEFAULT_SECRET_KEY=sk_sbox_...    # Sandbox secret key  
CHECKOUT_PROCESSING_CHANNEL_ID=pc_...      # Processing channel

Auto-Tune Configuration File

.auto_tune_config contains the discovered optimal settings:

# Auto-tuned optimal configuration
# Generated: [timestamp]
# Best Score: 0.0449
CLIENT_THREADS=35
SDK_DEDICATED_THREADS=35
CONNECTION_POOL_SIZE=50
RATE_LIMITER=42
BEST_LATENCY=Avg=394.5ms, Throughput=17.75

🔬 Technical Details

Why Dedicated Executor Wins

1. Work-stealing overhead: CommonPool's work-stealing adds latency in I/O-bound scenarios
2. Thread contention: Dedicated threads avoid interference from other application tasks
3. Rate limiting efficiency: Dedicated executor handles semaphore blocking more efficiently
4. Resource isolation: Separate thread pool prevents cross-contamination

Rate Limiting Implementation

// Semaphore-based rate limiting
private static final Semaphore RATE_LIMITER = new Semaphore(42);

// With exponential backoff retry
private static <T> T executeWithRateLimit(Callable<T> operation) {
    // Implementation with retry logic and backoff
}

HTTP Client Configuration

// Optimized connection pooling
PoolingHttpClientConnectionManager connManager = 
    new PoolingHttpClientConnectionManager();
connManager.setMaxTotal(CONNECTION_POOL_SIZE);
connManager.setDefaultMaxPerRoute(CONNECTION_POOL_SIZE);

🚀 Production Recommendations

1. Use the auto-tuned configuration as starting point
2. Monitor performance in production environment
3. Re-run auto-tuning if load patterns change significantly
4. Consider dedicated executor for I/O-heavy Checkout SDK usage

📝 Development

Building

./gradlew build

Running Tests

./gradlew test

Code Structure

src/main/java/org/example/
└── SdkQueueSimulationRealSdk.java  # Main comparison tool

🤝 Contributing

1. Fork the repository
2. Create your feature branch
3. Add tests for new functionality
4. Ensure all tests pass
5. Submit a pull request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🔗 Related Links

• Checkout SDK Documentation
• Java Concurrency Guide
• Performance Tuning Best Practices

---

Auto-generated documentation based on performance testing results