AutoTrain Advanced - Tinker-Inspired Features Documentation

Version: 1.0.0 Date: October 2, 2025 Status: Production Ready

Overview
Feature Comparison with Standard AutoTrain
Prompt Distillation
Message Rendering System
Completers Interface
Hyperparameter Sweeps
Enhanced Evaluation Framework
CLI & API Integration
Best Practices & When to Use

Overview

This document describes 5 new enterprise-grade features added to AutoTrain Advanced, inspired by Mira Murati's Tinker framework. These features provide advanced training capabilities beyond standard AutoTrain options.

What's New

Feature	Purpose	Key Benefit
Prompt Distillation	Knowledge transfer from complex prompts	Reduce inference costs by 50-90%
Message Rendering	Unified conversation formatting	Support 6 chat formats with token control
Completers	Two-level generation API	Simplified inference with streaming
Hyperparameter Sweeps	Automated optimization	Find optimal hyperparameters automatically
Enhanced Evaluation	Comprehensive metrics	Deep model analysis with 10+ metrics

Feature Comparison with Standard AutoTrain

Evaluation: Standard vs Enhanced

Standard AutoTrain Evaluation

# Basic evaluation in standard AutoTrain
from transformers import Trainer

trainer = Trainer(...)
metrics = trainer.evaluate()
# Returns: {'eval_loss': 0.5, 'eval_runtime': 10.5}

Limitations:

✗ Only loss metric
✗ No text generation metrics
✗ No perplexity
✗ No model comparison
✗ No benchmark datasets

Enhanced Evaluation (NEW)

from autotrain.evaluation import Evaluator, EvaluationConfig, MetricType

config = EvaluationConfig(
    metrics=[
        MetricType.PERPLEXITY,
        MetricType.BLEU,
        MetricType.ROUGE,
        MetricType.BERTSCORE,
        MetricType.F1,
    ],
    task="generation",
    save_predictions=True,
)

evaluator = Evaluator(model, tokenizer, config)
result = evaluator.evaluate(dataset)

# Returns comprehensive metrics:
# {
#     'perplexity': 12.5,
#     'bleu': 0.45,
#     'rouge1': 0.52,
#     'rouge2': 0.31,
#     'rougeL': 0.48,
#     'bert_score_f1': 0.89,
#     'f1': 0.72,
#     'loss': 0.35,
#     'samples_per_second': 125.3
# }

When to use Enhanced Evaluation:

✅ Evaluating text generation models
✅ Comparing multiple models
✅ Running standard benchmarks (MMLU, HellaSwag, etc.)
✅ Need detailed metrics for production deployment
✅ Want to track metrics during training

Training: Standard vs Prompt Distillation

Standard Fine-Tuning

aitraining llm --train \
  --model gpt2 \
  --data-path ./data \
  --text-column text \
  --lr 2e-5 \
  --epochs 3

Use Case: Regular supervised fine-tuning Cost: Full model training costs

Prompt Distillation (NEW)

from autotrain.trainers.clm.train_clm_distill import (
    PromptDistillationConfig,
    train_prompt_distillation
)

config = PromptDistillationConfig(
    teacher_model_name="gpt-3.5-turbo",
    teacher_prompt_template="""You are an expert assistant.
    Use chain-of-thought reasoning.
    Be precise and concise.

    Query: {input}""",

    student_model_name="gpt2",
    student_prompt_template="{input}",  # Simple prompt

    temperature=3.0,  # Soften teacher distributions
    alpha=0.7,        # 70% KL loss, 30% CE loss
    use_peft=True,    # Use LoRA for efficient training
)

trainer = train_prompt_distillation(
    config=config,
    base_inputs=training_data,
    output_dir="./distilled_model"
)

When to use Prompt Distillation:

✅ You have a good prompt that's too expensive for production
✅ Want to reduce inference costs (complex prompt → simple model)
✅ Using API models (GPT-4, Claude) and want local alternative
✅ Need consistent behavior without prompt engineering at inference

Cost Savings Example:

Before: 500 tokens/request (long prompt) × $0.03/1K = $0.015/request
After: 50 tokens/request (no prompt) × $0.002/1K = $0.0001/request
Savings: 99% reduction in inference costs

Hyperparameter Tuning: Standard vs Sweep

Standard AutoTrain

# Manual trial and error
aitraining llm --train --lr 1e-5 ...  # Try 1
aitraining llm --train --lr 2e-5 ...  # Try 2
aitraining llm --train --lr 5e-5 ...  # Try 3

Limitations:

✗ Manual process
✗ No optimization strategy
✗ Time-consuming
✗ May miss optimal values

Hyperparameter Sweep (NEW)

from autotrain.utils import HyperparameterSweep, SweepConfig, ParameterRange

config = SweepConfig(
    backend="optuna",  # or "random", "grid", "ray"
    optimization_metric="eval_loss",
    optimization_mode="minimize",
    num_trials=20,
    parallel_jobs=4,
)

sweep = HyperparameterSweep(
    objective_function=train_model,
    config=config,
    parameters=[
        ParameterRange("learning_rate", "log_uniform", low=1e-5, high=1e-3),
        ParameterRange("batch_size", "categorical", choices=[4, 8, 16, 32]),
        ParameterRange("epochs", "int", low=3, high=10),
        ParameterRange("dropout", "float", low=0.0, high=0.3),
    ]
)

result = sweep.run()
print(f"Best params: {result.best_params}")
print(f"Best loss: {result.best_value}")

# Visualize results
result.plot_optimization_history(save_path="sweep_history.png")

When to use Hyperparameter Sweep:

✅ First time training on new dataset
✅ Want best possible performance
✅ Have compute budget for optimization
✅ Need reproducible hyperparameter selection
✅ Multiple hyperparameters to tune

Typical Results:

Manual tuning: ~2-5% improvement
Optuna sweep (20 trials): ~8-15% improvement
Time investment: 2-3x training time, but automated

1. Prompt Distillation

Concept

Train a smaller "student" model to internalize complex prompts from a "teacher" model, eliminating the need for expensive prompts at inference time.

Architecture

┌─────────────────────────────────┐
│  Teacher Model + Complex Prompt │
│  "You are an expert... [500 tokens]" │
└────────────┬────────────────────┘
             │
             │ Generate outputs + logits
             ▼
      ┌──────────────┐
      │  KL Divergence Loss
      └──────┬───────┘
             │
             ▼
┌─────────────────────────────────┐
│  Student Model (Simple/No Prompt) │
│  Learns to mimic teacher behavior │
└─────────────────────────────────┘

API Reference

Option 1: Integrated SFT Distillation (NEW - Recommended)

from autotrain.trainers.clm.params import LLMTrainingParams

# Distillation is now integrated into SFT trainer
config = LLMTrainingParams(
    trainer="sft",  # Use SFT trainer

    # Enable distillation mode
    use_distillation=True,
    teacher_model="gpt-3.5-turbo",  # Teacher model
    teacher_prompt_template="Complex prompt: {input}",
    student_prompt_template="{input}",
    distill_temperature=3.0,
    distill_alpha=0.7,

    # Standard training params
    model="gpt2",  # Student model
    data_path="./data",
    project_name="distilled_model",
    lr=1e-5,
    batch_size=4,
    epochs=3,

    # PEFT (optional)
    peft=True,
    lora_r=16,
    lora_alpha=32,
)

# Train via CLI
# python -m autotrain.cli.aitraining llm --train \
#   --trainer sft \
#   --use-distillation \
#   --teacher-model gpt-3.5-turbo \
#   --distill-temperature 3.0 \
#   --distill-alpha 0.7

Option 2: Standalone Distillation (Legacy)

from autotrain.trainers.clm.train_clm_distill import (
    PromptDistillationConfig,
    train_prompt_distillation
)

# Still available for backward compatibility
config = PromptDistillationConfig(
    teacher_model_name="gpt-3.5-turbo",
    student_model_name="gpt2",
    teacher_prompt_template="{input}",
    student_prompt_template="",
    temperature=3.0,
    alpha=0.7,
)

trainer = train_prompt_distillation(
    config=config,
    base_inputs=training_data,
    output_dir="./distilled_model"
)

Usage Examples

Example 1: Distill GPT-4 Chain-of-Thought to GPT-2

from autotrain.trainers.clm.train_clm_distill import (
    PromptDistillationConfig,
    train_prompt_distillation
)

# Load your queries
queries = [
    "What is the capital of France?",
    "Explain quantum entanglement simply",
    # ... more queries
]

config = PromptDistillationConfig(
    teacher_model_name="gpt-4",  # or API model
    teacher_prompt_template="""You are a helpful AI assistant.
    Think step-by-step and explain your reasoning.

    Question: {input}

    Let's think through this carefully:""",

    student_model_name="gpt2-medium",
    student_prompt_template="{input}",  # Just the question

    temperature=3.0,  # Higher = softer distributions
    alpha=0.7,        # Balance KL and CE loss

    # Training
    num_samples=len(queries),
    num_epochs=3,
    batch_size=4,
    learning_rate=1e-5,

    # Use LoRA for memory efficiency
    use_peft=True,
    lora_r=16,
    lora_alpha=32,
)

trainer = train_prompt_distillation(
    config=config,
    base_inputs=queries,
    output_dir="./gpt2_distilled",
    validation_inputs=queries[:100],  # Validation set
)

# Use the distilled model
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("./gpt2_distilled")
tokenizer = AutoTokenizer.from_pretrained("./gpt2_distilled")

# No prompt needed!
response = model.generate(
    tokenizer("What is the capital of France?", return_tensors="pt").input_ids
)

Example 2: Distill Domain-Specific Expertise

# Medical domain example
medical_config = PromptDistillationConfig(
    teacher_model_name="meditron-70b",
    teacher_prompt_template="""You are a medical expert.
    Provide evidence-based answers citing relevant research.
    Consider differential diagnoses and treatment options.

    Medical query: {input}

    Analysis:""",

    student_model_name="llama-2-7b",
    student_prompt_template="{input}",

    temperature=2.5,  # Lower for more focused transfer
    alpha=0.8,        # Higher weight on KL loss

    num_epochs=5,     # More epochs for domain expertise
)

# Medical Q&A dataset
medical_queries = load_medical_dataset()

trainer = train_prompt_distillation(
    config=medical_config,
    base_inputs=medical_queries,
    output_dir="./medical_llama_distilled"
)

Parameters Explained

`temperature`

Purpose: Controls how "soft" the teacher's probability distributions are
Range: 1.0 - 5.0 (typically 2.0 - 4.0)
Effect:
- Low (1-2): Hard targets, faster convergence, less knowledge transfer
- High (3-5): Soft targets, better knowledge transfer, may need more epochs
Recommendation: Start with 3.0

`alpha`

Purpose: Balance between KL divergence loss and cross-entropy loss
Formula: loss = alpha * KL_loss + (1 - alpha) * CE_loss
Range: 0.0 - 1.0
Effect:
- 0.0: Pure cross-entropy (normal training, no distillation)
- 0.5: Equal weight
- 1.0: Pure KL divergence
Recommendation: 0.6 - 0.8 for best results

`use_peft` (LoRA)

Purpose: Memory-efficient training using Low-Rank Adaptation
When to use:
- ✅ Limited GPU memory
- ✅ Want faster training
- ✅ Fine-tuning large models (>1B params)
Memory savings: ~60-70% reduction
Performance: Comparable to full fine-tuning

When to Use Prompt Distillation

✅ Use when:

You have a complex prompt that works well but is expensive
Using API models (GPT-4, Claude) and want local alternative
Need consistent behavior without runtime prompt engineering
Want to deploy smaller models with complex capabilities

❌ Don't use when:

Your task doesn't benefit from complex prompting
You need the model to adapt to different prompts at runtime
Training data is extremely limited (<100 examples)

Common Issues & Solutions

Issue: Student doesn't learn well Solution:

Increase temperature to 4.0 or 5.0
Increase alpha to 0.8 or 0.9
Train for more epochs
Ensure teacher outputs are diverse

Issue: Out of memory Solution:

Enable use_peft=True
Reduce batch_size
Reduce max_length
Increase gradient_accumulation_steps

2. Message Rendering System

Concept

Unified conversation-to-token conversion supporting 6 chat formats with fine-grained token-level control for advanced training techniques.

Supported Formats

Format	Example	Use Case
ChatML	`<\|im_start\|>user\nHello<\|im_end\|>`	OpenAI models, most general
Alpaca	`### Instruction:\nHello\n### Response:\n`	Instruction tuning
Llama 2	`<s>[INST] Hello [/INST]`	Meta Llama models
Vicuna	`USER: Hello\nASSISTANT:`	Vicuna models
Zephyr	`<\|user\|>\nHello</s>`	HuggingFace Zephyr
Mistral	`[INST] Hello [/INST]`	Mistral AI models

API Reference

Core Classes

@dataclass
class Message:
    role: str                    # "system", "user", "assistant"
    content: str                 # Message text
    weight: float = 1.0          # Loss weight (0.0 - 1.0)
    metadata: Dict = field(default_factory=dict)

@dataclass
class Conversation:
    messages: List[Message]
    metadata: Dict = field(default_factory=dict)

    def add_message(self, role: str, content: str, weight: float = 1.0)
    def to_dict(self) -> Dict
    @classmethod
    def from_dict(cls, data: Dict) -> Conversation

@dataclass
class RenderConfig:
    format: ChatFormat              # Chat format to use
    add_generation_prompt: bool = True
    separator: str = "\n"
    system_prefix: str = ""

`MessageRenderer`

class MessageRenderer(ABC):
    def render_conversation(self, conversation: Conversation) -> str
    def tokenize_conversation(self, conversation: Conversation) -> Dict[str, Tensor]
    def build_generation_prompt(self, conversation: Conversation) -> str
    def parse_response(self, response: str) -> str
    def get_stop_sequences(self) -> List[str]

Usage Examples

Example 1: Basic Conversation Rendering

from autotrain.rendering import (
    Message, Conversation, MessageRenderer,
    ChatFormat, RenderConfig, get_renderer
)
from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("gpt2")
config = RenderConfig(format=ChatFormat.CHATML)
renderer = get_renderer(ChatFormat.CHATML, tokenizer, config)

# Create conversation
conversation = Conversation(messages=[
    Message(role="system", content="You are a helpful assistant."),
    Message(role="user", content="What is 2+2?"),
    Message(role="assistant", content="4"),
])

# Render as text
text = renderer.render_conversation(conversation)
# Output:
# <|im_start|>system
# You are a helpful assistant.<|im_end|>
# <|im_start|>user
# What is 2+2?<|im_end|>
# <|im_start|>assistant
# 4<|im_end|>

# Tokenize for training
tokenized = renderer.tokenize_conversation(conversation)
# Returns: {'input_ids': ..., 'attention_mask': ..., 'labels': ...}

Example 2: Token-Level Weight Control

# Use case: Don't train on system prompts and questions, only answers
conversation = Conversation(messages=[
    Message(role="system", content="You are helpful.", weight=0.0),  # Ignore
    Message(role="user", content="Question?", weight=0.0),           # Ignore
    Message(role="assistant", content="Answer!", weight=1.0),        # Train
])

tokenized = renderer.tokenize_conversation(conversation)
# labels will have -100 for system/user tokens, actual IDs for assistant

Example 3: Multi-Format Dataset Preparation

from autotrain.rendering.utils import convert_dataset_to_conversations

# Your dataset
dataset = [
    {"instruction": "Translate to French", "input": "Hello", "output": "Bonjour"},
    {"instruction": "Summarize", "input": "Long text...", "output": "Summary"},
]

# Convert to conversations
conversations = convert_dataset_to_conversations(
    dataset,
    instruction_column="instruction",
    input_column="input",
    output_column="output",
    system_message="You are a helpful AI assistant."
)

# Render in different formats
for format in [ChatFormat.CHATML, ChatFormat.ALPACA, ChatFormat.LLAMA]:
    renderer = get_renderer(format, tokenizer)
    for conv in conversations:
        text = renderer.render_conversation(conv)
        print(f"{format.value}:", text)

Example 4: Generation Prompts

# For generation, we need a prompt that ends with the assistant marker
conversation = Conversation(messages=[
    Message(role="system", content="Be concise."),
    Message(role="user", content="What is AI?"),
])

generation_prompt = renderer.build_generation_prompt(conversation)
# Output: "<|im_start|>system\nBe concise.<|im_end|>\n<|im_start|>user\nWhat is AI?<|im_end|>\n<|im_start|>assistant\n"

# Use for generation
inputs = tokenizer(generation_prompt, return_tensors="pt")
outputs = model.generate(**inputs, stop_strings=renderer.get_stop_sequences())

When to Use Each Format

Format	Use When
ChatML	• General purpose • OpenAI-style models • Need clear role separation
Alpaca	• Instruction following • Single-turn Q&A • Simple datasets
Llama 2	• Using Meta's Llama models • Need official format
Vicuna	• Multi-turn conversations • Chatbot applications
Zephyr	• HuggingFace ecosystem • Alignment with Zephyr models
Mistral	• Mistral AI models • Need official format

Advanced: Token-Level Loss Control

from autotrain.rendering.utils import build_supervised_example

# Create supervised training example
conversation = Conversation(messages=[
    Message("system", "Expert mode", weight=0.0),    # Don't train on this
    Message("user", "Solve: 2x + 3 = 7", weight=0.2),  # Low weight
    Message("assistant", "x = 2", weight=1.0),       # Full weight
])

tokenized = build_supervised_example(
    conversation,
    tokenizer,
    renderer,
    max_length=512,
    mask_inputs=True,  # Mask user inputs (weight=0)
)

# Labels will be:
# [-100, -100, ..., 2, ..., -100]  # -100 for masked, actual IDs for assistant

Best Practices

Consistency: Use the same format for training and inference
Stop Sequences: Always use renderer.get_stop_sequences() during generation
Token Weights: Use weight=0.0 to ignore system prompts and focus on responses
Format Selection: Match the format to your base model's training

3. Completers Interface

Concept

Two-level generation API providing simple interfaces for both token-level (RL) and message-level (chat) generation with streaming and async support.

Architecture Levels

┌─────────────────────────────────┐
│     MessageCompleter            │  High-level: Chat conversations
│  - chat(message) → response     │
│  - batch messages               │
└───────────┬─────────────────────┘
            │ Uses
            ▼
┌─────────────────────────────────┐
│     TokenCompleter              │  Low-level: Token sequences
│  - complete(prompt) → tokens    │
│  - stream_tokens()              │
│  - get_logprobs()               │
└─────────────────────────────────┘

API Reference

`CompletionConfig`

@dataclass
class CompletionConfig:
    max_new_tokens: int = 100
    temperature: float = 1.0
    top_k: int = 50
    top_p: float = 0.95
    repetition_penalty: float = 1.0
    do_sample: bool = True
    num_return_sequences: int = 1

    # Advanced
    use_cache: bool = True
    return_log_probs: bool = False
    stop_strings: List[str] = field(default_factory=list)

`TokenCompleter`

class TokenCompleter:
    def __init__(self, model, tokenizer, config: CompletionConfig)

    def complete(self, prompt: Union[str, Tensor]) -> TokenCompletionResult
    def batch_complete(self, prompts: List[Union[str, Tensor]]) -> List[TokenCompletionResult]
    def stream_tokens(self, prompt: Union[str, Tensor]) -> Iterator[Tensor]

`MessageCompleter`

class MessageCompleter:
    def __init__(self, model, tokenizer, config: CompletionConfig, chat_format: ChatFormat)

    def complete(self, conversation: Union[Conversation, List[Dict]]) -> MessageCompletionResult
    def chat(self, user_input: str, conversation: Optional[Conversation] = None) -> MessageCompletionResult
    def batch_complete(self, conversations: List[Conversation]) -> List[MessageCompletionResult]

Usage Examples

Example 1: Simple Chat Interface

from autotrain.generation import create_chat_session

# Create interactive session
session = create_chat_session(
    model="gpt2",
    system_prompt="You are a helpful coding assistant.",
    chat_format=ChatFormat.CHATML
)

# Chat
response1 = session.chat("How do I reverse a list in Python?")
print(response1)  # "Use list.reverse() or list[::-1]"

response2 = session.chat("Show me an example")
print(response2)  # "my_list = [1, 2, 3]; my_list.reverse()"

# History is maintained
history = session.get_history()
# [
#   {'role': 'system', 'content': 'You are...'},
#   {'role': 'user', 'content': 'How do I...'},
#   {'role': 'assistant', 'content': 'Use list.reverse()...'},
#   {'role': 'user', 'content': 'Show me...'},
#   {'role': 'assistant', 'content': 'my_list = ...'}
# ]

# Save/load
session.save_history("conversation.json")
session.load_history("conversation.json")

Example 2: Token-Level Generation (for RL)

from autotrain.generation import TokenCompleter, CompletionConfig

config = CompletionConfig(
    max_new_tokens=50,
    temperature=0.8,
    return_log_probs=True,  # For RL reward calculation
)

completer = TokenCompleter(model, tokenizer, config)

# Single completion
result = completer.complete("The capital of France is")
print(result.text)          # " Paris."
print(result.tokens)        # [3681, 13]
print(result.log_probs)     # [-0.5, -1.2]

# Streaming
for token_id in completer.stream_tokens("Once upon a time"):
    token_text = tokenizer.decode([token_id])
    print(token_text, end="", flush=True)

Example 3: Batch Processing

from autotrain.generation import create_completer, batch_complete

# Create completer
completer = create_completer(
    model="gpt2-medium",
    completer_type="message",
    chat_format=ChatFormat.ALPACA,
    config=CompletionConfig(max_new_tokens=100, temperature=0.7)
)

# Batch conversations
conversations = [
    Conversation(messages=[Message("user", "What is 2+2?")]),
    Conversation(messages=[Message("user", "What is the capital of France?")]),
    Conversation(messages=[Message("user", "Explain gravity")]),
]

results = batch_complete(
    completer,
    conversations,
    batch_size=8,
    show_progress=True
)

for conv, result in zip(conversations, results):
    print(f"Q: {conv.messages[0].content}")
    print(f"A: {result.content}\n")

Example 4: Async Generation

from autotrain.generation import AsyncMessageCompleter
import asyncio

async def generate_responses(questions):
    completer = AsyncMessageCompleter(model, tokenizer, config, chat_format)

    tasks = []
    for q in questions:
        conv = Conversation(messages=[Message("user", q)])
        tasks.append(completer.complete_async(conv))

    # Process concurrently
    results = await asyncio.gather(*tasks)
    return results

questions = ["What is AI?", "Explain ML", "What is DL?"]
responses = asyncio.run(generate_responses(questions))

Sampling Strategies

from autotrain.generation.sampling import (
    TopKSampler, TopPSampler, TypicalSampler, BeamSearchSampler
)

# Top-K sampling
top_k_sampler = TopKSampler(k=50)
output = top_k_sampler.sample(logits)

# Nucleus (Top-P) sampling
top_p_sampler = TopPSampler(p=0.95)
output = top_p_sampler.sample(logits)

# Typical sampling (better than Top-P for some tasks)
typical_sampler = TypicalSampler(tau=0.95)
output = typical_sampler.sample(logits)

# Beam search
beam_sampler = BeamSearchSampler(num_beams=5)
outputs = beam_sampler.sample(logits)

When to Use Token vs Message Completer

Use Case	Use
Reinforcement Learning	`TokenCompleter` (need token-level control)
Interactive Chat	`MessageCompleter`
Batch Inference	Either (Message for convenience)
Streaming Chat	`MessageCompleter`
Custom Sampling	`TokenCompleter`
Production API	`MessageCompleter` (cleaner interface)

4. Hyperparameter Sweeps

Concept

Automated hyperparameter optimization using multiple backends (Optuna, Ray Tune, Grid Search, Random Search) to find optimal training configurations.

Supported Backends

Backend	Best For	Pros	Cons
Optuna	General purpose	Smart sampling, pruning, parallel	Requires `pip install optuna`
Ray Tune	Large-scale	Distributed, advanced algorithms	Requires `pip install ray[tune]`
Grid Search	Systematic exploration	Exhaustive, reproducible	Slow for many params
Random Search	Quick experiments	No dependencies, fast	Less efficient

API Reference

`SweepConfig`

@dataclass
class SweepConfig:
    backend: SweepBackend         # OPTUNA, RAY, GRID, RANDOM
    optimization_metric: str       # Metric to optimize
    optimization_mode: str         # "minimize" or "maximize"
    num_trials: int = 10
    parallel_jobs: int = 1
    early_stopping: bool = False
    early_stopping_patience: int = 5
    output_dir: str = "./sweep_results"

`ParameterRange`

@dataclass
class ParameterRange:
    name: str
    param_type: str               # "float", "int", "categorical", "log_uniform"
    low: Optional[float] = None   # For float/int
    high: Optional[float] = None  # For float/int
    choices: Optional[List] = None  # For categorical
    step: Optional[float] = None  # For int (optional)

`HyperparameterSweep`

class HyperparameterSweep:
    def __init__(
        self,
        objective_function: Callable,
        config: SweepConfig,
        parameters: List[ParameterRange]
    )

    def run(self) -> SweepResult
    def plot_optimization_history(self, save_path: str)
    def plot_parameter_importance(self, save_path: str)

Usage Examples

Example 1: Optuna-Based Sweep

from autotrain.utils import (
    HyperparameterSweep, SweepConfig, ParameterRange, SweepBackend
)

def train_model(params):
    """Your training function."""
    model = create_model(
        learning_rate=params["learning_rate"],
        batch_size=params["batch_size"],
        dropout=params["dropout"],
    )

    trainer = Trainer(
        model=model,
        args=training_args,
        train_dataset=train_data,
        eval_dataset=val_data,
    )

    metrics = trainer.train()
    return metrics["eval_loss"]

# Configure sweep
config = SweepConfig(
    backend=SweepBackend.OPTUNA,
    optimization_metric="eval_loss",
    optimization_mode="minimize",
    num_trials=20,
    parallel_jobs=4,
    early_stopping=True,
    early_stopping_patience=5,
)

# Define parameter space
parameters = [
    ParameterRange("learning_rate", "log_uniform", low=1e-5, high=1e-3),
    ParameterRange("batch_size", "categorical", choices=[4, 8, 16, 32]),
    ParameterRange("dropout", "float", low=0.0, high=0.5),
    ParameterRange("epochs", "int", low=3, high=10),
]

# Run sweep
sweep = HyperparameterSweep(
    objective_function=train_model,
    config=config,
    parameters=parameters
)

result = sweep.run()

# Best parameters
print(f"Best params: {result.best_params}")
# {'learning_rate': 2.3e-4, 'batch_size': 16, 'dropout': 0.15, 'epochs': 5}

print(f"Best loss: {result.best_value}")
# 0.234

# Visualize
result.plot_optimization_history(save_path="sweep_history.png")
result.plot_parameter_importance(save_path="param_importance.png")

# Save results
result.save("sweep_results.json")

Example 2: Grid Search for Systematic Exploration

# Grid search over all combinations
config = SweepConfig(
    backend=SweepBackend.GRID,
    optimization_metric="accuracy",
    optimization_mode="maximize",
)

parameters = [
    ParameterRange("learning_rate", "categorical", choices=[1e-5, 5e-5, 1e-4]),
    ParameterRange("batch_size", "categorical", choices=[8, 16]),
    ParameterRange("epochs", "categorical", choices=[3, 5]),
]
# Total trials: 3 × 2 × 2 = 12

sweep = HyperparameterSweep(train_model, config, parameters)
result = sweep.run()

Example 3: Random Search for Quick Exploration

# Fast random sampling
config = SweepConfig(
    backend=SweepBackend.RANDOM,
    optimization_metric="f1",
    optimization_mode="maximize",
    num_trials=10,
)

parameters = [
    ParameterRange("learning_rate", "log_uniform", low=1e-6, high=1e-3),
    ParameterRange("weight_decay", "log_uniform", low=1e-6, high=1e-1),
    ParameterRange("warmup_ratio", "float", low=0.0, high=0.2),
]

sweep = HyperparameterSweep(train_model, config, parameters)
result = sweep.run()

Example 4: AutoTrain Integration

from autotrain.utils import run_autotrain_sweep

# Sweep for aitraining llm training
result = run_autotrain_sweep(
    task_type="llm",
    train_params={
        "model": "gpt2",
        "data_path": "./data",
        "project_name": "sweep_test",
    },
    sweep_params=[
        ParameterRange("lr", "log_uniform", low=1e-5, high=1e-3),
        ParameterRange("batch_size", "categorical", choices=[4, 8, 16]),
        ParameterRange("epochs", "int", low=1, high=5),
    ],
    num_trials=15,
)

Parameter Types Explained

Log Uniform

Use for: Learning rate, weight decay, epsilon values
Why: Explores orders of magnitude evenly
Example: ParameterRange("lr", "log_uniform", low=1e-5, high=1e-3)
- Will sample: 1e-5, 3e-5, 1e-4, 3e-4, 1e-3 (not 1e-5, 2.5e-4, 5e-4, 7.5e-4, 1e-3)

Float

Use for: Dropout, temperature, alpha values
Why: Linear exploration
Example: ParameterRange("dropout", "float", low=0.0, high=0.5)

Int

Use for: Epochs, hidden dimensions, layers
Why: Discrete values
Example: ParameterRange("epochs", "int", low=3, high=10)

Categorical

Use for: Batch size, optimizer type, activation functions
Why: Discrete choices
Example: ParameterRange("optimizer", "categorical", choices=["adam", "adamw", "sgd"])

Best Practices

Start Small: Begin with 5-10 trials to test your setup
Use Log Uniform: For learning rate and similar parameters
Parallel Jobs: Set based on available GPUs/CPUs
Early Stopping: Enable to save compute on poor configurations
Save Results: Always save sweep results for reproducibility

Common Sweep Strategies

Quick Sweep (30 mins)

config = SweepConfig(
    backend=SweepBackend.RANDOM,
    num_trials=5,
    parallel_jobs=5,
)
parameters = [
    ParameterRange("lr", "log_uniform", low=1e-5, high=1e-3),
    ParameterRange("batch_size", "categorical", choices=[8, 16]),
]

Thorough Sweep (4-8 hours)

config = SweepConfig(
    backend=SweepBackend.OPTUNA,
    num_trials=50,
    parallel_jobs=4,
    early_stopping=True,
)
parameters = [
    ParameterRange("lr", "log_uniform", low=1e-6, high=1e-3),
    ParameterRange("batch_size", "categorical", choices=[4, 8, 16, 32]),
    ParameterRange("dropout", "float", low=0.0, high=0.5),
    ParameterRange("warmup_ratio", "float", low=0.0, high=0.2),
    ParameterRange("weight_decay", "log_uniform", low=1e-6, high=1e-1),
]

5. Enhanced Evaluation Framework

Concept

Comprehensive model evaluation with 10+ metrics (BLEU, ROUGE, BERTScore, Perplexity, F1, etc.), benchmarking support, and training callbacks.

Available Metrics

Metric	Task Type	Description	Range
Perplexity	Language Modeling	Prediction confidence	Lower is better
BLEU	Generation	N-gram overlap	0-1 (higher better)
ROUGE	Generation	Recall-oriented overlap	0-1 (higher better)
BERTScore	Generation	Semantic similarity	0-1 (higher better)
METEOR	Generation	Alignment with synonyms	0-1 (higher better)
Exact Match	QA	Exact string match	0-1 (higher better)
F1	Classification	Precision-recall balance	0-1 (higher better)
Accuracy	Classification	Correct predictions	0-1 (higher better)

API Reference

`EvaluationConfig`

@dataclass
class EvaluationConfig:
    metrics: List[MetricType]
    task: str = "language_modeling"  # or "generation", "classification"
    batch_size: int = 8
    max_samples: Optional[int] = None
    device: str = "auto"
    fp16: bool = False
    save_predictions: bool = False
    output_dir: str = "./eval_results"
    verbose: bool = True

    # For generation tasks
    max_length: int = 512
    generation_config: Optional[Dict] = None

`Evaluator`

class Evaluator:
    def __init__(self, model, tokenizer, config: EvaluationConfig)

    def evaluate(self, dataset) -> EvaluationResult
    def evaluate_generation(self, dataset, references: List[str]) -> EvaluationResult
    def evaluate_classification(self, dataset, labels: List[int]) -> EvaluationResult

`Benchmark`

class Benchmark:
    def __init__(self, config: BenchmarkConfig)

    def run_benchmark(self, model, model_name: str) -> BenchmarkResult
    def compare_models(self, models: Dict[str, Any]) -> pd.DataFrame

Usage Examples

Example 1: Language Model Evaluation

from autotrain.evaluation import Evaluator, EvaluationConfig, MetricType

# Configure evaluation
config = EvaluationConfig(
    metrics=[MetricType.PERPLEXITY, MetricType.ACCURACY],
    task="language_modeling",
    batch_size=16,
    max_samples=1000,
    save_predictions=False,
)

# Create evaluator
evaluator = Evaluator(model, tokenizer, config)

# Evaluate
result = evaluator.evaluate(test_dataset)

print(f"Perplexity: {result.metrics['perplexity']:.2f}")
print(f"Loss: {result.metrics['loss']:.4f}")
print(f"Samples/sec: {result.metrics['samples_per_second']:.1f}")

# Save results
result.save("eval_results.json")

Example 2: Generation Quality Evaluation

from autotrain.evaluation import evaluate_generation

# Your generated texts
predictions = [
    "The capital of France is Paris.",
    "Machine learning is a subset of AI.",
]

# Reference texts
references = [
    "Paris is the capital of France.",
    "ML is part of artificial intelligence.",
]

# Evaluate
config = EvaluationConfig(
    metrics=[
        MetricType.BLEU,
        MetricType.ROUGE,
        MetricType.BERTSCORE,
        MetricType.METEOR,
    ],
    task="generation",
)

result = evaluate_generation(
    model=model,
    tokenizer=tokenizer,
    dataset=test_data,
    references=references,
    config=config
)

print(f"BLEU: {result.metrics['bleu']:.3f}")
print(f"ROUGE-1: {result.metrics['rouge1']:.3f}")
print(f"ROUGE-L: {result.metrics['rougeL']:.3f}")
print(f"BERTScore F1: {result.metrics['bert_score_f1']:.3f}")

Example 3: Benchmarking

from autotrain.evaluation import Benchmark, BenchmarkConfig

# Configure benchmark
config = BenchmarkConfig(
    benchmarks=["mmlu", "hellaswag", "arc"],
    metrics=["accuracy"],
    max_samples_per_benchmark=100,
    output_dir="./benchmark_results"
)

benchmark = Benchmark(config)

# Run on single model
result = benchmark.run_benchmark(model, "gpt2-finetuned")

print(f"MMLU: {result.benchmark_scores['mmlu']['accuracy']:.1%}")
print(f"HellaSwag: {result.benchmark_scores['hellaswag']['accuracy']:.1%}")
print(f"ARC: {result.benchmark_scores['arc']['accuracy']:.1%}")
print(f"Overall: {result.overall_score:.1%}")

# Save results
result.save("benchmark_results.json")

Example 4: Model Comparison

from autotrain.evaluation import Benchmark

benchmark = Benchmark(config)

# Compare multiple models
models = {
    "gpt2-base": gpt2_model,
    "gpt2-finetuned": finetuned_model,
    "gpt2-distilled": distilled_model,
}

comparison_df = benchmark.compare_models(models)
print(comparison_df)

# Output:
#         model          mmlu  hellaswag    arc  overall
# 0   gpt2-base         0.45       0.52   0.48     0.48
# 1   gpt2-finetuned    0.58       0.65   0.61     0.61
# 2   gpt2-distilled    0.56       0.63   0.59     0.59

# Visualize
benchmark.plot_comparison(save_path="model_comparison.png")

Example 5: Training Callbacks

from autotrain.evaluation import PeriodicEvalCallback, BestModelCallback
from transformers import Trainer, TrainingArguments

# Create callbacks
eval_callback = PeriodicEvalCallback(
    evaluator=evaluator,
    eval_dataset=val_dataset,
    eval_steps=500,
)

best_model_callback = BestModelCallback(
    metric="perplexity",
    mode="minimize",
    save_path="./best_model"
)

# Training
training_args = TrainingArguments(
    output_dir="./output",
    evaluation_strategy="steps",
    eval_steps=500,
)

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=val_dataset,
    callbacks=[eval_callback, best_model_callback],
)

trainer.train()
# Automatically evaluates with full metrics every 500 steps
# Saves best model based on perplexity

Metric Selection Guide

Task	Recommended Metrics
Chat Model	Perplexity, ROUGE, BLEU
Summarization	ROUGE, BERTScore
Translation	BLEU, METEOR, BERTScore
QA	Exact Match, F1, ROUGE
Code Generation	Exact Match, BLEU
Classification	F1, Accuracy, Precision, Recall

When to Use Standard vs Enhanced Evaluation

Use Standard Evaluation When:

✓ Quick iteration during development
✓ Only care about loss
✓ Training time is primary concern
✓ No need for detailed metrics

Use Enhanced Evaluation When:

✓ Production deployment decisions
✓ Model comparison required
✓ Publishing results
✓ Need to explain model quality to stakeholders
✓ Debugging generation quality issues

CLI & API Integration

Current Status

Programmatic API: ✅ Fully Available All features accessible via Python imports:

from autotrain.trainers.clm.train_clm_distill import train_prompt_distillation  # Legacy
from autotrain.trainers.clm.params import LLMTrainingParams  # Integrated approach
from autotrain.rendering import MessageRenderer, get_renderer
from autotrain.generation import create_completer, create_chat_session
from autotrain.utils import HyperparameterSweep, run_autotrain_sweep  # Note: utils, not utils.sweep
from autotrain.evaluation import Evaluator, Benchmark

CLI Integration: ✅ Fully Available

All Tinker-inspired features are now accessible via CLI:

CLI Parameters for Tinker Features

Distillation (via SFT trainer)

python -m autotrain.cli.aitraining llm --train \
  --trainer sft \
  --use-distillation \
  --teacher-model gpt-3.5-turbo \
  --teacher-prompt-template "Complex prompt: {input}" \
  --student-prompt-template "{input}" \
  --distill-temperature 3.0 \
  --distill-alpha 0.7 \
  --model gpt2 \
  --data-path ./data \
  --project-name distilled_model

Hyperparameter Sweep

python -m autotrain.cli.aitraining llm --train \
  --use-sweep \
  --sweep-backend optuna \
  --sweep-n-trials 20 \
  --sweep-params '{"lr": [1e-5, 1e-3, "log_uniform"], "batch_size": [4, 8, 16]}' \
  --sweep-metric eval_loss \
  --sweep-direction minimize \
  --model gpt2 \
  --data-path ./data \
  --project-name sweep_run

Enhanced Evaluation

python -m autotrain.cli.aitraining llm --train \
  --use-enhanced-eval \
  --eval-metrics perplexity,bleu,rouge,bert_score \
  --eval-save-predictions \
  --eval-callback-period 500 \
  --model gpt2 \
  --data-path ./data \
  --project-name eval_run

PPO with TRL (Reinforcement Learning)

python -m autotrain.cli.aitraining llm --train \
  --trainer ppo \
  --rl-gamma 0.99 \
  --rl-kl-coef 0.1 \
  --rl-clip-range 0.2 \
  --rl-env-type text_generation \
  --rl-reward-model path/to/reward_model \
  --model gpt2 \
  --data-path ./data \
  --project-name ppo_run

Message Rendering (Chat Templates)

python -m autotrain.cli.aitraining llm --train \
  --chat-format chatml \  # or zephyr, alpaca, vicuna, llama, mistral
  --model gpt2 \
  --data-path ./data \
  --project-name chat_model

Inference Mode (Using Completers)

python -m autotrain.cli.aitraining llm \
  --inference-mode \
  --model my_finetuned_model \
  --inference-prompts prompts.txt \
  --inference-output results.json \
  --inference-max-tokens 100 \
  --inference-temperature 0.7 \
  --project-name inference_run

Programmatic Usage (Current Recommended Approach)

Python Script

# train_with_distillation.py
from autotrain.trainers.clm.train_clm_distill import (
    PromptDistillationConfig,
    train_prompt_distillation
)

config = PromptDistillationConfig(
    teacher_model_name="gpt-4",
    teacher_prompt_template="Complex prompt: {input}",
    student_model_name="gpt2",
    num_samples=1000,
)

trainer = train_prompt_distillation(
    config=config,
    base_inputs=data,
    output_dir="./output"
)

# Run it
python train_with_distillation.py

Jupyter Notebook

# Interactive exploration
from autotrain.generation import create_chat_session

session = create_chat_session("gpt2")
response = session.chat("Hello!")

API Deployment

# api.py
from fastapi import FastAPI
from autotrain.generation import create_completer, CompletionConfig
from autotrain.rendering import ChatFormat

app = FastAPI()

completer = create_completer(
    model="./my_model",
    completer_type="message",
    chat_format=ChatFormat.CHATML,
    config=CompletionConfig(max_new_tokens=100)
)

@app.post("/chat")
def chat(message: str):
    result = completer.complete(
        Conversation(messages=[Message("user", message)])
    )
    return {"response": result.content}

Future CLI Extensions

To add CLI support, extend LLMTrainingParams:

# In autotrain/trainers/clm/params.py
@dataclass
class LLMTrainingParams:
    # ... existing params ...

    # Distillation
    use_distillation: bool = False
    teacher_model: Optional[str] = None
    teacher_prompt_template: Optional[str] = None
    distill_temperature: float = 3.0
    distill_alpha: float = 0.7

    # Sweep
    use_sweep: bool = False
    sweep_backend: str = "optuna"
    sweep_trials: int = 10

    # Enhanced eval
    eval_metrics: Optional[List[str]] = None

Then use:

aitraining llm --train \
  --model gpt2 \
  --use-distillation \
  --teacher-model gpt-4 \
  --teacher-prompt-template "Be helpful: {input}"

Best Practices & When to Use

Decision Tree

Need to train a model?
│
├─ Expensive prompts? → Use Prompt Distillation
│  └─ 50-90% cost reduction
│
├─ Need optimal hyperparameters? → Use Hyperparameter Sweep
│  └─ 8-15% performance improvement
│
├─ Multiple chat formats? → Use Message Rendering
│  └─ 6 formats supported
│
├─ Building chat interface? → Use Completers
│  └─ Simple API, streaming support
│
└─ Need detailed metrics? → Use Enhanced Evaluation
   └─ 10+ metrics available

Common Workflows

Workflow 1: Production Chat Model

1. Train base model (AutoTrain)
   ↓
2. Optimize hyperparameters (Hyperparameter Sweep)
   ↓
3. Fine-tune with best params (AutoTrain)
   ↓
4. Evaluate comprehensively (Enhanced Evaluation)
   ↓
5. Deploy with Completers API

Workflow 2: Cost-Optimized Deployment

1. Develop complex prompt (manual)
   ↓
2. Distill to smaller model (Prompt Distillation)
   ↓
3. Evaluate quality (Enhanced Evaluation)
   ↓
4. Compare with baseline (Benchmark)
   ↓
5. Deploy if quality acceptable

Workflow 3: Research & Development

1. Experiment with formats (Message Rendering)
   ↓
2. Quick hyperparameter search (Random Sweep, 10 trials)
   ↓
3. Train best model
   ↓
4. Thorough evaluation (All metrics)
   ↓
5. Iterate based on weak points

Performance Considerations

Memory Usage

Feature	Memory Impact	Mitigation
Prompt Distillation	+50% (teacher + student)	Use PEFT, smaller teacher
Message Rendering	Minimal	-
Completers	Minimal	Use batch_size limit
Hyperparameter Sweep	1x per parallel job	Limit parallel_jobs
Enhanced Evaluation	+20% (multiple metrics)	Reduce batch_size

Speed Optimization

# Fast evaluation
config = EvaluationConfig(
    metrics=[MetricType.PERPLEXITY],  # Fastest metric
    batch_size=32,  # Larger batch
    max_samples=1000,  # Subsample
    fp16=True,  # Half precision
)

# Comprehensive but slow
config = EvaluationConfig(
    metrics=[
        MetricType.BLEU,
        MetricType.ROUGE,
        MetricType.BERTSCORE,  # Slowest
    ],
    batch_size=8,
    max_samples=None,  # All data
)

Common Pitfalls

Prompt Distillation: Using alpha=1.0 (pure KL) without CE loss → Poor generalization
- Fix: Use alpha=0.6-0.8
Hyperparameter Sweep: Too many trials on expensive training → Wasted compute
- Fix: Start with 5-10 trials, increase if needed
Message Rendering: Mixing formats between training and inference → Poor performance
- Fix: Consistent format throughout
Enhanced Evaluation: Running BERTScore on CPU → Very slow
- Fix: Use GPU or skip BERTScore for speed
Completers: Not using stop sequences → Model generates forever
- Fix: Always set stop_strings=renderer.get_stop_sequences()

Support & Troubleshooting

Getting Help

Check test files in tests/ for usage examples
Read docstrings in source code
Review test results in docs/TEST_RESULTS.md

Common Issues

Issue: Out of memory during distillation Solution: Enable use_peft=True, reduce batch size, use gradient accumulation

Issue: Sweep not improving Solution: Check parameter ranges, ensure objective function works, try different backend

Issue: Low evaluation scores Solution: Verify dataset format, check if model was actually fine-tuned, compare with baseline

Issue: Slow generation Solution: Use smaller batch size, enable fp16, reduce max_new_tokens

Device Management Utilities (NEW)

Centralized Device Configuration

AutoTrain now provides centralized utilities for consistent device management across all modules:

from autotrain.utils import get_model_loading_kwargs, maybe_move_to_mps

# Get consistent model loading configuration
kwargs = get_model_loading_kwargs(
    token="your_hf_token",
    fp16_if_cuda=True,  # Use FP16 on CUDA for memory efficiency
    trust_remote_code=True,
    extra_kwargs={"low_cpu_mem_usage": True}
)

# Load model with automatic device configuration
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("gpt2", **kwargs)

# Move to MPS if available (for Apple Silicon)
model = maybe_move_to_mps(model, kwargs)

Device Detection Logic

The utilities automatically handle:

CUDA (NVIDIA GPUs)
- Sets device_map="auto" for multi-GPU support
- Uses FP16 by default (configurable)
- Enables memory optimization
MPS (Apple Silicon)
- Detects M1/M2/M3 chips
- Uses FP32 (required for MPS)
- Moves model after loading
CPU Fallback
- Uses FP32
- No device_map
- Memory-efficient loading

Usage in Custom Training

from autotrain.utils import get_model_loading_kwargs, maybe_move_to_mps
from transformers import AutoModelForCausalLM, AutoTokenizer

# Consistent loading across all devices
def load_model_and_tokenizer(model_name: str):
    # Get device-specific configuration
    kwargs = get_model_loading_kwargs(
        fp16_if_cuda=True,  # FP16 for CUDA
        trust_remote_code=True
    )

    # Load model
    model = AutoModelForCausalLM.from_pretrained(model_name, **kwargs)

    # Handle MPS if needed
    model = maybe_move_to_mps(model, kwargs)

    # Load tokenizer
    tokenizer = AutoTokenizer.from_pretrained(model_name)

    return model, tokenizer

# Works on any device
model, tokenizer = load_model_and_tokenizer("gpt2")

Benefits

Consistency: Same loading logic across all modules
Automatic Optimization: Chooses best settings per device
Memory Efficiency: FP16 on CUDA, proper device_map
MPS Support: Full Apple Silicon compatibility
No Code Duplication: Single source of truth

Changelog & Versions

Version 1.1.0 (October 2, 2025 - Refactoring Update)

✅ All features now accessible via CLI
✅ Centralized device management utilities
✅ PPO now uses TRL implementation
✅ Distillation integrated into SFT trainer
✅ Message rendering integrated with process_data_with_chat_template
✅ Sweep functionality moved to autotrain.utils (package structure)
✅ Fixed unified chat rendering (render_conversation method)
✅ ~1,150 lines of duplicate code eliminated
✅ Full backward compatibility maintained via shim modules

Version 1.0.0 (October 2, 2025)

✅ Initial release
✅ 5 major features
✅ 153 tests passing
✅ Full MPS/CUDA/CPU support

License & Attribution

Part of AutoTrain Advanced by Hugging Face Enhanced with features inspired by Tinker (Mira Murati)

For questions or contributions, see the main AutoTrain repository.

Uh oh!

FilesExpand file tree

PYTHON_API.md

Latest commit

History

PYTHON_API.md

File metadata and controls

AutoTrain Advanced - Tinker-Inspired Features Documentation

Table of Contents

Overview

What's New

Feature Comparison with Standard AutoTrain

Evaluation: Standard vs Enhanced

Standard AutoTrain Evaluation

Enhanced Evaluation (NEW)

Training: Standard vs Prompt Distillation

Standard Fine-Tuning

Prompt Distillation (NEW)

Hyperparameter Tuning: Standard vs Sweep

Standard AutoTrain

Hyperparameter Sweep (NEW)

1. Prompt Distillation

Concept

Architecture

API Reference

Option 1: Integrated SFT Distillation (NEW - Recommended)

Option 2: Standalone Distillation (Legacy)

Usage Examples

Example 1: Distill GPT-4 Chain-of-Thought to GPT-2

Example 2: Distill Domain-Specific Expertise

Parameters Explained

temperature

alpha

use_peft (LoRA)

When to Use Prompt Distillation

Common Issues & Solutions

2. Message Rendering System

Concept

Supported Formats

API Reference

Core Classes

MessageRenderer

Usage Examples

Example 1: Basic Conversation Rendering

Example 2: Token-Level Weight Control

Example 3: Multi-Format Dataset Preparation

Example 4: Generation Prompts

When to Use Each Format

Advanced: Token-Level Loss Control

Best Practices

3. Completers Interface

Concept

Architecture Levels

API Reference

CompletionConfig

TokenCompleter

MessageCompleter

Usage Examples

Example 1: Simple Chat Interface

Example 2: Token-Level Generation (for RL)

Example 3: Batch Processing

Example 4: Async Generation

Sampling Strategies

When to Use Token vs Message Completer

4. Hyperparameter Sweeps

Concept

Supported Backends

API Reference

SweepConfig

ParameterRange

HyperparameterSweep

Usage Examples

Example 1: Optuna-Based Sweep

Example 2: Grid Search for Systematic Exploration

Example 3: Random Search for Quick Exploration

Example 4: AutoTrain Integration

Parameter Types Explained

Log Uniform

Float

Int

`temperature`

`alpha`

`use_peft` (LoRA)

`MessageRenderer`

`CompletionConfig`

`TokenCompleter`

`MessageCompleter`

`SweepConfig`

`ParameterRange`

`HyperparameterSweep`

`EvaluationConfig`

`Evaluator`

`Benchmark`