Skip to content

Add Conformer audio encoder as default architecture - #15

Merged
asfilion merged 3 commits into
mainfrom
feature/conformer-encoder
Feb 17, 2026
Merged

Add Conformer audio encoder as default architecture#15
asfilion merged 3 commits into
mainfrom
feature/conformer-encoder

Conversation

@asfilion

Copy link
Copy Markdown
Owner

Summary

  • Add Conformer encoder (Gulati et al., 2020) combining self-attention with depthwise convolutions to capture both global and local temporal audio patterns
  • Three new classes: ConformerFeedForward, ConformerConvModule, ConformerBlock in transformer.py
  • AudioEncoder branches on use_conformer config flag (default True), transparent to decoder
  • Tune medium configs: LR 1e-4 → 3e-5, warmup 500 → 1000 (fixes val loss divergence with larger models)
  • New configs/medium_conformer.json (9.4M params, fits 8GB VRAM at batch_size=8)
  • 9 new tests in TestConformer class (178 total tests passing)
  • Updated all project docs (CLAUDE.md, README.md, RESEARCH.md, CODEBASE_REFERENCE.md)

Training results (medium conformer, 7 epochs so far)

Epoch Val Loss Val Acc
1 3.277 37.8%
4 2.530 52.1%
7 2.396 53.6%

Stable training with no divergence. On track to surpass the small model baseline (60.6% at epoch 16).

Test plan

  • All 178 tests pass (python -m pytest tests/ -v)
  • Conformer output shape, padding, sinusoidal PE, backward pass, onset features, conv module, param count tested
  • Existing encoder/decoder tests unchanged and passing
  • Medium conformer config instantiates and reports 9,422,371 params
  • Training completes and surpasses baseline accuracy

🤖 Generated with Claude Code

asfilion and others added 3 commits February 16, 2026 11:08
Replace pure transformer encoder with Conformer blocks (Gulati et al., 2020)
that combine self-attention with depthwise convolutions to capture both global
and local temporal patterns in audio. Adds ConformerFeedForward, ConformerConvModule,
and ConformerBlock classes behind a use_conformer config flag (default True).
Medium conformer config: 9.4M params, fits 8GB VRAM at batch_size=4.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previous LR (1e-4) caused val loss instability with the 6.5M+ param models —
train loss dropped but val loss exploded after epoch 4. Lower LR and longer
warmup should stabilize training.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- CLAUDE.md: Conformer architecture details, 3 training result sections
  (small baseline, medium transformer divergence, medium conformer progress),
  updated config descriptions and test counts
- README.md: Conformer encoder in architecture, updated project status
- RESEARCH.md: New Conformer section with full architecture, param counts,
  training comparison, and key findings (LR scaling, stability)
- CODEBASE_REFERENCE.md: Updated transformer.py description, added
  medium_conformer.json, updated test counts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@asfilion
asfilion merged commit 699b724 into main Feb 17, 2026
4 checks passed
@asfilion
asfilion deleted the feature/conformer-encoder branch February 17, 2026 00:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant