Skip to content

Releases: Design-Arena/MicroEvals

Release list

Initial Release

Initial Release Latest
Latest

Choose a tag to compare

View all tags
@kamrynohly kamrynohly released this 10 Nov 23:33
v0.1.0
87982b3
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

MicroEvals v0.1.0

Initial release of MicroEvals - an automated evaluation framework for AI-generated code quality.

Overview

MicroEvals provides a lightweight, extensible framework for evaluating codebases against specific criteria grounded in real-world agent failures. Born from observing over 240,000 developers building on Design Arena, each evaluation targets authentic failure modes that agents encounter in production—not synthetic or contrived benchmarks.

Using Claude Code as an LLM judge, MicroEvals assesses code quality, best practices, and framework-specific patterns with focused, reproducible tests. Each eval is carefully selected for being both common enough to surface in real-world use cases and difficult enough to provide discriminating signal.

Learn more about the methodology: Introducing Micro Evals

Features

Core Functionality

  • CLI Tool: Run evaluations directly from the command line using microeval
  • Local and Remote Repositories: Evaluate both local directories and GitHub repositories
  • Extensible Eval System: YAML-based evaluation definitions for easy customization
  • Multiple Frameworks: Pre-built evaluations for Next.js, React, Tailwind, TypeScript, Supabase, and more

Evaluation Categories

Built on foundations like Vercel's next-evals-oss, refined for real-world agent development:

  • Next.js: Server components, client components, routing, data fetching patterns (25+ evaluations)
  • React: Hooks, state management, event handlers, best practices
  • Tailwind CSS: Configuration, dynamic classes, v4 migration
  • TypeScript: Type safety, null checks, assertions
  • Supabase: Authentication, database operations, RLS policies
  • Shadcn/ui: Component setup, configuration, dependencies

Each evaluation produces a binary pass/fail outcome with detailed breakdowns of what succeeded and what didn't.

Installation

pip install microevals

Quick Start

# Run a specific evaluation
microeval --eval-id nextjs/001-server-component --repo /path/to/repo

# List available evaluations
microeval --list

# Run batch evaluations
microeval --batch --framework nextjs --repo /path/to/repo

Requirements

  • Python 3.8 or higher
  • Uses Claude Code for LLM-based evaluation

Documentation

Full documentation and evaluation catalog available at: https://github.com/Design-Arena/MicroEvals

What's Next

Future releases will include:

  • Additional framework evaluations
  • Custom evaluation creation tools
  • Performance improvements
  • Enhanced reporting capabilities

Contributors

License

MIT License - see LICENSE file for details

Assets 2
Loading