Evidence-first infrastructure for reproducible, isolated executable evaluation and online rewards.
-
Updated
Sep 2, 2026 - Python
#
llm-infrastructure
Here are 94 public repositories matching this topic...
Efficient LLM inference on Slurm clusters.
inference speech-to-text vlm text-embedding multimodal audio-transcription llm vllm reward-model llm-infernece sglang llm-infrastructure
-
Updated
Sep 8, 2026 - Python
Phi-Bench (Φ-Bench): 85 open-source LLM-infrastructure engineering tasks for frontier LLMs & coding agents (KFC/LH/E2E). Self-contained public Dockerfile + offline scoring + reference solution per task. Website: llminfrabench.com
-
Updated
Sep 9, 2026 - Python
PipelineLLM 是一个系统性的大语言模型（LLM）后训练学习项目，涵盖从监督微调（SFT）到偏好优化（DPO）、强化学习（RLHF/PPO/GRPO）再到持续学习（Continual Learning)的完整技术栈。
reinforcement-learning lora fine-tuning post-training continual-learning sft rlhf llm-reasoning preference-optimization llm-infrastructure llm-processing
-
Updated
Jan 16, 2026 - Python
A practical, multi-layered JSON repair library for Elixir that intelligently fixes malformed JSON strings commonly produced by LLMs, legacy systems, and data pipelines.
json elixir otp functional-programming data-validation beam json-parser erlang-vm error-recovery data-quality json-validation json-repair json-fix llm-infrastructure malformed-json nshkr-ai-infra
-
Updated
Apr 4, 2026 - Elixir
Multi-model AI agent runtime. Define agents in YAML, route each role to a model, orchestrate with 7 patterns (ReAct, Plan & Execute, Fan-Out, Pipeline, Supervisor, Swarm, Glyph), and deploy as a REST/WebSocket API with RAG, memory, MCP tools, guardrails and OpenTelemetry observability.
open-source scalability developer-tools multi-model observability multi-agent-systems ai-agents rag ai-platform vector-database agent-framework llm generative-ai llm-ops agentic-ai ai-infrastructure agent-orchestration agent-runtime llm-infrastructure developer-tools-ai-agent
-
Updated
Sep 8, 2026 - Python
AI Workload Control Layer for routing deterministic, reusable, retrieval-needed, tool-needed, and provider-needed work before model invocation.
python open-source task-graph ai-infrastructure tool-routing llm-infrastructure deterministic-routing workload-routing ai-workload-control retrieval-routing
-
Updated
Sep 10, 2026 - Python
A browser-based UI for launching, monitoring, clustering and managing multiple llama.cpp server instances from inside a Docker container. Includes an Ollama-compatible API proxy
frontend clustering rest-api proxy llm llamacpp llm-inference llm-proxy llm-manager llm-infrastructure
-
Updated
Sep 8, 2026 - Python
Reliability control for the Anthropic Python SDK
python reliability retries resilience claude opentelemetry llm anthropic llm-infrastructure retry-budget
-
Updated
Jul 20, 2026 - Python
Persistent Cognitive Memory Infrastructure — durable, multi-tenant memory layer for AI agents. HTTP + gRPC, hybrid retrieval, background workers, observability. Go.
go golang multi-tenant memory grpc ai-agents rag vector-database apache-age pgvector agent-memory llm-infrastructure
-
Updated
Sep 7, 2026 - Go
A production-grade, schema-aware PostgreSQL MCP server for enterprise AI. Features Zero-Trust SQL validation, multi-tier permissions, and real-time schema introspection for secure, autonomous database operations.
postgresql ai-agents zero-trust database-security sql-validation model-context-protocol database-agent llm-infrastructure
-
Updated
Mar 25, 2026 - TypeScript
nvProbe — Open-source NVIDIA GPU benchmark suite for CUDA workload automation, Slurm HPC cluster profiling, and MLPerf reporting
benchmarking benchmark hpc gpu cuda slurm nvidia slurm-cluster nvidia-gpu cuda-benchmark gpu-benchmark mlperf llm-infrastructure hpc-benchmarking
-
Updated
Sep 9, 2026 - Python
Krako 2.0 – Energy-efficient, triadic multi-tier inference infrastructure enabling adaptive routing across heterogeneous edge–cloud nodes.
distributed-computing multi-agent-systems inference-engine edge-computing cost-optimization edge-cloud adaptive-routing heterogeneous-compute decentralized-ai ai-infrastructure task-decomposition llm-infrastructure triadic-inference krako
-
Updated
Feb 25, 2026 - Python
A humble exploratory PoC for a hardware-native, optical timing-frozen control plane engine. Fuses inline CUDA PTX, RAII memory tunnels, and 4D JAX shard_map structures to investigate 0ns-overhead fault-tolerant routing for hyperscale distributed AI.
distributed-systems fault-tolerance zero-copy cuda high-performance-computing compiler-optimization interconnect silicon-photonics jax pytorch-extension xla custom-kernels hardware-software-co-design ahead-of-time-compilation ai-infrastructure llm-infrastructure deepseek-v4 ptx-assembly hlo-compiler
-
Updated
Aug 9, 2026 - Python
One command. Full LLM stack. Zero config.
docker self-hosted developer-tools cli-tool privacy-first openai-api generative-ai langchain local-llm llm-infrastructure
-
Updated
Jul 27, 2026 - Python
An intelligent gateway for Claude APIs that dynamically routes requests to the most cost-efficient model, caches responses, and escalates based on confidence signals — reducing LLM spend without sacrificing quality.
api-proxy observability claude cost-optimization generative-ai llmops prompt-caching claude-api ai-gateway agentic-ai llm-gateway llm-routing inference-gateway semantic-caching llm-infrastructure
-
Updated
May 6, 2026 - Python
Forward-Only Homeostasis Accelerator Kernel for 0ns Distributed Overlapping & Static O(1) VRAM Memory Wall Liquidation via JAX/XLA & PyTorch.
hpc zero-copy distributed-computing cuda pytorch high-performance-computing computer-architecture control-theory fluid-dynamics forward-only autonomous-systems jax inference-optimization xla memory-optimization neural-differential-equations non-linear-dynamics pinns llm-infrastructure
-
Updated
Aug 24, 2026 - Python
High-performance Triton kernels for NVIDIA H100. Implements fused FP8 LayerNorm, tiled FlashAttention, and SRAM-optimized memory primitives for Hopper architecture.
-
Updated
Sep 1, 2026 - Python
Fail-fast admission control for LLM workloads, with concurrency limits, token budgeting, deduplication, streaming support, and overload protection.
nodejs typescript latency reliability fail-fast cost-control concurrency-control backpressure bulkhead load-shedding admission-control llm overload-protection ai-infrastructure llm-infrastructure
-
Updated
Sep 3, 2026 - TypeScript
Add this topic to your repo
To associate your repository with the llm-infrastructure topic, visit your repo's landing page and select "manage topics."