A curated list of awesome responsible machine learning resources.

AI Agent Governance Toolkit — Policy enforcement, zero-trust identity, execution sandboxing, and reliability engineering for autonomous AI agents. Covers 10/10 OWASP Agentic Top 10.

Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback

Secrets of RLHF in Large Language Models Part I: PPO

UQLM: Uncertainty Quantification for Language Models, is a Python package for UQ-based LLM hallucination detection

This is a repository that aims to provide updates on the status of jailbreaking the OpenAI GPT language model.

Internal Safety Collapse: Turning the LLM or an AI Agent into a sensitive data generator.

A resource repository for machine unlearning in large language models

Deliver safe & effective language models

PromptInject is a framework that assembles prompts in a modular fashion to provide a quantitative analysis of the robustness of LLMs to adversarial prompt attacks. 🏆 Best Paper Awards @ NeurIPS ML Safety Workshop 2022

The open agent control plane. Govern autonomous AI agents with pre-execution policy enforcement, approval gates, and audit trails. Works with LangChain, CrewAI, MCP, and any framework.

The open-source diagnostic for AI misalignment. 32 tests across fabrication, manipulation, deception, unpredictability, and opacity. Provider-agnostic. Runs against OpenAI, Anthropic, Bedrock, Azure, Gemini, and more. Letter grade in under 5 minutes, content-addressed manifest for bit-identical replay. Built by iMe.

Stop AI agents from doing things you didn't ask for.

Open Source LLM toolkit to build trustworthy LLM applications. TigerArmor (AI safety), TigerRAG (embedding, RAG), TigerTune (fine-tuning)

Runtime policy enforcement for AI agents. Cryptographic audit trail, human-in-the-loop approvals, kill switch. Zero code changes.

Measuring how well CLI agents like Claude Code or Codex CLI can post-train base LLMs on a single H100 GPU in 10 hours

Aligning AI With Shared Human Values (ICLR 2021)

🦀 Prevents outdated Rust code suggestions from AI assistants. This MCP server fetches current crate docs, uses embeddings/LLMs, and provides accurate context via a tool call.

A self-hosted AI infrastructure for private RAG and multi-model applications.

[NeurIPS '23 Spotlight] Thought Cloning: Learning to Think while Acting by Imitating Human Thinking