Sniper Architecture

[ramonasuncion]

Both Greptile and CodeRabbit use LLMs, but you can't dump 50K lines of code into a prompt.

• RAG: Converts code into chunks and stores them as vectors. When a PR comes in, the system searches for the most relevant chunks.
• Knowledge Graphs: Instead of just text search, this uses Tree-sitter to parse code into an AST so the AI knows exactly where "Function X" is defined.
• LSP Integration: Uses the Language Server Protocol to track definitions and references across the whole project.

The setup needs a GitHub app, a backend webhook, and the GitHub API to post comments on specific lines. It also needs a way to parse git diffs.

The Code Reviewer logic:

• Summarizes the PR intent.
• Runs security and logic scans for things like SQL injection.
• Uses a second AI agent to validate the first one and prevent hallucinations.

The stack:

• ChromaDB for vectors.
• Tree-sitter for parsing.
• LangChain for orchestration.
• Python and Elixir for the backend.

The workflow:

1. User opens a PR.
2. App pulls the git diff.
3. If a function changed, it grabs related context from the vector store.
4. Diff + context + PR info goes to the LLM.
5. LLM spits out line numbers and comments.

Starting with a CLI tool. It’ll just be git diff -> parse -> spawn task -> print terminal suggestions.

Elixir is perfect for the reviewer part. You can have separate workers for snippets, leak detection, and logic. If the security worker crashes, the supervisor tree just restarts it.

I'll use Groq for tool calling because it's fast and cheap. This lets the LLM read files or make commits. I can also mix models: a fast 8B model

[changminbark]

This design doc seems good. Just a few questions though:

1. How are you going to integrate elixir with python (there is an Elixir version of LangChain here, but I am not sure how robust/extensive it is)?
2. How are you going to break up the chunks for RAG?
3. How are the knowledge graphs/AST going to be used?
4. Are you going to support any programming language or just stick to common ones first?

[ramonasuncion]

1. Elixir will just create workers, since I really hate doing concurrency in Python because of the GIL.
2. Probably use cAST: Research Paper and Code
3. Knowledge graphs basically give the LLM a context of how something in one file is connected to a totally different part of the codebase. So they're not just snippets of code that we give it, it's more like, "Here is the bigger picture".
4. Yeah, probably Python, Java, C#, and TS (that's what's fully supported in that source code from the paper). I think that's the same for Greptile, they only support mainstream languages.

[ramonasuncion]

@changminbark I added most of the issues except for knowledge graphs since I haven't done enough research. I understand the high level and that there is a graph database called Neo4j.

All I understand is that instead of working with just snippets of code the LLM can understand defining relationships how does something in User.py change/affect Auth.ts.

I'll start working on the #3 and then after that you can start working on any of the tasks.

[changminbark]

@ramonasuncion Another idea that I thought of (but let's finish this project first) would be a QA/code testing AI agent. Maybe we can create a collection of these AI agents. Wdyt?

[ramonasuncion]

Are you thinking of something like this https://www.testsprite.com/ and https://www.testmu.ai/kane-ai/? I really like the idea. Like an AI agent being able to open up a browser or run docker containers.

Imagine we just have a swarm of agents doing the mundane tasks. hopefully we don't get roasted though once we share this. https://x.com/jarrodwatts/status/2009221830436180432

[changminbark]

Yes, but we should focus on finishing sniper first.

[ramonasuncion]

Lol for sure. 😄

[changminbark]

@ramonasuncion Actually, after thinking about the design doc a little more, I am kind of confused about the role of elixir here.

Can you explain what you mean by "create workers"?

Will it manage processes that are responsible for different tasks (like indexing and reviewing)?

Will all of these workers then communicate with the underlying Python backend through the PythonBridge GenServer (which will talk to the Python bridge.py router/handler that handles these messages)?

Is communication via this bridge bi-directional? (I ask this because I am also unfamiliar with how GitHub apps work -> will it only have access to the Elixir interface or also interact with the Python backend directly?)

[ramonasuncion]

Basically Elixir will spawn separate process for different tasks from the bridge.py e.g. SQL injection, secrets api keys, checking logic... etc. Why I love is Elixir is that each one is isolated so if one crashes, the supervisor just restarts it.

So, yes, Elixir manages the different processes for indexing and reviewing etc. Currently we don't have to worry about it too much. It's a straight pipeline #19, but I'm setting up the arch now because we can easily parallelize things later. I'll be cool to have multiple security scans, check multiples files in parallel, that's the vision I have.

Yep exactly that. Workers sends message to PythonBridge GenServer then that forwards the results to bridge.py (python router). Python does the heavy lifting (LangChain, ChromaDB, Treesitter, LLM calls, etc), then results the results back through the same bridge (Bidirectional).

I'm not too sure with GitHub apps either. The little I know is that app receives a webhook and posts comments through the Github API? I'm thinking this might just all be handled through Elxir so that python never touches Github just handles AI tasks.

TLDR: Python does all the AI stuff and Elixir handles concurrency, supervision, event processing. I'm doing this because I hate Python Gil for parallel work and I don't think LangChain in Elixir is mature enough yet. So just a little bit of work so we don't have to rewrite everything later when it scales.

[ramonasuncion]

I was about to do all this in Elixir but this is my compromise since Python is just superior in every other way lol.

[changminbark]

So will we have multiple python processes running? If we end up having one bridge, then wouldn't we still be limited by the python GIL?

[ramonasuncion]

nah luckily the GIL only affects threads in a single python process. each one will gets it's own independent execution.