Claude Code on Amazon EC2 with a Self-Hosted Open-Source Model

This is sample code intended for demonstration and learning purposes only. It is not meant for production use. Review and harden all scripts, configurations, and IAM permissions before using in any production or sensitive environment.

Run Claude Code — Anthropic's AI coding assistant — backed by an open-source model on Amazon EC2. Your code stays inside your AWS account, costs are predictable, and you keep full control over the model.

Your Machine
  │
  │  SSH tunnel (encrypted, no open ports needed)
  │
  ▼
Amazon EC2 GPU Instance (g6e.xlarge)
  Open-source model (Qwen 3.5-35B) via Ollama
  NVIDIA L40S, 45GB VRAM — runs fully within your VPC

Why Run on EC2?

	Cloud API (Anthropic / Amazon Bedrock)	Self-Hosted on EC2
Pricing	Per token — variable cost	Fixed hourly EC2 rate
Data residency	Sent to external API	Stays in your AWS account
Model choice	Provider-managed	Any open-source model
Network path	Public internet	Within your VPC
Compliance controls	Limited	Full — IAM, VPC, CloudTrail

Heavy coding sessions with many tool calls (file reads, edits, bash commands) add up fast on pay-per-token APIs. At ~117 tokens/sec on a g6e.xlarge, a self-hosted model is cost-effective for sustained daily use.

Choosing an EC2 Instance

Instance	GPU	VRAM	On-Demand	Best For
g6e.xlarge	NVIDIA L40S	45GB	~$1.86/hr	35B models, recommended
g5.xlarge	NVIDIA A10G	24GB	~$1.01/hr	7B-13B models
g4dn.xlarge	NVIDIA T4	16GB	~$0.53/hr	7B models, budget option
p3.2xlarge	NVIDIA V100	16GB	~$3.06/hr	Older generation

Prerequisites

• AWS account with EC2 access
• AWS CLI configured (aws configure)
• Claude Code installed locally (npm install -g @anthropic-ai/claude-code)
• An EC2 key pair

What's Inside

File	What it does
scripts/ec2-setup.sh	One-shot EC2 setup: installs Ollama, pulls model, verifies GPU
scripts/tunnel.sh	Opens and closes the SSH tunnel between your machine and EC2
scripts/claude-local.sh	Launches Claude Code pointed at the local model, restores config on exit
scripts/bench.sh	Benchmarks local model vs Amazon Bedrock side by side
BENCHMARK.md	Full benchmark guide: tasks, verification, and result interpretation
config/settings.template.json	Claude Code configuration template
SETUP-GUIDE.md	Advanced walkthrough using llama.cpp directly (more tuning control)

Quick Start

Step 1 — Launch the EC2 instance

Use the Deep Learning Base OSS Nvidia Driver AMI (Ubuntu 22.04) — NVIDIA drivers and CUDA come pre-installed.

aws ec2 run-instances \
  --region us-east-1 \
  --image-id ami-014135eb43056a305 \
  --instance-type g6e.xlarge \
  --key-name <your-key-pair> \
  --security-group-ids <your-sg> \
  --block-device-mappings '[{"DeviceName":"/dev/sda1","Ebs":{"VolumeSize":100,"VolumeType":"gp3"}}]' \
  --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=llm-server}]'

Security group: port 22 inbound from your IP only. The model port never needs to be open — access is via SSH tunnel.

Step 2 — Set up the model server on EC2

SSH in and run the setup script:

curl -fsSL https://raw.githubusercontent.com/shekharprateek/claude-code-on-amazon-ec2/main/scripts/ec2-setup.sh | bash

Or clone and run:

git clone https://github.com/shekharprateek/claude-code-on-amazon-ec2 ~/claude-code-on-amazon-ec2
bash ~/claude-code-on-amazon-ec2/scripts/ec2-setup.sh

This installs Ollama, pulls Qwen 3.5-35B (~22GB on first run), and verifies the model is running on GPU. To use a smaller model: MODEL=qwen3.5:7b bash ec2-setup.sh

Step 3 — Connect from your machine

export G6E_IP=<your-ec2-public-ip>
export G6E_KEY=~/.ssh/<your-key>.pem
./scripts/tunnel.sh start

Step 4 — Run Claude Code

./scripts/claude-local.sh

Or run /install inside Claude Code for a guided setup experience.

See SETUP-GUIDE.md for the full walkthrough including GPU monitoring, spot instance setup, and cost optimization.

Cost Optimization

Stop when not in use — model weights persist on EBS and reload in seconds:

aws ec2 stop-instances --instance-ids <instance-id>
aws ec2 start-instances --instance-ids <instance-id>

Spot instances save up to 70% for interruptible workloads:

aws ec2 request-spot-instances \
  --instance-count 1 \
  --type one-time \
  --launch-specification \
    '{"ImageId":"ami-014135eb43056a305","InstanceType":"g6e.xlarge","KeyName":"<key>","SecurityGroupIds":["<sg>"]}'

Monitor GPU Usage

Live on the instance:

nvidia-smi --query-gpu=utilization.gpu,memory.used,temperature.gpu \
  --format=csv --loop=1

Benchmark vs Amazon Bedrock

Run 9 coding tasks against both backends and compare quality and speed:

./scripts/bench.sh both

9 tasks across 3 tiers (simple → medium → complex), each with an automated pass/fail verifier. See BENCHMARK.md for the full task descriptions, how verification works, and how to read the results.

Tear Down

aws ec2 terminate-instances --region us-east-1 --instance-ids <instance-id>
aws ec2 delete-security-group --region us-east-1 --group-id <sg-id>

See Also

• Claude Code Multi-Model on Amazon Bedrock — Run Claude Code with 43 models from 12 providers (Qwen, DeepSeek, Mistral, Kimi, MiniMax, and more) via Bedrock Mantle. No GPU needed — fully managed.