Skip to content

Commit d080195

Browse files
ShashankpgitShashankpgit
committed
updated plan
1 parent 5122b2c commit d080195

13 files changed

Lines changed: 1997 additions & 764 deletions

File tree

‎claude-instructions/README.md‎

Lines changed: 141 additions & 135 deletions
Large diffs are not rendered by default.

‎claude-instructions/debugging-philosophy.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,47 @@ The user will be told: **"This is a deliberate mistake — let's debug it."**
107107

108108
---
109109

110+
---
111+
112+
## Planned deliberate mistakes — DevOps phases
113+
114+
### Phase 10 — K8s Step 1 (Concepts)
115+
**Mistake**: Use `vault-api:latest` as the image name in deployment.yaml without importing it into k3d.
116+
**Error**: `ImagePullBackOff` or `ErrImagePull` on `kubectl get pods`
117+
**Lesson**: K8s clusters have their own image storage. Local Docker images are not automatically available inside k3d. Either import with `k3d image import` or push to a registry.
118+
119+
### Phase 10 — K8s Step 2 (Full Deployment)
120+
**Mistake**: Use `Deployment` instead of `StatefulSet` for PostgreSQL.
121+
**Error**: App appears to work, but after `kubectl delete pod vault-db-xxx`, data may be lost or the PVC may not reattach cleanly.
122+
**Lesson**: Deployment pods are anonymous and interchangeable. StatefulSet pods have stable identity (`postgres-0`) and stable storage binding — required for databases.
123+
124+
### Phase 10 — K8s Step 3 (Ingress + Probes)
125+
**Mistake**: Set `initialDelaySeconds: 0` on the liveness probe for Keycloak.
126+
**Error**: `CrashLoopBackOff` — K8s kills Keycloak repeatedly because it fails the liveness check during startup.
127+
**Lesson**: Health probe timing must match actual startup time. Keycloak takes ~60-90 seconds to start. Aggressive probes on slow-starting services cause restart loops.
128+
129+
### Phase 11 — Helm
130+
**Mistake**: Run `helm install vault ./helm/vault` twice without checking if a release already exists.
131+
**Error**: `Error: INSTALLATION FAILED: cannot re-use a name that is still in use`
132+
**Lesson**: `helm install` fails if a release already exists. Use `helm upgrade --install` instead — it installs if absent, upgrades if present. This is what CI/CD pipelines use.
133+
134+
### Phase 12 — Container Registry
135+
**Mistake**: Tag the image as `vault-api:latest` (without registry prefix) and try to push to ghcr.io.
136+
**Error**: `denied: requested access to the resource is denied`
137+
**Lesson**: Docker uses the image name to determine the registry. `vault-api:latest` tries Docker Hub. `ghcr.io/username/vault-api:latest` targets GitHub. The registry is part of the tag.
138+
139+
### Phase 12 — CI/CD
140+
**Mistake**: Missing `needs: test` on the `build` job in the GitHub Actions workflow.
141+
**Error**: The pipeline builds and deploys even when tests fail.
142+
**Lesson**: GitHub Actions jobs run in parallel by default. `needs: test` makes `build` wait for `test` to succeed. Without it, a broken deploy can ship despite failing tests.
143+
144+
### Phase 13 — Monitoring
145+
**Mistake**: Configure Prometheus to scrape `localhost:8000/metrics` instead of `api:8000/metrics`.
146+
**Error**: Prometheus shows `connection refused` for the vault-api target.
147+
**Lesson**: In K8s, `localhost` means the Prometheus pod itself, not the API. Service names (DNS) must be used for cross-pod communication: `api.vault.svc.cluster.local` or simply `api` within the same namespace.
148+
149+
---
150+
110151
## How to use DevTools for debugging (reference)
111152

112153
When the error is frontend-side, always open DevTools FIRST:

‎claude-instructions/phase-10/01-k8s-concepts.md‎

Lines changed: 342 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,342 @@
1+
# Phase 10 — K8s Concepts + Minikube Setup
2+
3+
## What this phase covers
4+
5+
Pure concepts and setup. No application deployment yet. By the end you will:
6+
- Understand what Kubernetes is and why it exists
7+
- Know every K8s object you will use in Phase 13
8+
- Have minikube running and kubectl working
9+
- Have run a single practice pod to see kubectl in action
10+
11+
This is the foundation. Rushing past it means you'll be copy-pasting YAML without understanding what any of it does.
12+
13+
---
14+
15+
## Pre-step checklist
16+
17+
- [ ] Create `docs/plans/018-k8s-concepts-plan.md` first
18+
- [ ] Teach ALL concepts before writing any YAML
19+
- [ ] Docker must be installed and running (minikube uses Docker as driver)
20+
21+
---
22+
23+
## Why Kubernetes at all — the honest explanation
24+
25+
docker-compose solves: "run multiple containers together on one machine."
26+
27+
Kubernetes solves: "run containers reliably across many machines, automatically restart failures, deploy new versions without downtime, and scale up/down on demand."
28+
29+
In production, a single machine is a single point of failure. If that machine dies, your app dies. K8s runs your containers across a **cluster** of machines. If one machine dies, K8s moves your containers to another machine automatically — without you doing anything.
30+
31+
The tradeoff: K8s is significantly more complex than docker-compose. The complexity pays off at scale. For a single-developer project, docker-compose is often the right choice. We're learning K8s because it's the industry standard for production deployments.
32+
33+
---
34+
35+
## K8s Architecture (control plane + worker nodes)
36+
37+
```
38+
┌─────────────────────────────────────────┐
39+
│ Control Plane (the "brain") │
40+
│ │
41+
│ ┌──────────┐ ┌──────────┐ │
42+
│ │ API │ │ etcd │ │
43+
│ │ Server │ │ (state) │ │
44+
│ └──────────┘ └──────────┘ │
45+
│ ┌──────────┐ ┌──────────┐ │
46+
│ │Scheduler │ │Controller│ │
47+
│ │ │ │ Manager │ │
48+
│ └──────────┘ └──────────┘ │
49+
└─────────────────────────────────────────┘
50+
│ kubectl talks to API Server
51+
┌───────▼────────┐ ┌────────────────┐
52+
│ Worker Node 1 │ │ Worker Node 2 │
53+
│ ┌──────────┐ │ │ ┌──────────┐ │
54+
│ │ kubelet │ │ │ │ kubelet │ │
55+
│ └──────────┘ │ │ └──────────┘ │
56+
│ ┌───┐ ┌───┐ │ │ ┌───┐ ┌───┐ │
57+
│ │Pod│ │Pod│ │ │ │Pod│ │Pod│ │
58+
│ └───┘ └───┘ │ │ └───┘ └───┘ │
59+
└────────────────┘ └────────────────┘
60+
```
61+
62+
- **API Server**: Every command you run with `kubectl` hits the API server. It's the single entry point to the cluster.
63+
- **etcd**: The cluster's database. Stores the desired state of every object. "I want 3 replicas of the API pod." etcd holds that.
64+
- **Scheduler**: Decides which worker node a new pod should run on.
65+
- **Controller Manager**: Watches the actual state vs desired state. If you want 3 pods but only 2 are running, the controller creates a 3rd.
66+
- **kubelet**: Agent running on every worker node. Receives pod specs from the control plane and runs containers.
67+
68+
With **minikube**, the control plane and one worker node all run inside a single VM/container on your laptop. Same API, same YAML, different scale.
69+
70+
---
71+
72+
## Core Objects — what you WILL use
73+
74+
### Namespace
75+
A virtual cluster inside the physical cluster. Isolates resources by name. Two teams can both have a `deployment/api` if they're in different namespaces.
76+
77+
We'll put everything in namespace `vault`.
78+
79+
```yaml
80+
apiVersion: v1
81+
kind: Namespace
82+
metadata:
83+
name: vault
84+
```
85+
86+
### Pod
87+
The smallest deployable unit. One or more containers that share:
88+
- The same network interface (same IP, same ports)
89+
- The same storage volumes
90+
91+
In practice: almost always one container per pod.
92+
93+
Pods are **ephemeral**. They can be killed and replaced at any time. They get new IPs when restarted. You never address a pod by IP directly — that's what Services are for.
94+
95+
```yaml
96+
apiVersion: v1
97+
kind: Pod
98+
metadata:
99+
name: vault-api
100+
namespace: vault
101+
spec:
102+
containers:
103+
- name: api
104+
image: vault-api:latest
105+
ports:
106+
- containerPort: 8000
107+
```
108+
109+
You almost never write a raw Pod. You write a Deployment that manages pods for you.
110+
111+
### Deployment
112+
Manages a set of identical pods. Tells K8s: "I want exactly N copies of this pod always running."
113+
114+
When you update the image → Deployment does a rolling update (zero downtime).
115+
When a pod crashes → Deployment starts a replacement automatically.
116+
117+
```yaml
118+
apiVersion: apps/v1
119+
kind: Deployment
120+
metadata:
121+
name: vault-api
122+
namespace: vault
123+
spec:
124+
replicas: 2
125+
selector:
126+
matchLabels:
127+
app: vault-api
128+
template: # ← this is the pod template
129+
metadata:
130+
labels:
131+
app: vault-api
132+
spec:
133+
containers:
134+
- name: api
135+
image: vault-api:latest
136+
```
137+
138+
### ReplicaSet
139+
Deployment actually creates a ReplicaSet under the hood. ReplicaSet ensures N pods are running. You almost never interact with ReplicaSets directly — Deployment manages them.
140+
141+
### Service
142+
Pods are ephemeral (they get new IPs). A Service gives a stable DNS name and IP that always points to the healthy pods.
143+
144+
```
145+
Service (stable: api.vault.svc.cluster.local)
146+
├── Pod A (IP: 10.0.0.1) ← may come and go
147+
├── Pod B (IP: 10.0.0.2)
148+
└── Pod C (IP: 10.0.0.3)
149+
```
150+
151+
Service types:
152+
- **ClusterIP** (default): accessible only inside the cluster. `http://api:8000` from another pod in the same namespace.
153+
- **NodePort**: exposes a port on every node. Accessible from outside but on a weird port (30000-32767). For debugging only.
154+
- **LoadBalancer**: provisions a cloud load balancer (GKE, EKS). This is how production traffic enters.
155+
156+
```yaml
157+
apiVersion: v1
158+
kind: Service
159+
metadata:
160+
name: api
161+
namespace: vault
162+
spec:
163+
selector:
164+
app: vault-api # routes to pods with this label
165+
ports:
166+
- port: 8000
167+
targetPort: 8000
168+
type: ClusterIP
169+
```
170+
171+
### Ingress
172+
An Ingress defines HTTP routing rules: "traffic for `/api/` → Service `api`."
173+
174+
An **Ingress Controller** (a pod, usually nginx) reads these rules and enforces them. The controller is installed once; you write many Ingress resources.
175+
176+
This replaces your custom nginx container from docker-compose. You write routing rules, the controller handles the rest.
177+
178+
```yaml
179+
apiVersion: networking.k8s.io/v1
180+
kind: Ingress
181+
metadata:
182+
name: vault
183+
namespace: vault
184+
spec:
185+
rules:
186+
- http:
187+
paths:
188+
- path: /api/
189+
pathType: Prefix
190+
backend:
191+
service:
192+
name: api
193+
port:
194+
number: 8000
195+
```
196+
197+
minikube ships with a built-in nginx ingress controller: `minikube addons enable ingress`.
198+
199+
### ConfigMap
200+
Non-sensitive configuration stored in K8s. Pods read it as environment variables or as mounted files.
201+
202+
```yaml
203+
apiVersion: v1
204+
kind: ConfigMap
205+
metadata:
206+
name: vault-config
207+
namespace: vault
208+
data:
209+
APP_ENV: "production"
210+
KEYCLOAK_URL: "http://keycloak:8080"
211+
```
212+
213+
Used in a pod:
214+
```yaml
215+
envFrom:
216+
- configMapRef:
217+
name: vault-config
218+
```
219+
220+
### Secret
221+
Same as ConfigMap but for sensitive values. Stored base64-encoded (not encrypted by default — requires additional cluster config for encryption at rest).
222+
223+
```bash
224+
kubectl create secret generic vault-secrets \
225+
--from-literal=POSTGRES_PASSWORD=strongpassword \
226+
--from-literal=VAULT_ENCRYPTION_KEY=mykey \
227+
-n vault
228+
```
229+
230+
Secrets created from the command line are NOT in git — correct. YAML with base64 values in git = bad practice for real secrets.
231+
232+
---
233+
234+
## Storage Objects
235+
236+
### PersistentVolume (PV)
237+
Actual storage provisioned in the cluster. A directory on a node, a cloud disk, an NFS mount. Usually provisioned automatically.
238+
239+
### PersistentVolumeClaim (PVC)
240+
A request for storage: "I need 5Gi." K8s finds a matching PV and binds them.
241+
242+
Pods reference the PVC, not the PV directly. This separates "what I need" from "where it physically is."
243+
244+
### StatefulSet
245+
Like a Deployment but for stateful services (databases). Guarantees:
246+
- Stable pod names (`postgres-0`, `postgres-1` — not random)
247+
- Stable storage (same PVC reattaches after restart)
248+
- Ordered startup (pod-0 before pod-1)
249+
250+
Use Deployment for stateless services (API, frontend).
251+
Use StatefulSet for databases (PostgreSQL).
252+
253+
---
254+
255+
## Minikube setup
256+
257+
```bash
258+
# Install minikube
259+
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
260+
sudo install minikube-linux-amd64 /usr/local/bin/minikube
261+
262+
# Install kubectl
263+
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
264+
chmod +x kubectl && sudo mv kubectl /usr/local/bin/kubectl
265+
266+
# Start cluster (uses Docker driver)
267+
minikube start --driver=docker
268+
269+
# Verify
270+
kubectl get nodes
271+
# NAME STATUS ROLES AGE VERSION
272+
# minikube Ready control-plane 1m v1.x.x
273+
274+
# Enable ingress controller
275+
minikube addons enable ingress
276+
277+
# Check ingress controller is running
278+
kubectl get pods -n ingress-nginx
279+
```
280+
281+
---
282+
283+
## Practice exercise — run a single pod
284+
285+
Before deploying the vault app, run a simple nginx pod to get comfortable with kubectl:
286+
287+
```bash
288+
# Create a namespace
289+
kubectl create namespace practice
290+
291+
# Run a pod
292+
kubectl run nginx --image=nginx:alpine -n practice
293+
294+
# Watch it start
295+
kubectl get pods -n practice -w
296+
297+
# Get into the container
298+
kubectl exec -it nginx -n practice -- sh
299+
300+
# Clean up
301+
kubectl delete pod nginx -n practice
302+
kubectl delete namespace practice
303+
```
304+
305+
---
306+
307+
## kubectl commands reference for this phase
308+
309+
```bash
310+
kubectl get nodes
311+
kubectl get pods -n vault
312+
kubectl get pods -n vault -w # watch live
313+
kubectl get all -n vault # all objects
314+
kubectl describe pod <name> -n vault # detailed info + events
315+
kubectl logs <pod-name> -n vault
316+
kubectl logs <pod-name> -n vault -f # follow
317+
kubectl exec -it <pod> -n vault -- sh # shell into pod
318+
kubectl apply -f <file or dir>
319+
kubectl delete -f <file>
320+
kubectl port-forward svc/<name> 8000:8000 -n vault
321+
minikube ip # cluster IP for browser access
322+
minikube dashboard # web UI
323+
```
324+
325+
---
326+
327+
## Success criteria
328+
329+
```bash
330+
minikube status
331+
# → Running
332+
333+
kubectl get nodes
334+
# → minikube Ready
335+
336+
kubectl get pods -n ingress-nginx
337+
# → ingress-nginx-controller-xxx Running
338+
339+
# Ran practice pod, executed into it, deleted it
340+
```
341+
342+
Concepts are solid. Ready for Phase 11: push images to GHCR.

0 commit comments

Comments
 (0)