rename worker to exporter, add importer, rename templates & clear README

Author: ggtrd

README.md

@@ -1,98 +1,86 @@
-# ClusterSnap - Cloud-Agnostic Kubernetes Backup Tool
+# ClusterSnap
+**Cloud-Agnostic Kubernetes Backup Tool**
 
-> ClusterSnap runs as a Kubernetes CronJob, zips PVC data from a shared volume,
-> uploads the archive to any cloud storage backend (AWS S3, MinIO, SeaweedFS, or
-> Azure Blob Storage), and pushes structured telemetry to Grafana Loki and
-> Prometheus - all from a single, provider-agnostic codebase.
+> Clustersnap is a lightweight, cloud-agnostic backup tool that safely secures Kubernetes volumes into remote storage like Azure Blob Storage or AWS S3.
+
+
+* **Export** Kubernetes data to remote storage.
+* **Import** backed-up data to Kubernetes.
+* **Backups Isolation:** Each Kubernetes namespace is backed up into a dedicated space.
+* **Ensure data Integrity:** Scale-down running workload, backup their volumes and restore the workload.
+* **Cloud-Native & Minimalist:** Light footprint, easy to install, and fully customizable from standard Kubernetes manifests without third-party CRD dependencies.
+
+
+## Requirements
+* Remote storage (Azure Blob Storage, S3 etc...), with credentials
+* Working Kubernetes cluster
 
-- 🗜️ **Single ZIP archive** per backup run using Python's built-in `zipfile`
-- ☁️ **Provider-agnostic** storage layer - swap S3 for Azure with one env var
-- 📋 **JSON-structured logs** parsed natively by Grafana Loki
-- 🛡️ **ZIP path-traversal protection** during restore
-- 🔒 **Dual Azure auth** - shared-key or service-principal
-- 📊 **Prometheus metrics** pushed to Pushgateway after every run
 
 ## Installation
-### Install from sources
-* create venv
-  ```
-  python -m .venv
-  source .venv/bin/activate
-  ```
-
-* install dependencies
-  ```
-  pip install --upgrade pip
-  pip install -r requirements.txt
-  ```
-
-* create `.env` file
-  * AWS/S3
+* install CLI
+  * create venv
     ```
-    tofill
+    python -m .venv
+    source .venv/bin/activate
     ```
 
-  * Azure Blob Storage (shared-key)
+  * install dependencies
     ```
-    tofill
+    pip install --upgrade pip
+    pip install -r requirements.txt
     ```
 
-* run
-  ```
-  python -m src.main
-  ```
-
-<!-- ## Install locally with Docker
-* build
-  ```
-  docker build . --tag clustersnap:local
-  ```
-
-* run
-  ```
-  docker run clustersnap:local
-  ``` -->
-
-## Install on Kubernetes (pod)
-* create secret to pull image
-  ```
-  kubectl create secret docker-registry registry-auth-clustersnap \
-    --docker-server=ghcr.io \
-    --docker-username=GITHUB_ACCOUNT \
-    --docker-password=GITHUB_PAT \
-    --docker-email=GITHUB_ACCOUNT
-  ```
-* create secret for remote storage access
-  * edit `src/templates/backup.secret.yaml` with your own config
-  * apply
+* create required Kubernetes objects
+  * `pullImage` secret
+    ```
+    kubectl create secret docker-registry registry-auth-clustersnap \
+      --docker-server=ghcr.io \
+      --docker-username=GITHUB_ACCOUNT \
+      --docker-password=GITHUB_PAT \
+      --docker-email=GITHUB_ACCOUNT
+    ```
+
+  * `config` Secret
+    * edit `src/templates/backup.secret.yaml` with your own config
+    * apply
+      ```
+      NAMESPACE=YOUR_NAMESPACE envsubst < templates/backup.secret.yaml | kubectl apply -f -
+      ```
+
+  * `backup-map` ConfigMap
+    > Note: this ConfigMap is used to orchestrate the backup. \
+    > You can manually edit it to customize the backup behaviors, the command below is just a quick way to create it.
     ```
-    NAMESPACE=tenant-test0 envsubst < templates/backup.secret.yaml | kubectl apply -f -
+    python -m src.master -c KUBECTL_CONTEXT -rm
     ```
-* create pod
-  ```
-  NAMESPACE=tenant-test0 envsubst < tests/resources_test.yaml | kubectl apply -f -
-  ```
 
 ## Usage
-* local
+> Ensure required secrets & configmap are created
+### Kubernetes
+* create pod (& required resources such as RBAC etc...)
 ```
-python -m src.main --c CONTEXT -n NAMESPACE
+NAMESPACE=YOUR_NAMESPACE envsubst < tests/resources_test.yaml | kubectl apply -f -
 ```
+* wait for it to end...
 
-* docker/kubernetes
-just run the image
+### Local CLI
+* backup cluster-wide
+```
+python -m src.master -c KUBECTL_CONTEXT
+```
+
+* backup given namespace
+```
+python -m src.master -c KUBECTL_CONTEXT -n NAMESPACE
+```
 
 
 ## Developpers
 ### Architecture
-```
-clustersnap/
-├── src/
-│   ├── main.py               # Entrypoint for CLI & Kubernetes main pod
-│   ├── worker.py             # Entrypoint for Kubernetes job pods
-│   └── helpers/
-│       ├── kubernetes.py     # Kubernetes class
-```
+* `master.py`   => entrypoint of the master pod (orchestrate backup jobs, scaling replicas etc...)
+* `exporter.py` => entrypoint of the exporter pod (do the backup = dump persisted storage & sync it to remote storage)
+* `importer.py` => entrypoint of the importer pod (import backups from remote storage to Kubernetes)
+
 
 ### Workflow of export
 ```

src/worker.py src/exporter.pysrc/worker.py renamed to src/exporter.py

@@ -0,0 +1 @@
+# todo

src/importer.py

@@ -25,7 +25,7 @@ def create_backup_job(kube, pvc_name: str, mounted_path: str, node_name: str) ->
     print(f"starting backup job: {job_name}")
 
     project_root = Path(__file__).resolve().parents[2]
-    template_path = f"{project_root}/templates/backup.job.yaml"
+    template_path = f"{project_root}/templates/exporter.job.yaml"
     try:
         with open(template_path, "r") as f:
             yaml_content = f.read()

src/kubernetes/job.py

@@ -16,32 +16,62 @@ def get_backup_map(kube) -> dict:
 
 
 def backup_map_scanner(kube, target_namespaces: list) -> dict:
-    """Scan the cluster to get a YAML map of what will be backuped."""
+    """Scan the cluster to get a YAML map to orchestrate the backups."""
     map_data = {
         "cluster": kube.get_cluster_name(),
         "namespaces": []
     }
 
     print(f"cluster: {kube.get_cluster_name()}")
     print(f"namespaces:")
+    original_ns = kube.namespace
 
     for ns in target_namespaces:
         kube.namespace = ns
-        print(f"- name: {kube.get_namespace()}")
+        current_ns = kube.get_namespace()
+        print(f"- name: {current_ns}")
 
         ns_data = {
-            "name": kube.get_namespace(),
+            "name": current_ns,
+            "workloads": [],  # <--- Nouvelle section pour conserver l'état des replicas
             "claims": []
         }
+        
+        workload_list = set()
 
-        pvcs = kube.list_pvc()
-        if not pvcs:
+        pvc_list = kube.list_pvc()
+        if not pvc_list:
             print(f"   claims: []")
             map_data["namespaces"].append(ns_data)
             continue
 
+        # Collect workload replicas
+        for pvc in pvc_list:
+            mounts = kube.get_pvc(pvc['name'])
+            if mounts:
+                for m in mounts:
+                    resource_str = m['resource']  # ex: "StatefulSet/keycloak-postgresql"
+                    if resource_str not in workload_list:
+                        workload_list.add(resource_str)
+                        resource_type, resource_name = resource_str.split('/')
+                        
+                        # Récupération live du nombre de replicas via ton wrapper kube
+                        replicas = kube.get_replicas(resource_type, resource_name)
+                        
+                        ns_data["workloads"].append({
+                            "resource": resource_str,
+                            "replicas": replicas
+                        })
+
+        print(f"   workloads:")
+        for wl in ns_data["workloads"]:
+            print(f"    - resource: {wl['resource']}")
+            print(f"      replicas: {wl['replicas']}")
+
+
+        # Collect PVC
         print(f"   claims:")
-        for pvc in pvcs:
+        for pvc in pvc_list:
             print(f"    - name: {pvc['name']}")
             print(f"      status: {pvc['status']}")
             print(f"      capacity: {pvc['capacity']}")
@@ -79,4 +109,5 @@ def backup_map_scanner(kube, target_namespaces: list) -> dict:
             ns_data["claims"].append(pvc_data)
         map_data["namespaces"].append(ns_data)
 
+    kube.namespace = original_ns
     return map_data

src/utils/backup_map.py

@@ -22,7 +22,7 @@ spec:
         - name: backup-worker
           image: ghcr.io/cosmo-tech/clustersnap:latest
           imagePullPolicy: Always
-          command: ["python", "-m", "src.worker"]
+          command: ["python", "-m", "src.exporter"]
           args: ["--pvc", "{{ PVC_NAME }}", "--path", "{{ MOUNTED_PATH }}"]
           resources:
             requests: