Punk GPU Lab - Your Private, GPU-charged Cloud Development Workspace

Spin up a private and secure cloud-based development lab in minutes! Perfect for data scientists, ML engineers, and developers who need GPU-backed workspaces. Jump straight into VS Code as if it were running on your own machine. No public IPs, no SSH keys, no worries, just secure, fully auditable, GPU-charged environment from anywhere.

This workspace is ideal not only for AI/ML workloads but also for general-purpose remote development. You can choose to deploy instances with or without GPU support depending on your needs.

Table of Contents

• Why this project?
• Prerequisites
• Getting Started
• AI/ML Development Readiness
• Storage handling
• Cleanup, stop and start

Why this project?

Modern AI/ML workflows demand secure, high-performance, and easy-to-use development environments. This project provides:

• Private and isolated VPC networking: all resources are deployed in a private subnet with no public IPs, reducing exposure to the internet and lowering the attack surface.
• Secure, keyless access via AWS Systems Manager: developers connect to instances through Session Manager, eliminating the need to manage or store SSH keys and credentials. No open ports. No inbound traffic. Just secure, auditable, encrypted access.
• GPU-ready EC2 instances: easily deploy EC2 instances with GPU support (e.g., G4/G5/VT1) for accelerated training, fine-tuning, or inference tasks.
• Simple setup using Terraform: infrastructure is fully automated using Terraform, enabling consistent, repeatable deployments in minutes.
• Seamless developer experience with VS Code: integrates with the AWS Toolkit and Remote SSH extensions in VS Code for local-like development on cloud resources.
• Cost-effective architecture: leverages and provides support for EC2 Spot Instances to minimize costs.

Prerequisites

Before you begin, make sure you have the following tools and configurations in place:

AWS Account Requirements

• An AWS Account with sufficient permissions to create and manage the resources handled in terraform (EC2, VPC, IAM, KMS, S3...).
• (Only for GPU-enabled instances) Service Quotas: If you're testing with a GPU-enabled instance (e.g., g4dn.xlarge), ensure your AWS account has sufficient EC2 On-Demand or Spot vCPU quota to launch it. For new accounts, this quota is often 0 by default, and you'll need to request an increase via the Service Quotas Console.
  • If you're just testing the setup and don't need GPU capabilities, you can use a different AMI (like Ubuntu or Amazon Linux) and a non-GPU instance type. This avoids the need for a quota increase.

Local Environment Setup

• Terraform version 1.11 or higher.
• AWS CLI properly installed and configured.
• Visual Studio Code with:
  • AWS Toolkit extension (v3.39.0 or later)
  • Remote - SSH extension

AWS Authentication & Named Profiles

To authenticate with AWS and allow Terraform to deploy resources on your behalf, configure the AWS CLI using a named profile:

aws configure --profile workspace-dev

This command will prompt you to enter your AWS Access Key, Secret Key, default region, and output format.

The Terraform configuration in this project uses a named profile (workspace-dev) to authenticate. If you want to change or avoid using a profile:

1. Edit the provider block in Terraform (providers.tf):

provider "aws" {
  ...
  profile = "${var.project_name}-${var.environment}" # <- Change or remove this line
  ...
}

2. To use environment variables instead, export credentials before running Terraform:

export AWS_ACCESS_KEY_ID="your-access-key-id"
export AWS_SECRET_ACCESS_KEY="your-secret-access-key"
export AWS_DEFAULT_REGION="your-region"

This gives you flexibility whether you use profiles or direct environment configuration.

Getting Started

Follow the steps below to deploy your private and secure AI-ready development environment.

1. Clone the Repository

git clone https://github.com/nicolaDeCristofaro/punk-gpu-lab
cd punk-gpu-lab/

2. Decide How to Manage Terraform State backend

By default, Terraform stores its state locally. This is fine for quick tests or development purposes, but it's not recommended for team environments or production use. For a deep dive on the different options to handle terraform state backends read Hashicorp docs. Below are described the local defualt and the remote state on AWS used in this project.

Option 1: Local State (default)

No extra configuration is needed, Terraform will create a terraform.tfstate file in your working directory.

⚠️ Warning: Local state is not shared, lacks locking, and can be accidentally lost or overwritten.

*In this repo it is assumed the use of remote backend for storing state, so if you want to use local state, comment or delete the backend.tf file, before launch.

Option 2: Remote State with S3 (recommended)

This project includes a helper module to configure remote backend support. Navigate to the terraform-backend folder and follow the setup instructions in the README to enable:

• S3 bucket for state storage with:
  • versioning
  • encryption
  • native state locking

Using remote state ensures safer, auditable, and team-friendly deployments.

*In this repo it is assumed the use of remote backend for storing state, revise the backend.tf file and asjust it as needed.

3. Configure Your Environment

Ensure your AWS CLI is authenticated using a named profile (see Prerequisites) or exported credentials.

4. Initialize Terraform

terraform init

If using remote state, make sure you've already configured the backend before running this command.

5. Apply the Infrastructure

terraform apply

Terraform will prompt for approval before creating the infrastructure.

6. Wait for EC2 Status Checks

After Terraform creates the instance, you must wait for the EC2 instance to pass all 3/3 status checks before connecting. You can:

• Go to the EC2 Console and check the instance status manually.
• Or run the script at ./scripts/wait_for_ec2_status_check.sh to wait for the instance to be fully ready:

# inputs: sh ./scripts/wait_for_ec2_status_check.sh <aws-profile-name> <aws-ec2-instance-id>
sh ./scripts/wait_for_ec2_status_check.sh workspace-dev i-0d6ca630e5983970b

Expected output:

Waiting for instance i-0d6ca630e5983970b to reach 3/3 status checks...
✅ Instance is ready: all 3/3 checks passed.

This ensures you’re connecting only after the instance is healthy and fully bootstrapped.

7. Connect to Your EC2 Instance via AWS Toolkit in VS Code

Once your infrastructure is deployed and the status checks passed, you can connect to the EC2 instance using the AWS Toolkit extension in VS Code.

Step 1: Open AWS Toolkit in VS Code

1. Click the AWS icon in the left sidebar to open the AWS Toolkit panel.
2. Make sure you're logged in using the correct AWS profile (e.g., workspace-dev).
   • You can verify or switch profiles using the "Select a Connection" button in the Toolkit panel.
   • If no connection is available, click "Connect to AWS" and choose your configured profile.

Step 2: Connect to the EC2 Instance

1. In the AWS Toolkit panel, expand the EC2 Instances section.
2. Locate the EC2 instance you deployed (you can filter by name or instance ID).
3. Right-click on the instance and select "AWS: Connect VS Code to EC2 instance".

4. VS Code will launch a new window connected to the EC2 instance through AWS Systems Manager Session Manager.
   • This does not require SSH keys or public IP addresses.
   • Traffic is encrypted and routed through Systems Manager for secure and auditable access.

Note: You may need to enable "Experimental Features" in the AWS Toolkit extension settings for Session Manager-based connection to EC2.

✅ Supported configurations:

• EC2 instances running Linux or macOS
• Local development on Linux or macOS (Windows currently not supported for Session Manager + VS Code integration)

AI/ML Development Readiness

Given the growing demand for AI/ML development, this project focuses primarily on supporting development on those workloads. If your use case involves training or fine-tuning machine learning models, this setup can provision GPU-enabled EC2 instances (e.g., g4dn.xlarge, g5.xlarge, or g6.xlarge) to accelerate your workflows.

Example: Serve a 7B chat model with Gradio

This sample runs a Python app on a GPU‑backed EC2 instance to serve a lightweight chat UI for the open‑source HuggingFaceH4/zephyr-7b-beta model. The app stays private (no public IPs, no inbound ports) and is reachable only via AWS Systems Manager (SSM) port forwarding or via the AWS Toolkit in VS Code.

What the sample does (python-samples/small-llm.py)

• Loads HuggingFaceH4/zephyr-7b-beta (MIT‑licensed, ungated) from Hugging Face.
• Loads the model in 4‑bit NF4 quantization via BitsAndBytesConfig (with double quantization and FP16 compute) to fit comfortably on a single GPU.
• Uses device_map="auto" so layers are placed on the GPU automatically (e.g., the NVIDIA L4 on g6.xlarge).
• Streams tokens to the UI in real time using TextIteratorStreamer.
• Serves a Gradio web app.

Clone the repo on the remote machine

cd /mnt/persistent-data/
git clone https://github.com/nicolaDeCristofaro/punk-gpu-lab
cd punk-gpu-lab

Install dependencies

bash ./scripts/gpu_usecase_dependencies_install_uv.sh

Hers's an overview of what the script does:

• Set strict bash defaults (set -euo pipefail) to fail fast on errors.
• Use these environment variables (override as needed):
  • ENV_DIR — venv path (default ~/.venvs/gradio-venv)
  • PY_VER — Python version (default 3.11)
  • TORCH_INDEX_URL — PyTorch wheel index (default CUDA 12.1 wheels)
• Ensure the uv tool is installed and added to your PATH.

uv is a modern, fast Python package and environment manager; it’s used here instead of pip/venv for speed. If you prefer Conda instead of uv, there is an alternative script gpu_usecase_dependencies_install_conda.sh that performs similar setup steps using Conda.

• Install the requested Python with uv python install.
• Create a fresh virtual environment at $ENV_DIR.
• Generate a minimal requirements.txt (gradio, transformers, accelerate, bitsandbytes).
• Install PyTorch from $TORCH_INDEX_URL, then install the rest via uv pip.
• Run a smoke test printing Torch version and CUDA availability.

Run the app

/home/ubuntu/.venvs/gradio-venv/bin/python /mnt/persistent-data/punk-gpu-lab/python-samples/small-llm.py

• By default the app is reachable locally and since you’re already connected with AWS Toolkit and have port forwarding set up, you don’t need to do anything else, the Toolkit forwards the port automatically. Just open: http://localhost:7860 or click the button on the banner that appears.

• Alternatively, you can set share=True and Gradio will print an ephemeral public https://*.gradio.live URL that anyone can open. This is quick for demos, but be careful exposing sensitive data (you can also add basic auth for simple authentication).

What you’ll see

Here’s how the app looks in the browser:

Monitor GPU usage

From another terminal on the instance while the app is running:

# update view every second
watch -n 1 nvidia-smi

# or one-shot snapshot
nvidia-smi

Tip: To persist artifacts (logs, prompts, outputs) across instance recycling, write them under the configured mount point (default: /mnt/persistent-data). For more details about how persistent storage is handled or how to change the mount point see the next section Storage handling

Storage handling

The example code provided provisions a separate, persistent EBS volume in addition to the EC2 root volume:

• The root volume is typically ephemeral, especially when using Spot Instances, which can be terminated by AWS at any time.
• To prevent data loss, the additional EBS volume is managed independently in Terraform and automatically formatted and mounted at instance startup.

The purpose of handling this second EBS volume separately is to ensure data persistence:

• If the EC2 instance is terminated (manually or by AWS Spot reclamation), the secondary EBS volume remains intact.
• When a new EC2 instance is created, the volume is reattached automatically, and the mount point is restored.
• This ensures you can safely store project files, model checkpoints, datasets, and logs without worrying about losing them when the compute instance is recycled (be careful to save these file under the mount point path).

The mount point for this volume is fully configurable via a Terraform variable:

variable "ec2_workspace" {
  type = object({
    ...
    volume_size          = number
    volume_mount_point  = string
    ...
  })
}

By default, the volume is mounted at /mnt/persistent-data, but you can override this path to suit your project needs.

The formatting, mounting, and persistence logic (including fstab configuration and permissions for the ubuntu user) are automated via a startup script (user_data), so no manual setup is required post-launch.

Cleanup, stop and start

Stop and Start from AWS Toolkit

You can stop and start the EC2 instance directly from the AWS Toolkit in Visual Studio Code:

1. Open the AWS Toolkit panel.
2. Locate your EC2 instance.
3. Choose Stop Instance or Start Instance as needed.

⚠️ Important for Spot Instances: Stopping Spot instances is not always reliable. AWS may reclaim the instance, and stopped Spot instances may not restart later.

Complete Cleanup

To destroy all infrastructure created via Terraform:

terraform destroy

This will remove all resources described in the code and stored in the terraform state.

Terminate Only EC2 and Keep EBS Volume

To terminate just the EC2 instance but keep its secondary EBS volume (for data persistence):

1. Comment out the EC2 module and the aws_volume_attachment resource in your Terraform main.tf file.
2. Run:

terraform plan
terraform apply

• Terraform will destroy the EC2 instance but retain the EBS volume. Expected output: Plan: 0 to add, 0 to change, 7 to destroy.

3. To restore (re-create a new instance + EBS volume attachment):

• Uncomment the EC2 and volume attachment resources, then re-run terraform apply.
• The new EC2 instance will be created and the existing EBS volume reattached with all previous data intact.