Running Hyperparameter Sweeps with Weights & Biases on Distributed Systems

In this tutorial, you'll learn:

• Multiple ways to interact with a container from the command line
• How to automate a hyperparameter sweep with Weights & Biases (W&B) and Slurm arrays
• How to track these runs using the W&B dashboard and Slurm commands

Today's Tutorial Agenda

• Authenticating Weights & Biases on Sherlock or Farmshare [link]
• Setting up your training script [link]
• Defining your sweep configuration [link]
• Writing an Sbatch script to run your sweept [link]
• Using array to launch parallel wandb sweep agents [link]
• Monitoring your sweep on the W&B dashboard [link]
• Retrieve the best sweep configuration [link]

Do you have any questions?

Please reach out to us at matrics-bootcamp@stanford.edu or set up a General Consultation with us at this link.

Would you like to provide feedback?

Please provide anonymous feedback here.