getting_started_with_llm.md

Step-by-Step approach to Using LLMs on Hugging Face

Step 1: Install the Hugging Face Library

First, you need the transformers library to interact with models from Hugging Face.

Install required libraries:

pip install transformers
pip install torch  # Required for running models
pip install huggingface-hub  # Interact with Hugging Face API

---

Step 2: Log in to Hugging Face from Python

To use models or download large files, you need to authenticate yourself.

Generate Access Token:

1. Go to Hugging Face → Settings → Access Tokens.
2. Create a new token with read permissions.
3. Copy the token.

Log in with Python:

huggingface-cli login

This command will prompt you to paste the token.

---

Step 3: Explore Available Models

1. Visit the Hugging Face Model Hub.
2. Search for LLMs like gpt2, BERT, Llama, or falcon.

---

Step 4: Load a Model for Text Generation

You can load a model and generate text using transformers library.

Example: Using GPT-2 for Text Generation

from transformers import AutoModelForCausalLM, AutoTokenizer

# Load the model and tokenizer
model_name = "gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)

# Encode the prompt and generate output
prompt = "Once upon a time,"
inputs = tokenizer(prompt, return_tensors="pt")

# Generate text
output = model.generate(inputs['input_ids'], max_length=50, num_return_sequences=1)

# Decode the output
print(tokenizer.decode(output[0], skip_special_tokens=True))

---

Step 5: Use LLMs for Different Tasks

1. Text Summarization

   from transformers import pipeline
   
   summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
   text = "The quick brown fox jumps over the lazy dog. The fox is known for its speed."
   summary = summarizer(text, max_length=30, min_length=10, do_sample=False)
   print(summary[0]['summary_text'])

2. Question-Answering

      import torch
      from transformers import LlamaForCausalLM, LlamaTokenizer, AutoTokenizer
      
      # Model and tokenizer ID from Hugging Face Hub
      MODEL_ID = "meta-llama/Meta-Llama-3.1-8B-Instruct"
      
      # Load the tokenizer (use AutoTokenizer for flexibility)
      tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, use_fast=True)
      
      # Load the model with appropriate device settings
      device = "cuda" if torch.cuda.is_available() else "cpu"
      model = LlamaForCausalLM.from_pretrained(MODEL_ID, torch_dtype=torch.float16).to(device)
      
      def ask_question(question):
          # Tokenize input and move tensors to the appropriate device
          inputs = tokenizer(question, return_tensors="pt").to(device)
      
          print(f"Tokenized Input: {inputs}")
      
          # Generate a response from the model
          with torch.no_grad():
              output = model.generate(
                  inputs["input_ids"],
                  max_length=256,  # Adjust max response length
                  temperature=0.7,  # Controls randomness
                  top_k=50,  # Limits sampling to top k tokens
                  top_p=0.9,  # Nucleus sampling
                  eos_token_id=tokenizer.eos_token_id,  # Ensure proper end-of-sequence token
              )
      
          print(f"Raw Output: {output}")
      
          # Decode and clean up the output text
          answer = tokenizer.decode(output[0], skip_special_tokens=True, clean_up_tokenization_spaces=False)
          return answer
      
      if __name__ == "__main__":
          # Test the model with a sample question
          question = "What are the benefits of using AI in education?"
          answer = ask_question(question)
          print(f"Q: {question}\nA: {answer}")

3. Translation

   translator = pipeline("translation_en_to_fr", model="Helsinki-NLP/opus-mt-en-fr")
   result = translator("Hello, how are you?")
   print(result[0]['translation_text'])

---

Step 6: Use Custom Models or Fine-Tuning

If you want to fine-tune an LLM on your own data or use a specialized model:

1. Load a Custom Model (replace with a specific model name you found on Hugging Face):

   model_name = "tiiuae/falcon-7b"
   tokenizer = AutoTokenizer.from_pretrained(model_name)
   model = AutoModelForCausalLM.from_pretrained(model_name)

2. Fine-tuning your Model:

   • You can use Hugging Face’s Trainer API for fine-tuning if you need the model to perform better on your specific tasks. Read more here.

---

Step 7: Run Models on GPU for Faster Performance

If you want to run models faster, use GPU if available.

import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
model = AutoModelForCausalLM.from_pretrained(model_name).to(device)

---

Step 8: Use Hugging Face Spaces for Deployment

Hugging Face Spaces allow you to deploy your models in a web app using Gradio or Streamlit.

• Create a new Space on Hugging Face.
• Use Gradio to build a simple UI for text generation.

Example: Building a Gradio App

pip install gradio

import gradio as gr
from transformers import pipeline

text_generator = pipeline("text-generation", model="gpt2")

def generate_text(prompt):
    return text_generator(prompt, max_length=50)[0]['generated_text']

gr.Interface(fn=generate_text, inputs="text", outputs="text").launch()

---

Step 9: Monitor and Optimize Usage

LLMs can be computationally expensive. Use token limits (like limiting max_length) and batch processing to optimize performance.

---

Step 10: Explore More Resources on Hugging Face

• Hugging Face Forums: Ask questions and share ideas.
• Documentation: Explore the Hugging Face Docs.