Replace example datasets with prebaked "perfectblend" and "flickr30k" - #3062
Replace example datasets with prebaked "perfectblend" and "flickr30k"#3062kylesayrs wants to merge 1 commit into
Conversation
Simplify ~55 example files by replacing manual dataset loading, preprocessing, and tokenization with `dataset="perfectblend"`. DDP examples also benefit from automatic rank partitioning added in the parent PR. Add `examples/custom_dataset_example.py` showing how to use a custom dataset (e.g. ultrachat_200k) for users who need custom preprocessing. Examples with model-specific processing (custom processors, data collators, audio/vision pipelines, AutoRound) are left unchanged. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Merge Protections🔴 1 of 1 protections blocking · waiting on 👀 reviews
🔴 Require one maintainer reviewWaiting for any of
This rule is failing.All PRs must have at least one approving review from a maintainer before merging.
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request simplifies the repository's example scripts by removing manual dataset loading, preprocessing, and tokenization, replacing them with the prebaked "perfectblend" dataset inside the oneshot() function. To ensure users still have a reference for custom data, a new custom_dataset_example.py script has been added. The review feedback suggests removing unused tokenizer imports and variables in the prefetch benchmark example, and ensuring that the model is saved with save_compressed=True in the new custom dataset example to align with the comments and other scripts.
| from transformers import AutoModelForCausalLM, AutoTokenizer | ||
|
|
||
| from llmcompressor import oneshot | ||
|
|
||
| MODEL_ID = "meta-llama/Meta-Llama-3.1-8B-Instruct" | ||
| DATASET_ID = "HuggingFaceH4/ultrachat_200k" | ||
| DATASET_SPLIT = "train_sft" | ||
| NUM_CALIBRATION_SAMPLES = 20 | ||
| MAX_SEQUENCE_LENGTH = 2048 | ||
|
|
||
| model = AutoModelForCausalLM.from_pretrained(MODEL_ID) | ||
| tokenizer = AutoTokenizer.from_pretrained(MODEL_ID) |
There was a problem hiding this comment.
The AutoTokenizer import and tokenizer instantiation are completely unused in this prefetch benchmark script. Since the dataset loading and tokenization are now handled automatically by oneshot using the prebaked "perfectblend" dataset, we can safely remove them to clean up the code and avoid redundant overhead.
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| from llmcompressor import oneshot | |
| MODEL_ID = "meta-llama/Meta-Llama-3.1-8B-Instruct" | |
| DATASET_ID = "HuggingFaceH4/ultrachat_200k" | |
| DATASET_SPLIT = "train_sft" | |
| NUM_CALIBRATION_SAMPLES = 20 | |
| MAX_SEQUENCE_LENGTH = 2048 | |
| model = AutoModelForCausalLM.from_pretrained(MODEL_ID) | |
| tokenizer = AutoTokenizer.from_pretrained(MODEL_ID) | |
| from transformers import AutoModelForCausalLM | |
| from llmcompressor import oneshot | |
| MODEL_ID = "meta-llama/Meta-Llama-3.1-8B-Instruct" | |
| model = AutoModelForCausalLM.from_pretrained(MODEL_ID) |
|
|
||
| # Save to disk in compressed-tensors format. | ||
| SAVE_DIR = MODEL_ID.rstrip("/").split("/")[-1] + "-FP8-Dynamic" | ||
| model.save_pretrained(SAVE_DIR) |
There was a problem hiding this comment.
To ensure the model is saved in the serialized compressed-tensors format (as indicated by the comment on line 85), save_compressed=True should be passed to save_pretrained(). This is consistent with other examples in the repository.
| model.save_pretrained(SAVE_DIR) | |
| model.save_pretrained(SAVE_DIR, save_compressed=True) |
1 participant
Summary
dataset="perfectblend", removing ~1700 lines of repetitiveload_dataset+preprocess+tokenizecode.get_rank_partition()is no longer needed when using prebaked datasets.examples/custom_dataset_example.pyshowing how to use a custom dataset (e.g. ultrachat_200k) for users who need custom preprocessing.Examples kept as-is (custom processing required)
auto_round.calib_dataset.get_dataset)Test plan
torchrun --nproc_per_node=2🤖 Generated with Claude Code