Skip to content

If I just want to evaluate the nine benchmark datasets with opt-1.3b ,without train. How should I do? #4

@cgt-woailol

Description

@cgt-woailol

I have set the config.json. It is right?
{ "mode": "general_lm_eval", "wandb_project": "Knowledge Unlearning", "wandb_run_name": "example", "num_train_epochs": 20, "check_val_every_n_epoch": 1, "check_validation_only": true, "do_init_eval": true, "train_set": "data/main/lm_extraction_32_0.csv", "valid_sets": [ "validation_data/lambada.csv", "piqa", "hellaswag", "ai2_arc", "ai2_arc", "super_glue", "winogrande", "math_qa", "validation_data/pubmed_qa.csv" ], "valid_subset_path": [ "", "", "", "ARC-Easy", "ARC-Challenge", "copa", "winogrande_s", "", "" ], "valid_type_path": [ "test", "validation", "validation", "validation", "validation", "validation", "validation", "validation", "" ], "cache_dir":"/home/data0/cgt/knowledge-unlearning/val", "train_batch_size": 8, "eval_batch_size": 8, "gradient_accumulation_steps": 4, "ngpu": 1, "learning_rate": 5e-5, "model_name_or_path": "/home/chen/.cache/huggingface/hub/models--facebook--opt-1.3b/snapshots/3f5c25d0bc631cb57ac65913f76e22c2dfb61d62", "el_threshold": 0.0499, "ma_threshold": 0.2994, "input_length": 512, "output_length": 512, "target_length": 200, "num_workers": 64, "strategy": "deepspeed_stage_2_offload", "fp16": true, "wandb_log": false }

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions