Repository navigation
bug: load study.pkl.gz / exp_args.pkl with a restricted unpickler (sfind-862fb47f) - #65
Open
devin-ai-integration[bot] wants to merge 2 commits into
Open
devin-ai-integration[bot] wants to merge 2 commits into
devin-ai-integration[bot] wants to merge 2 commits into
Conversation
Study.load and ExpResult.exp_args used pickle.load on files from any study directory, so a planted study folder could run code on load (archive_studies, load_most_recent, main.py relaunch). Add agentlab.experiments.safe_pickle, which only resolves dataclasses/Enums/pydantic models from agentlab and browsergym (plus AGENTLAB_TRUSTED_PICKLE_MODULES) and an allowlist of stdlib/numpy/pandas/pyarrow data types. Fixes code scan finding sfind-862fb47f130745099616ac639c00fc9c.
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resumo
Corrige o finding de code scan sfind-862fb47f (deserialização insegura):
Study.loadfaziapickle.loadem<dir>/study.pkl.gze, no ramo legado,ExpResult.exp_argsfaziapickle.loademexp_args.pkl. Uma pasta de estudo plantada (drive compartilhado, arquivo enviado por colaborador) executava código ao ser carregada porarchive_studies.search_for_reasons_to_archive,Study.load_most_recent(), relaunch domain.pyetc.Novo módulo
agentlab.experiments.safe_picklecomRestrictedUnpickler.find_class:Usado em
Study.load(study.pkl.gz) eExpResult.exp_args(exp_args.pkl — cobre o ramo legado deStudy.load,load_result_df,find_incomplete). Funções, classes não-dados e gadgets via pacotes confiáveis (agentlab.experiments.study+os.system,make_study,ExpResult, …) são recusados antes de serem chamados. O formato em disco não muda (Study.savecontinua compickle.dump), então estudos existentes continuam carregando.Mudança de contrato (sinalizada): agent args definidos fora de
agentlab/browsergymagora exigemAGENTLAB_TRUSTED_PICKLE_MODULES=meu_pacoteparaStudy.load/exp_argsfuncionarem (documentado no README).GAIA: o
GaiaBenchmarkguardava oDatasetDictdo Hugging Face no pickle do study, o que o unpickler restrito recusa. AgoraGaiaBenchmark.__getstate__gravadataset=None(as tarefas já ficam emenv_args_list), sem ampliar a allowlist paradatasets.*.tests/agents/test_agent.pydefineCheat*Argsno próprio módulo de teste; um fixture autouse adiciona esse módulo aAGENTLAB_TRUSTED_PICKLE_MODULESpara oload_result_df(é exatamente a mudança de contrato acima).Fora do escopo:
step_*.pkl.gz/goal_object.pkl.gz(carregados pelo AgentXray) continuam compicklepuro — contêm objetos de resposta de provedores LLM e exigiriam uma allowlist maior; o README avisa para só abrir pastas de fontes confiáveis.Como testar
pytest tests/experiments/test_safe_pickle.py— 18 passam. Os 3 testes com payload (study.pkl.gzmalicioso,exp_args.pkllegado malicioso,search_for_reasons_to_archiveem um results root plantado) falham emmain(o arquivo marcador é criado) e passam nesta branch (marcador nunca criado,UnsafePickleError).make_study(...).save()+Study.loadpara miniwob_tiny_test, miniwob, workarena_l1, webarena/visualwebarena (SequentialStudies), assistantbench, weblinx,ParallelStudiescomWebArenaInstanceVars, e osexp_args.pklde 2024 emtests/data/test_study— 22/22 OK.pytest tests/experiments tests/analyze -k "not miniwob and not launch_system and not ray"— 32 passed, 1 skipped.DatasetDictem memória →GaiaBenchmark→Study.save→Study.load: antesUnsafePickleError: datasets.arrow_dataset.Dataset, depois carrega as tarefas. Rodado com stubs detapeagents/hydra(não instalados no dev/CI), por isso não virou teste no repo.tests/agents/test_agent.py(precisa do servidor MiniWoB) passa no jobagentlab.TapeAgentArgs.config: DictConfig) —omegaconf/hydranão estão instalados no ambiente.Devin-Org: engineering
Link to Devin session: https://devin-gtm.devinenterprise.com/sessions/7c6efa1d1d1d4d0f844f5a4bf0ce8f99
Open in Devin Desktop: https://devin-gtm.devinenterprise.com/desktop/session/7c6efa1d1d1d4d0f844f5a4bf0ce8f99?variant=devin
Devin Review