Skip to content

Repository loading: update load from diff mechanism - #6871

Draft
rjbou wants to merge 11 commits into
ocaml:masterfrom
rjbou:update-load-from-diff-rw
Draft

Repository loading: update load from diff mechanism#6871
rjbou wants to merge 11 commits into
ocaml:masterfrom
rjbou:update-load-from-diff-rw

Conversation

@rjbou

@rjbou rjbou commented Mar 24, 2026

Copy link
Copy Markdown
Collaborator

It simplifies the readability of the code and no longer loads unrelated paths.

TODO:

  • add changelog
  • check documentation
  • update with new code
  • sanitise functions

In the story of #6625

Comment thread src/repository/opamRepositoryPath.ml Outdated
@rjbou
rjbou force-pushed the update-load-from-diff-rw branch from 40a76f6 to bbdea9b Compare March 25, 2026 16:32
@kit-ty-kate

Copy link
Copy Markdown
Member

(before incremental loading)

does this mean the test results are from 2.4.1?

@rjbou

rjbou commented Mar 25, 2026

Copy link
Copy Markdown
Collaborator Author

at commit f370351, just before incremental loading merge

@kit-ty-kate kit-ty-kate left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partial review (i'll look at the more complicated load_opams_from_diff later)

Comment thread src/core/opamFilename.ml Outdated
Filename.concat (Dir.to_string t.dirname) (Base.to_string t.basename)

let to_list t =
(String.split_on_char Filename.dir_sep.[0] t.dirname)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm not a huge fan of this pattern but if we have to use it i think we should put it in a separate internal function so that we can at least use it for OpamFilename.link. I would rather use the implementation from link instead of this one in case we ever get problems with multiple directory separators (e.g. a//b == a/b)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the end, we only need to split on "packages" and "files" (nv can be retrieved with dirname), so maybe we don't need this function in the end

Comment thread src/repository/opamRepositoryPath.mli Outdated
let install_nv_dir filename =
let rec find_files prefix_files tail =
match tail with
| "files"::_ -> Some prefix_files

@kit-ty-kate kit-ty-kate Mar 25, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we'd also check that files is a directory and that there exist a valid opam file at the same level named opam

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree, but i don't know if we should do it here or in a wrapping function. For plain repo, it is simple to check, but for tarred repo, we'll need to take all the other information (files, contents) to be able to have if. Maybe a check function ?

Comment thread src/repository/opamRepositoryPath.ml Outdated
| Some pkg ->
let dir =
OpamFilename.Dir.of_list
(List.rev (nv :: prefix_files @ ("packages"::pre)))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm unsure what pre is here. If it is because filename can be an absolute filename i think we should avoid that and always start at the repository root directory. Otherwise it's a recipe for disaster (e.g. users are allowed to put their opam root in a directory called packages)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ftm it is used directly with paths from diff file, so from repo root directly. But we can restrict it yes

@kit-ty-kate

Copy link
Copy Markdown
Member

at commit f370351, just before incremental loading merge

i think it would be great to have that information in the commit message

@rjbou

rjbou commented Mar 25, 2026

Copy link
Copy Markdown
Collaborator Author

general notes:

  • i'm not fan of the functions i've added (filename and repo path), it's a middly quick and dirty way (it works ™️), very open to other way to do it :D
  • modification in this PR need to take account of the modification needed by 6625, some adjustment may occur

Comment thread src/core/opamFilename.ml Outdated
Filename.concat (Dir.to_string t.dirname) (Base.to_string t.basename)

let to_list t =
(String.split_on_char Filename.dir_sep.[0] t.dirname)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the end, we only need to split on "packages" and "files" (nv can be retrieved with dirname), so maybe we don't need this function in the end

let install_nv_dir filename =
let rec find_files prefix_files tail =
match tail with
| "files"::_ -> Some prefix_files

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree, but i don't know if we should do it here or in a wrapping function. For plain repo, it is simple to check, but for tarred repo, we'll need to take all the other information (files, contents) to be able to have if. Maybe a check function ?

Comment thread src/repository/opamRepositoryPath.ml Outdated
| Some pkg ->
let dir =
OpamFilename.Dir.of_list
(List.rev (nv :: prefix_files @ ("packages"::pre)))

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ftm it is used directly with paths from diff file, so from repo root directly. But we can restrict it yes

Comment thread src/repository/opamRepositoryPath.mli Outdated
Comment thread src/repository/opamRepositoryPath.mli Outdated
Comment thread tests/reftests/repository-formats.test Outdated
Comment thread tests/reftests/repository-formats.test Outdated
Comment thread master_changes.md Outdated
Comment thread master_changes.md Outdated
Comment thread master_changes.md Outdated
rjbou added a commit to rjbou/opam that referenced this pull request Mar 26, 2026
…nd constructor (OpamFilename.Dir.of_list) to and from string list
rjbou added a commit to rjbou/opam that referenced this pull request Mar 26, 2026
…tion (package, main directory) of a filename if it is an extra file
rjbou added a commit to rjbou/opam that referenced this pull request Mar 26, 2026
rjbou added a commit to rjbou/opam that referenced this pull request Mar 26, 2026
rjbou added a commit to rjbou/opam that referenced this pull request Mar 26, 2026
rjbou added a commit to rjbou/opam that referenced this pull request Mar 26, 2026
rjbou added a commit to rjbou/opam that referenced this pull request Mar 27, 2026
…nd constructor (OpamFilename.Dir.of_list) to and from string list
rjbou added a commit to rjbou/opam that referenced this pull request Mar 27, 2026
…tion (package, main directory) of a filename if it is an extra file
rjbou added a commit to rjbou/opam that referenced this pull request Mar 27, 2026
rjbou added a commit to rjbou/opam that referenced this pull request Mar 27, 2026
rjbou added a commit to rjbou/opam that referenced this pull request Mar 27, 2026
rjbou added a commit to rjbou/opam that referenced this pull request Mar 27, 2026
@rjbou
rjbou force-pushed the update-load-from-diff-rw branch from 17de064 to aa59255 Compare April 29, 2026 20:03
@rjbou

rjbou commented Apr 29, 2026

Copy link
Copy Markdown
Collaborator Author

Updated

@rjbou
rjbou force-pushed the update-load-from-diff-rw branch from aa59255 to d666138 Compare April 30, 2026 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants