chore: remove leftovers inherited from Arrow - #2474
NoahKusaba wants to merge 5 commits into
Conversation
- Drop .gitignore and rat_exclude_files.txt entries for files this repository does not have (C++, R, flatbuffers, parquet-testing, ...). - Drop the prettier exclusion for a ballista/CHANGELOG.md that does not exist, and document the pinned prettier command CI actually runs. - Delete the root take.yml, a diverged duplicate of the live workflow in .github/workflows/. - Document dev/update_datafusion_versions.py, which nothing referenced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
github-actions
Bot
added
documentation
andygrove
left a comment
andygrove
left a comment
There was a problem hiding this comment.
Thanks for splitting this out, much easier to review! The Arrow cleanup looks good to me. Left one comment on the new script docs and a small nit.
| that ports any DataFusion API changes, refreshes affected golden files, and keeps `cargo test` and | ||
| `cargo clippy --all-targets --workspace -- -D warnings` green. | ||
|
|
||
| `dev/update_datafusion_versions.py` rewrites the `datafusion*` entries across the workspace for you. |
There was a problem hiding this comment.
I tried this on a clean checkout of main (uv run --no-project --with tomlkit dev/update_datafusion_versions.py 56.0.0) and it only changed python/Cargo.toml and python/pyproject.toml. The real pins live in [workspace.dependencies] in the root Cargo.toml, and the member crates all use workspace = true, so the script skips them. Its crates list is also still the upstream DataFusion one (datafusion-jit, datafusion-row, and so on) and is missing datafusion-spark, datafusion-substrait, datafusion-proto-common and datafusion-functions-aggregate-common.
Could we either leave this doc out of this PR, or fix the script to update the root manifest first? Happy to see that as a follow-up.
There was a problem hiding this comment.
Good catch, and thanks for actually running it. You're right: the pins all live in [workspace.dependencies], which the script never reads, and its crate list is still DataFusion's own. I've dropped the doc from this PR so it doesn't send anyone to a script that doesn't work, and I'll fix the script in a follow-up.
| *.iml | ||
|
|
||
| # Linux perf sample data | ||
| perf.data |
There was a problem hiding this comment.
Small nit, feel free to ignore. perf.data isn't really Arrow specific and folks do run perf against the executor, so it might be worth keeping.
There was a problem hiding this comment.
Fair point, restored. That made me recheck the other removals for the same thing, and I'd also dropped python/examples/.ipynb_checkpoints, which isn't Arrow-specific either since we ship notebooks there. It's back too, as .ipynb_checkpoints/ so it covers any directory.
….py docs perf.data is not Arrow-specific; perf is run against the executor. dev/update_datafusion_versions.py does not update the root manifest's [workspace.dependencies], where the DataFusion pins live, so documenting it would point contributors at a script that does not work. Fixing it is left to a follow-up. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
python/examples/ has notebooks, and opening them creates .ipynb_checkpoints/, so that ignore was not Arrow-specific. Generalise it from python/examples/ to any directory. Also group *.iml with .idea/ and drop a comment line left empty in dev.yml. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Four rust.yml jobs exported ARROW_TEST_DATA and PARQUET_TEST_DATA pointing at testing/ and parquet-testing/, which this repository does not have, and checked out submodules it does not define. Nothing reads either variable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
github-actions
Bot
removed
the
documentation
|
Thanks for the review. Addressed both comments, and while I was at it I added one more Arrow leftover: four |
3 participants
Rationale for this change
Some repository config still refers to files and test data from Apache Arrow, where this project started, and never matches anything here.
What changes are included in this PR?
.gitignoreentries for files this repository does not produce: C++, R, Visual Studio, flatbuffers, and Arrow's docker and packaging directories. Entries that aren't Arrow-specific stay:perf.data(perf is run against the executor) and Jupyter's.ipynb_checkpoints/, sincepython/examples/has notebooks. The checkpoints entry now covers any directory, not onlypython/examples/.dev/release/rat_exclude_files.txtentries that match no file in the repository, such asci/vcpkg,helm/,*.pxdandparquet-testing.rust.yml: four jobs exportedARROW_TEST_DATAandPARQUET_TEST_DATA, pointing attesting/andparquet-testing/, which this repository does not have, and checked out submodules it does not define. Nothing reads either variable. Remove both.dev.ymlfor aballista/CHANGELOG.mdthat does not exist. Have CONTRIBUTING show the pinnednpx prettier@2.7.1command CI actually runs, rather than a global install that may format differently.take.yml, a diverged duplicate of the live workflow in.github/workflows/. GitHub never ran the root copy.Are there any user-facing changes?
No.
Checked:
.gitignorepattern un-ignores a file in a built working tree or matches a tracked file.rust.ymlanddev.ymlparse.Not changed here:
dev/release/verify-release-candidate.shstill clonesarrow-testingandparquet-testingand exports the same variables. Since it is part of the release process, it is better done separately.Split out of #2438 to make it easier to review.
🤖 Generated with Claude Code