moving forward

[tpapp]

Now that #28 is merged and we have a test setup, I am wondering how to move forward with updating this repository.

Here is a proposal about things I would want to do, but I am open to suggestions about other things, and/or the order.

1. Update the grammar. Currently it is 0.23.1, the "official" repo is at 0.25.0, there is a more up-to-date fork by @clason with 0.26.6. I could not find a changelog so we have to figure out what changed from the code, or ask the maintainers.

2. Go though open issues in julia-emacs, see if they are still present with tree-sitter. If not, label those issues with something like fixed-by-tree-sitter, we can close them en masse at some point when this becomes the recommended solution. Add tests #24 should be consulted.

I am pinging maintainers of various repos mentioned, and people who made recent contributions, @savq @clason @dhanak @aviatesk, any advice would be appreciated.

[clason]

• tree-sitter/tree-sitter-julia@master...tree-sitter-grammars:tree-sitter-julia:master
• tree-sitter-grammars/tree-sitter-julia@master...tree-sitter:tree-sitter-julia:master

(That fork is maintained; there's just nothing to do right now. But it looks like all our changes have been backported by now.)

Fair warning: There are internal discussions about the future of "upstream" tree-sitter grammar repos; some are under the org because they are used as test fixtures in CI (so there is a desire to have control over them), but many are there simply for historical reasons and nothing but a burden on the current maintainers. I consider the Julia parser to be one of them.

[dhanak]

Thanks, @clason, for the heads up. IIUC, we can safely switch over to tree-sitter-grammars/tree-sitter-julia, because this grammar is at least as up-to-date as the original repo, and you have all intentions to keep it this way, too.

[clason]

Yes, the fork is a drop-in replacement (meaning queries are fully compatible; I dropped most bindings but can add back individual ones on reasonable request) and currently up-to-date with the upstream project (which I follow, in case there are updates I want to cherry-pick).

[dhanak]

Hmm, I'm confused. If the tree-sitter-grammars version is supposedly 0.26.6, then why do package.json and tree-sitter.json still claim "version": "0.23.1"?

[clason]

I don't publish my fork, so I don't worry about the grammar version.

(Don't confuse the grammar version and the tree-sitter version used to generate it; these are completely independent.)

[dhanak]

I asked the AI to summarize changes from 0.23.1 to 0.25.0, and how they affect this codebase. Here's what it said:

v0.25.0 is latest release, dated Nov 8 2025, and GitHub compare shows 35 commits / 46 files changed from v0.23.1 to v0.25.0. Sources: release page and compare page. (lists.gnu.org (https://lists.gnu.org/r/emacs-devel/2025-04/msg00508.html))

Must Update

• julia-ts-mode.el:46: bump grammar pin only after query fixes.
  Current pin: v0.23.1.

• julia-ts-mode.el:160: let_binding is gone.
  v0.25.0 removed node types:
  let_binding, named_argument, named_field, scoped_identifier, anonymous "=".
  New node types:
  block, generator.

• julia-ts-mode.el:165: named_argument is gone.
  Keyword args are now parsed as assignment inside argument_list.
  Existing generic assignment highlighting may now catch keyword args as variables unless adjusted.

• julia-ts-mode.el:233: second let_binding query also invalid.
  This will make tree-sitter query compilation fail with v0.25.0.

• julia-ts-mode.el:343: indentation needs visible block.
  v0.25.0 wraps bodies of function, macro, struct, if, for, while, try, begin, quote, etc. in a visible block.
  Current indent rules rely on children being direct under _statement, _definition, _clause. That assumption is now false.

Likely Update

• Generator forms now have visible generator nodes. Commit says generators were made visible and _comprehension_clause was replaced by generator.
  Existing el does not directly query for_clause under tuple/argument lists, so not fatal here. Still test multiline generators.

• macro_identifier shape changed.
  scoped_identifier was hidden/replaced by field_expression for qualified macros. Existing (macro_identifier) @julia-ts-macro-face still compiles, but qualified macro highlighting should be tested.

• Upstream highlights added/changed:
  pipe call highlighting, updated builtin functions/types, :: handling, modifier keywords, query files for locals.scm and injections.scm.

Non-el Breaking

• Parser ABI changed: LANGUAGE_VERSION 14 -> 15.
  Emacs linked against older libtree-sitter may reject ABI 15 grammars. GNU emacs-devel notes pre-0.25 tree-sitter supports ABI up to 14, while tree-sitter 0.25 supports ABI 15. (lists.gnu.org
  (https://lists.gnu.org/r/emacs-devel/2025-04/msg00508.html))
  Docs should warn users to check:
  M-: (treesit-library-abi-version)

Bottom Line
Do not only bump the pin. Minimum safe patch:

1. Replace let_binding queries.
2. Replace named_argument query with assignment under argument_list.
3. Prevent keyword-argument assignments from being mis-highlighted as normal LHS variables.
4. Add visible block indentation handling.
5. Test font-lock corpus against grammar v0.25.0.
6. Document ABI 15 caveat.

[tpapp]

@clason,

I don't publish my fork. (Make sure to check the right repository on Github.)

would you mind linking what you think the right repository is? I am now throughly confused.

[dhanak]

This ABI version bump from 14 to 15 is something we need to explore in depth: Emacs version 30.1 reports (treesit-library-abi-version) to be 14.

[clason]

would you mind linking what you think the right repository is? I am now throughly confused.

Sorry, I thought that was clear since we're only talking about two repositories, one of which is forked from the other. The fork that I am maintaining (that is up-to-date with tree-sitter 0.26.x (which implies ABI 15+, same as upstream HEAD) but does not have a release and hence no new version) is https://github.com/tree-sitter-grammars/tree-sitter-julia, of course.

(And you absolutely want to track tree-sitter (lib) updates in Emacs. You're missing out on crucial bugfixes and significant performance improvements otherwise, and older ABIs will age out eventually. There's about 160 parsers on ABI 15 that I know about alone (which is about half of them).)

[dhanak]

So, @clason keeps tree-sitter-grammar up-to-date, but there are no releases, so we can't lock julia-ts-mode to a specific version (as it is done now). In that case, I think I'd prefer to stick to the official release (as long as it exists), and update to version 0.25.0 for now. I'm still concerned about the ABI update, though. I need to understand what that means and what effect it has on earlier versions of Emacs, and if there's a compatibility issue, then can we work around it?

[clason]

You can always lock to a specific commit; this is what all other editors do.

(The vast majority of tree-sitter parsers have no release policy -- let alone semver -- so every commit has to be assumed to be breaking.)

[tpapp]

I am still figuring out tree-sitter, but I am under the impression that a number of previous ABI versions can be generated. Specifically, with 15 we can still generate 13.

So the question is: which Emacs version supports at least 13? I am under the impression that 29.1 works with 14, but it needs to be checked.

[dhanak]

I have a dream. We, as a community, have at least two de-facto standard tools for formatting Julia code automatically: JuliaFormatter and Runic. The former accepts a configuration file; the latter uses a built-in, opinionated configuration. What if we modified julia-ts-mode such that the user was able to select which formatter to mimic, and in case of JuliaFormatter, it would also parse the configuration file in the project root (if it exists). Then, the mode would adjust its rules to indent the code adhering to these formatting conventions, as much as possible. I'm aware that these formatters do a lot more than indenting, but at least we could copy their indentation rules. WDYT?

[tpapp]

I am open to everything but my priority is to get treesitter working and integrate it with julia-mode, so that we can close various issues.

[dhanak]

I asked the AI about this ABI compatibility concern. In brief:

• the 0.25 tree-sitter parser supports ABI v13-15;
• the 0.25 generator, OTOH, can generate ABI v14-15 only;
• the ABI version is fixed in the generated parser.c, which is part of the tree-sitter-julia release/distribution/source;
• if we want to support older emacsen with pre 0.25 tree-sitter, we need to maintain an ABI v14 mirror of the tree-sitter-julia source tree.

In slightly more detail:

Handle it as runtime capability, not Emacs version.

Best compatibility contract:

Emacs package: loads on Emacs 29.1+
julia-ts-mode: works when tree-sitter runtime supports needed parser ABI
grammar target: tree-sitter-julia 0.25.0 parse tree

Recommended matrix:

treesit max ABI >= 15  -> use upstream tree-sitter-julia v0.25.0
treesit max ABI == 14  -> use v0.25.0 regenerated with --abi 14
treesit max ABI == 13  -> unsupported, fall back to julia-mode / clear error

Reason: current CLI docs list ABI 14 as older generation target. ABI 13 is loadable by newer runtimes, but not currently a practical v0.25 generation target per generate --abi docs.

Implementation shape:

(defun julia-ts--treesit-abi-max ()
  (and (fboundp 'treesit-library-abi-version)
       (treesit-library-abi-version)))

(defconst julia-ts--grammar-source
  (let ((abi (julia-ts--treesit-abi-max)))
    (cond
     ((and abi (>= abi 15))
      '(julia "https://github.com/tree-sitter/tree-sitter-julia" "v0.25.0"))
     ((and abi (>= abi 14))
      '(julia "https://github.com/JuliaEditorSupport/tree-sitter-julia"
              "v0.25.0-abi14"))
     (t nil))))

Then only add source when non-nil. For ABI 14, maintain a small fork/tag/branch whose only meaningful difference is regenerated src/parser.c with:

tree-sitter generate --abi 14

Typical Emacs package practice:

• Package-Requires covers Lisp API only, e.g. ((emacs "29.1")).
• Native libraries, grammar ABI, executables: checked at runtime.
• Package load must not fail just because optional native support is absent.
• Major mode activation may user-error with exact missing capability.
• Use fboundp, boundp, treesit-available-p, treesit-language-available-p.
• Keep docs with external dependency matrix.

For this package, do not support old grammar and new queries in one path unless you want dual query sets. Cleaner: require v0.25 parse tree; provide ABI15 and ABI14 builds of that grammar; reject ABI13.

Sources:

• https://tree-sitter.github.io/tree-sitter/using-parsers/7-abi-versions.html
• https://tree-sitter.github.io/tree-sitter/cli/generate.html

I also asked about Emacs versions vs. ABI:

Emacs 29.1 with tree-sitter enabled supports at least ABI 13.

Reason: Emacs 29.1 build accepts tree-sitter >=0.20.2; tree-sitter 0.20.2 supports ABI 13..13.

ABI 14 on Emacs 29.1 depends on linked tree-sitter >=0.20.3.
ABI 15 depends on linked tree-sitter >=0.25.

Check real runtime, not Emacs version:

(list (treesit-library-abi-version t)
      `(treesit-library-abi-version))`

For Julia grammar v0.25.0: best portable target is ABI 14, not 13. It should cover Emacs 29.1 builds linked with tree-sitter 0.20.3+, but not rare/minimal 0.20.2 builds.