Use get_language from tree-sitter-language-pack

[dhirschfeld]

Currently textual has a custom get_language function:

textual/src/textual/_tree_sitter.py

Lines 14 to 35 in 88cbc50

	def get_language(language_name: str) -> Language | None:
	if language_name in _LANGUAGE_CACHE:
	return _LANGUAGE_CACHE[language_name]
	try:
	module = import_module(f"tree_sitter_{language_name}")
	except ImportError:
	return None
	else:
	try:
	if language_name == "xml":
	# xml uses language_xml() instead of language()
	# it's the only outlier amongst the languages in the `textual[syntax]` extra
	language = Language(module.language_xml())
	else:
	language = Language(module.language())
	except (OSError, AttributeError):
	log.warning(f"Could not load language {language_name!r}.")
	return None
	else:
	_LANGUAGE_CACHE[language_name] = language
	return language

It might be easier to just depend on tree-sitter-language-pack and reuse the get_language function defined there?
https://github.com/Goldziher/tree-sitter-language-pack

[dhirschfeld]

Jsut opening this discussion to see if you'd be open to a PR making this change?

[TomJGooding]

The current function allows developers to install only tree-sitter and the languages they need, rather than a bunch of other transitive dependencies.

[dhirschfeld]

I use conda packages and a number of them aren't available but I can easily install tree-sitter-language-pack and if I do it would be good to automatically benefit from language support in Textual without having to install all the underlying packages individually (where they're even available in conda in the first place!)

[willmcgugan]

If its an optional dependency, I don't see why not. Feel free to open a PR for that.

[TomJGooding]

Perhaps I'm missing something, but TextArea.register_language looks like it should work for languages installed with tree-sitter-language-pack without any changes in Textual?

[dhirschfeld]

The current inplementation imports top-level modules. If tree-sitter-language-pack exposes the same interface (top-level modules for each language) then it would work as-is.

I'm not familiar with the tre-sitter ecosystem so that could well be the case - I'll defintely check and report back...

[dhirschfeld]

Perhaps I'm missing something, but TextArea.register_language looks like it should work for languages installed with tree-sitter-language-pack without any changes in Textual?

It appears that when tree-sitter-language-pack is installed, the Textual get_language function does not use it:

In [1]: import tree_sitter_language_pack as tslp

In [2]: tslp.get_language('python')
Out[2]: <Language id=130952907975872, version=15, name="python">

In [3]: from textual._tree_sitter import get_language

In [4]: get_language('python')

In [5]: get_language??
Signature: get_language(language_name: 'str') -> 'Language | None'
Docstring: <no docstring>
Source:
    def get_language(language_name: str) -> Language | None:
        if language_name in _LANGUAGE_CACHE:
            return _LANGUAGE_CACHE[language_name]

        try:
            module = import_module(f"tree_sitter_{language_name}")
        except ImportError:
            return None
        else:
            try:
                if language_name == "xml":
                    # xml uses language_xml() instead of language()
                    # it's the only outlier amongst the languages in the `textual[syntax]` extra
                    language = Language(module.language_xml())
                else:
                    language = Language(module.language())
            except (OSError, AttributeError):
                log.warning(f"Could not load language {language_name!r}.")
                return None
            else:
                _LANGUAGE_CACHE[language_name] = language
                return language
File:      ~/code/deleteme/.pixi/envs/default/lib/python3.14/site-packages/textual/_tree_sitter.py
Type:      function

In [6]:

[dhirschfeld]

One other benefit would be access to many more languages than are available individually, as binary wheels.

[TomJGooding]

But you'd also need the highlight queries for those Language objects to actually be useful for syntax highlighting?

[dhirschfeld]

Ah right. I guess it's just an alternative for the existing languages then.

[Fosox]

tree-sitter-language-pack have a get_highlights_query function, not all supported language have one but most of them do.

[dhirschfeld]

Put in a PR to add this functionality:

• Allow using tree-sitter-language-pack if installed #6344