TrackMate v9

[tinevez]

TrackMate v9 - new major version

The main features we want to bring in the v9 are the following:

1/ Full support for 3D segmentation. 3D segmentation results will be represented by a surface object, and there will be a 3D viewer, and 3D morphological features.

2/ Change of file format. TrackMate v9 will ditch the XML file that has been in used since v2.2, and use GEFF instead, via GEFF-java

3/ All Python tools will be called using Appose. The previous way (introduced in v8) of calling Python
tools via command lines will be removed entirely.

Support for 3D segmentation

There is a lot of work already done for the support of 3D segmentation. The API support for feature 1/ should be ready to test. The 3D surface API is provided by imglib2-mesh. The 3D viewer relies on the bvv.

If you want to write a segmentation module that works in 3D, there are facilities to help. For instance, if the output of your segmentation algorithm is a 3D label image, you can use the facilities e.g. in SpotMeshUtils.
For an example, check the LabelImageDetector

This causes the most important API change. The core Spot class, that represents objects to track, is not a class anymore but an interface. In TrackMate there are now 3 implementations of this interface

• SpotBase that represents a spot with only X,Y,Z and radius coordinates (the original TrackMate <v7 spots)
• SpotRoi, that represents 2D spots that have a polygon as shapes. Introduced with TrackMate 7
• SpotMesh, spots with 3D shape, with a 3D mesh.

GEFF as the main file format

See #352

We are dropping the old XML file format in favor of GEFF. This will probably cause the most breaking changes to users, but I am convinced it is for the best, since quasi ALL tracking tools are adopting the same file format. So with this change, you will be able to run tracking in TrackMate and directly analyze the results in Python without the need to export to a specific file format. And conversely, you will be able to import results from another tracker and edit them in TrackMate.

This also means that we will remove the TrackMate code that was doing exports to and imports from other file formats, in an effort to shrink the code base.

However, it will still be possible to open TrackMate XML files in TrackMate v9.

Python interoperability with Appose

Appose is fantastic. So it will be the main way of adding integrating algorithms in TrackMate.
We thought initially of making a dedicated API in TrackMate to facilitate using Appose, but realized in the Pasteur hackathon that this is not the best idea. Paradoxically, TrackMate will not know of Appose.

Here is why: It is actually a better idea to write the integration of a Python algorithm at a lower level, ideally with plain imglib2. This way, the work done here can be reused many times, for instance in Fiji, in Icy, in TrackMate and in Mastodon.

The best example so far is the new Cellpose integration. There is a imglib2-cellpose artifact. This artifact depends only on Appose and imglib2, and is used in a Fiji plugin, in an Icy plugin and in the new version of the TrackMate-Cellpose module.

None of these 3 user-facing plugins know that the calls to Cellpose rely on Appose, and it is very good this way. For a user it means that there is only one environment installed on their disk to run Cellpose in any of these apps. This env is carefully brewed and maintained by only a few people, but used by many. We don't want to have one cellpose-appose for each Java app that wants Cellpose, duplicating efforts, wasting disk space.

We recommend using the same approach for other Python algos. If you want to integrate a Python algo, you can take inspiration from imglib2-cellpose
I give below a list of integration that have been started.

TrackMate modules with a Python algo that need to be (re)written with Appose

Algo	Repo	Status	Integration in TrackMate	Who
Cellpose	https://github.com/imglib/imglib2-cellpose	Done	Ready to test	@StRigaud @gletort @tinevez
StarDist	https://github.com/uw-loci/cellcast ?	?	Not done	@elevans @ctrueden
YOLO	https://github.com/imglib/imglib2-yolo	Not done	Not done	@tinevez @msphan @gletort
Omnipose	https://github.com/imglib/imglib2-omnipose	Not done	Not done	@tinevez
Trackastra	?	Not done	Not done	?
?	?	?	?	?