Add reference_double_float and reference_triple_float schemas#103
Open
nkemnitz wants to merge 1 commit into
Open
Add reference_double_float and reference_triple_float schemas#103nkemnitz wants to merge 1 commit into
nkemnitz wants to merge 1 commit into
Conversation
Two reference annotation schemas for attaching a pair or triple of float values to another annotation (e.g. multiple per-synapse scores), complementing the single-value reference_tag_float. Both subclass ReferenceAnnotation (target_id + value fields, no tag) and register in type_mapping as reference_double_float / reference_triple_float. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🧔♂️Summary
I have three distinct "scores" for each of my synapses. Right now I use one
reference_floattable for each of those, but it's causing a lot of bloat. As I see it, I currently have 3 options:Not sure what the best approach is. The scoring system might change in the future (new column names, more or fewer columns), so I don't really want to create a new synapse schema, and a schema with column names specifically for this scored synapse table won't be of any use to anybody else (including future me)...
So I think the best short-term compromise is have a generic
reference_triple_floatcolumn, which helps a bit with storage size and has some reuseability, I hope?🤖 Summary
Adds two reference annotation schemas for attaching a pair or triple of float
values to another annotation — e.g. multiple per-synapse scores (detection mean/median +
assignment confidence) — complementing the existing single-value
reference_tag_float.ReferenceDoubleFloat→reference_double_float(value,value2)ReferenceTripleFloat→reference_triple_float(value,value2,value3)Both subclass
ReferenceAnnotation(so they carrytarget_id+valid, notag) and areregistered in
type_mapping. Thevalue/value2/value3naming follows the existingmulti-field convention (e.g.
bound_double_tag'stag/tag2).Motivation
Storing N independent floats per annotation currently needs N separate
reference_tag_floattables — one row plus full annotation overhead per value. A single multi-float reference
collapses that into one row, materially reducing storage and query overhead when every target
annotation carries the same fixed set of scalars.
Changes
emannotationschemas/schemas/reference_float.py— the two schema classesemannotationschemas/__init__.py— imports +type_mappingentriestests/test_reference_float_schema.py— load/validation + registry testsTesting
The 4 new tests pass, and flattening yields the expected columns:
reference_double_float→target_id, valid, value, value2reference_triple_float→target_id, valid, value, value2, value3🤖 Generated with Claude Code