Scm/index descriptors - #199
Conversation
chiatt
left a comment
There was a problem hiding this comment.
This is a nice new feature, but it introduces a new pattern using a signal rather than the existing arches function (https://github.com/archesproject/arches-search/blob/main/arches_search/functions/search_indexing.py). This approach should really be refactored to add a new 'indexer' that is called by the function rather than using a signal.
I removed the signal and added this in to the post_save as transaction.on_commit. The descriptors update after the tile post save so needed to allow these to update before indexing as otherwise we will hit stale data. |
Issue
Search does not take into account the values from the resource descriptors and only take into account term indexed tile values.
This created an issue when we need to search via a number datatype node, which is not indexed into the term search. When a user searches for this number which is displayed in the descriptors nothing is returned.
Fix
Index the computed descriptor (
ResourceInstance.descriptors[lang]["name"]) intoTermSearchas its own term, under a reserved node alias__descriptor__. The descriptor is exactly the human-readable string a user searches for, so matchingit makes the resource findable regardless of which datatypes compose it.
No schema change: this reuses the existing
TermSearchtable.Why not a datatype indexer?
The existing indexers run per-tile / per-node. A descriptor is per-resource and is not a node value, so it does not fit the tile-indexer model. It is also only finalised after
Resource.save_descriptors()runs, which is after the tilepost_savefunctions (including the existingSearchIndexingFunction) have already fired - so a tile-level hook would index a stale descriptor.Changes
indexing/index_descriptor.py(new) -build_descriptor_terms(resource)/index_resource_descriptors(resource): builds oneTermSearchrow per language from the resource descriptor.tileidis NOT NULL but a descriptor is per-resource, so it borrows a real tile: the descriptor's name-nodegroup tile when the primarydescriptors config declares one, otherwise any tile. The tileid is only an FK/CASCADE anchor here - term search is keyed onresourceinstanceidand no nodegroup/permission filtering is applied to term rows - so the fallback is safe.signals.py(new) - apost_savereceiver onResource/ResourceInstancekeeps the descriptor term fresh incrementally.save_descriptors()persists the recomputed descriptor viasuper().save(), so the signal fires with the descriptor already current. Idempotent (delete-then-insert), guarded so it can never break a resource save.apps.py- connect the signal fromready().management/commands/arches_search.py-reindex_databasegains a descriptor pass after the tile pass, so a full reindex seeds descriptor terms too.tests/test_descriptor_indexing.py(new) - covers row creation with the borrowed tile, idempotency, and skipping empty /"Undefined"names.