Skip to content

statedb: handle empty index name in queries - #169

Draft
tklauser wants to merge 3 commits into
mainfrom
pr/tklauser/empty-index-name
Draft

statedb: handle empty index name in queries#169
tklauser wants to merge 3 commits into
mainfrom
pr/tklauser/empty-index-name

Conversation

@tklauser

Copy link
Copy Markdown
Member

Querying a table with an empty index name currently panics with an "index out of range" runtime error. This is because (*genTable[Obj]).indexPos reads name[0] without a length check.

This is reachable from normal API usage: NetIPPrefixIndex.QueryFromObject and LPMIndex.QueryFromObject return a zero-value Query (with an empty index name) when the object yields no prefixes or keys, and passing such a query to Get, List, Prefix or LowerBound panics.

Fix this by treating an empty index name as a reference to the primary index in indexPos, following the existing behavior in getIndexer.

@tklauser
tklauser requested a review from pippolo84 June 26, 2026 08:49
@tklauser
tklauser requested a review from a team as a code owner June 26, 2026 08:49
@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown
$ make
go build ./...
go: downloading github.com/cilium/hive v1.0.4
go: downloading golang.org/x/time v0.15.0
go: downloading go.yaml.in/yaml/v3 v3.0.4
go: downloading github.com/spf13/cobra v1.10.2
go: downloading github.com/spf13/pflag v1.0.10
go: downloading github.com/cilium/stream v0.0.1
go: downloading github.com/liggitt/tabwriter v0.0.0-20181228230101-89fcab3d43de
go: downloading github.com/spf13/viper v1.18.2
go: downloading go.uber.org/dig v1.17.1
go: downloading golang.org/x/term v0.16.0
go: downloading github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading golang.org/x/sys v0.17.0
go: downloading golang.org/x/tools v0.17.0
go: downloading github.com/spf13/cast v1.6.0
go: downloading github.com/fsnotify/fsnotify v1.7.0
go: downloading github.com/sagikazarmark/slog-shim v0.1.0
go: downloading github.com/spf13/afero v1.11.0
go: downloading github.com/subosito/gotenv v1.6.0
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading gopkg.in/ini.v1 v1.67.0
go: downloading github.com/magiconair/properties v1.8.7
go: downloading github.com/pelletier/go-toml/v2 v2.1.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading golang.org/x/text v0.14.0
STATEDB_VALIDATE=1 go test ./... -cover -vet=all -test.count 1
go: downloading github.com/stretchr/testify v1.11.1
go: downloading go.uber.org/goleak v1.3.0
go: downloading golang.org/x/exp v0.0.0-20240119083558-1b970713d09a
go: downloading github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2
ok  	github.com/cilium/statedb	403.899s	coverage: 78.8% of statements
ok  	github.com/cilium/statedb/index	0.004s	coverage: 33.7% of statements
ok  	github.com/cilium/statedb/internal	0.019s	coverage: 42.9% of statements
ok  	github.com/cilium/statedb/lpm	3.537s	coverage: 77.9% of statements
ok  	github.com/cilium/statedb/part	61.702s	coverage: 87.3% of statements
ok  	github.com/cilium/statedb/reconciler	0.290s	coverage: 92.5% of statements
	github.com/cilium/statedb/reconciler/benchmark		coverage: 0.0% of statements
	github.com/cilium/statedb/reconciler/example		coverage: 0.0% of statements
go test -race ./... -test.count 1
ok  	github.com/cilium/statedb	39.762s
ok  	github.com/cilium/statedb/index	1.013s
ok  	github.com/cilium/statedb/internal	1.029s
ok  	github.com/cilium/statedb/lpm	2.672s
ok  	github.com/cilium/statedb/part	35.688s
ok  	github.com/cilium/statedb/reconciler	1.409s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go test ./... -bench . -benchmem -test.run xxx
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb
cpu: AMD EPYC 9V74 80-Core Processor                
BenchmarkDB_WriteTxn_1-4                      	  665772	      1707 ns/op	    585707 objects/sec	    1016 B/op	      17 allocs/op
BenchmarkDB_WriteTxn_10-4                     	 1858098	       647.9 ns/op	   1543464 objects/sec	     521 B/op	       8 allocs/op
BenchmarkDB_WriteTxn_100-4                    	 2206436	       593.6 ns/op	   1684620 objects/sec	     536 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_1000-4                   	 1916460	       649.3 ns/op	   1540195 objects/sec	     522 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_100_SecondaryIndex-4     	  770366	      1387 ns/op	    721130 objects/sec	    1100 B/op	      20 allocs/op
BenchmarkDB_WriteTxn_CommitOnly_100Tables-4   	  949394	      1219 ns/op	    1128 B/op	       6 allocs/op
BenchmarkDB_WriteTxn_CommitOnly_1Table-4      	 1472521	       813.5 ns/op	     240 B/op	       6 allocs/op
BenchmarkDB_NewWriteTxn-4                     	 1660851	       721.8 ns/op	     216 B/op	       5 allocs/op
BenchmarkDB_WriteTxnCommit100-4               	  867204	      1208 ns/op	    1112 B/op	       6 allocs/op
BenchmarkDB_NewReadTxn-4                      	566612913	         2.121 ns/op	       0 B/op	       0 allocs/op
BenchmarkDB_Modify-4                          	    1641	    697077 ns/op	   1434561 objects/sec	  546098 B/op	    8095 allocs/op
BenchmarkDB_GetInsert-4                       	    1570	    755584 ns/op	   1323480 objects/sec	  530086 B/op	    8095 allocs/op
BenchmarkDB_RandomInsert-4                    	    1873	    636851 ns/op	   1570226 objects/sec	  522043 B/op	    7095 allocs/op
BenchmarkDB_RandomReplace-4                   	     430	   2832638 ns/op	    353028 objects/sec	 2073417 B/op	   29147 allocs/op
BenchmarkDB_SequentialInsert-4                	    1902	    629142 ns/op	   1589467 objects/sec	  522044 B/op	    7095 allocs/op
BenchmarkDB_SequentialInsert_Prefix-4         	     496	   2430390 ns/op	    411457 objects/sec	 3562808 B/op	   45542 allocs/op
BenchmarkDB_Changes_Baseline-4                	    1639	    744448 ns/op	   1343278 objects/sec	  582232 B/op	    9187 allocs/op
BenchmarkDB_Changes-4                         	     981	   1232146 ns/op	    811592 objects/sec	  783564 B/op	   12338 allocs/op
BenchmarkDB_RandomLookup-4                    	   25161	     46846 ns/op	  21346593 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_SequentialLookup-4                	   24732	     48597 ns/op	  20577579 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_Prefix_SecondaryIndex-4           	    7270	    149936 ns/op	   6669518 objects/sec	  124920 B/op	    1025 allocs/op
BenchmarkDB_FullIteration_All-4               	     646	   1795099 ns/op	  55707230 objects/sec	     104 B/op	       4 allocs/op
BenchmarkDB_FullIteration_Prefix-4            	     715	   1585267 ns/op	  63080846 objects/sec	     136 B/op	       5 allocs/op
BenchmarkDB_FullIteration_Get-4               	     168	   7397646 ns/op	  13517813 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_Get_Secondary-4     	      72	  15465733 ns/op	   6465908 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_ReadTxnGet-4        	     154	   7779440 ns/op	  12854396 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_PropagationDelay-4                	  649113	      1641 ns/op	        14.00 50th_µs	        17.00 90th_µs	        77.00 99th_µs	    1123 B/op	      20 allocs/op
BenchmarkDB_WriteTxn_100_LPMIndex-4           	  502567	      2435 ns/op	    410655 objects/sec	    1825 B/op	      37 allocs/op
BenchmarkDB_WriteTxn_1_LPMIndex-4             	  131870	     15374 ns/op	     65045 objects/sec	   15778 B/op	      82 allocs/op
BenchmarkDB_LPMIndex_Get-4                    	     345	   3468587 ns/op	   2883018 objects/sec	       0 B/op	       0 allocs/op
BenchmarkWatchSet_4-4                         	 2402666	       492.6 ns/op	     296 B/op	       4 allocs/op
BenchmarkWatchSet_16-4                        	  774174	      1555 ns/op	    1096 B/op	       5 allocs/op
BenchmarkWatchSet_128-4                       	   86371	     13914 ns/op	    8904 B/op	       5 allocs/op
BenchmarkWatchSet_1024-4                      	    8522	    141726 ns/op	   73744 B/op	       5 allocs/op
PASS
ok  	github.com/cilium/statedb	43.724s
PASS
ok  	github.com/cilium/statedb/index	0.009s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/internal
cpu: AMD EPYC 9V74 80-Core Processor                
Benchmark_SortableMutex-4   	 4984530	       241.4 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/cilium/statedb/internal	1.213s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/lpm
cpu: AMD EPYC 9V74 80-Core Processor                
Benchmark_txn_insert/batchSize=1-4         	    1964	    605051 ns/op	   1652754 objects/sec	  838412 B/op	   13975 allocs/op
Benchmark_txn_insert/batchSize=10-4        	    3394	    364210 ns/op	   2745666 objects/sec	  385198 B/op	    6668 allocs/op
Benchmark_txn_insert/batchSize=100-4       	    3511	    341356 ns/op	   2929490 objects/sec	  345615 B/op	    6027 allocs/op
Benchmark_txn_delete/batchSize=1-4         	    1566	    753768 ns/op	   1326668 objects/sec	 1286470 B/op	   13976 allocs/op
Benchmark_txn_delete/batchSize=10-4        	    3182	    370015 ns/op	   2702593 objects/sec	  372417 B/op	    5769 allocs/op
Benchmark_txn_delete/batchSize=100-4       	    3561	    330406 ns/op	   3026579 objects/sec	  286753 B/op	    5038 allocs/op
Benchmark_LPM_Lookup-4                     	    7728	    153740 ns/op	   6504499 objects/sec	       0 B/op	       0 allocs/op
Benchmark_LPM_All-4                        	  140799	      8555 ns/op	 116894748 objects/sec	      32 B/op	       1 allocs/op
Benchmark_LPM_Prefix-4                     	  135200	      8886 ns/op	 112540805 objects/sec	      32 B/op	       1 allocs/op
Benchmark_LPM_LowerBound-4                 	  249069	      4689 ns/op	 106629751 objects/sec	     288 B/op	       2 allocs/op
PASS
ok  	github.com/cilium/statedb/lpm	11.934s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/part
cpu: AMD EPYC 9V74 80-Core Processor                
Benchmark_Uint64Map_Random-4                  	    1609	    746833 ns/op	   1338988 items/sec	 2525024 B/op	    6039 allocs/op
Benchmark_Uint64Map_Sequential-4              	    1921	    620211 ns/op	   1612355 items/sec	 2216721 B/op	    5754 allocs/op
Benchmark_Uint64Map_Sequential_Insert-4       	    2127	    569708 ns/op	   1755285 items/sec	 2208718 B/op	    4753 allocs/op
Benchmark_Uint64Map_Sequential_Txn_Insert-4   	   10000	    105115 ns/op	   9513397 items/sec	   86352 B/op	    2028 allocs/op
Benchmark_Uint64Map_Random_Insert-4           	    1740	    684621 ns/op	   1460663 items/sec	 2518721 B/op	    5025 allocs/op
Benchmark_Uint64Map_Random_Txn_Insert-4       	    6480	    165641 ns/op	   6037163 items/sec	  119164 B/op	    2421 allocs/op
Benchmark_Insert_RootOnlyWatch-4              	   10000	    101844 ns/op	   9818897 objects/sec	   71504 B/op	    2033 allocs/op
Benchmark_Insert-4                            	    8287	    149672 ns/op	   6681294 objects/sec	  186937 B/op	    3060 allocs/op
Benchmark_Modify-4                            	   13140	     91173 ns/op	  10968130 objects/sec	   58224 B/op	    1007 allocs/op
Benchmark_GetInsert-4                         	    9988	    115501 ns/op	   8657955 objects/sec	   58224 B/op	    1007 allocs/op
Benchmark_Replace-4                           	32160358	        37.58 ns/op	  26613156 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Replace_RootOnlyWatch-4             	31961970	        37.09 ns/op	  26963265 objects/sec	       0 B/op	       0 allocs/op
Benchmark_txn_1-4                             	 5778738	       206.6 ns/op	   4840013 objects/sec	     168 B/op	       3 allocs/op
Benchmark_txn_10-4                            	10919932	       108.5 ns/op	   9214522 objects/sec	      86 B/op	       2 allocs/op
Benchmark_txn_100-4                           	12917178	        90.52 ns/op	  11047369 objects/sec	      80 B/op	       2 allocs/op
Benchmark_txn_1000-4                          	11417246	       102.8 ns/op	   9725284 objects/sec	      65 B/op	       2 allocs/op
Benchmark_txn_delete_1-4                      	 5131926	       235.3 ns/op	   4250636 objects/sec	     664 B/op	       4 allocs/op
Benchmark_txn_delete_10-4                     	12289430	        96.15 ns/op	  10400335 objects/sec	     106 B/op	       1 allocs/op
Benchmark_txn_delete_100-4                    	13995315	        84.25 ns/op	  11870115 objects/sec	      47 B/op	       1 allocs/op
Benchmark_txn_delete_1000-4                   	15937443	        74.28 ns/op	  13461769 objects/sec	      24 B/op	       1 allocs/op
Benchmark_Get-4                               	   47876	     25137 ns/op	  39781271 objects/sec	       0 B/op	       0 allocs/op
Benchmark_All-4                               	  144432	      8541 ns/op	 117084869 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterator_All-4                      	  133036	      9079 ns/op	 110148891 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterator_Next-4                     	  160261	      7217 ns/op	 138558013 objects/sec	     896 B/op	       1 allocs/op
Benchmark_Hashmap_Insert-4                    	   17545	     68292 ns/op	  14643090 objects/sec	   74264 B/op	      20 allocs/op
Benchmark_Hashmap_Get_Uint64-4                	  154057	      7794 ns/op	 128311188 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Hashmap_Get_Bytes-4                 	  124454	      9630 ns/op	 103844866 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Delete_Random-4                     	      57	  17871149 ns/op	   5595611 objects/sec	 2111981 B/op	  102364 allocs/op
Benchmark_find16-4                            	225254499	         5.507 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex16-4                       	100000000	        14.05 ns/op	       0 B/op	       0 allocs/op
Benchmark_find48-4                            	426146332	         2.816 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex48_hit-4                   	425036343	         2.824 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex48_miss-4                  	378415564	         3.181 ns/op	       0 B/op	       0 allocs/op
Benchmark_find4-4                             	421121660	         2.904 ns/op	       0 B/op	       0 allocs/op
Benchmark_findIndex4-4                        	340768832	         3.521 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/cilium/statedb/part	42.779s
PASS
ok  	github.com/cilium/statedb/reconciler	0.004s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go run ./reconciler/benchmark -quiet
1000000 objects reconciled in 1.88 seconds (batch size 1000)
Throughput 530524.19 objects per second
888MB total allocated, 6015205 in-use objects, 338MB bytes in use

Comment thread table.go

func (t *genTable[Obj]) indexPos(name string) int {
// An empty index name refers to the primary index, matching getIndexer.
if name == "" {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indexPos is executed for secondary indexes too. In case of a secondary index with an empty name we'll enter this if returning the primary index.
I think we should explicitly reject secondary indexes with empty names in NewTableAny.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should explicitly reject secondary indexes with empty names in NewTableAny.

I think rejecting secondary indexes with empty names is a worthwhile change. Fixed as suggested.

But AFAICS it still won't fix the issue here in all cases because the name passed to indexPos by callers may be the one extracted from Query.index or QueryRequest.Index and that could still be empty in some cases, depending on how the query was constructed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see.

Besides, looking again at the code I see that stateDB explicitly supports objects with an empty primary key (see TestDB_EmptyKeys) and with this change it is possible to have something like this:

emptyPrefixIndex := NetIPPrefixIndex[*testObject]{
	Name:   "empty-prefix",
	Unique: true,
	FromObject: func(*testObject) iter.Seq[netip.Prefix] {
		return func(func(netip.Prefix) bool) {}
	},
}

table, err := NewTable(
	db,
	"test",
	keyIndex,         // Primary index
	emptyPrefixIndex, // Secondary index
)

Now suppose that this object with an empty primary key is in the table:

testObject{
	Key:  "",
	Tags: part.NewSet("test-object"),
}

Then this query:

queryObject := &testObject{Key: "unrelated"}
q := emptyPrefixIndex.QueryFromObject(queryObject)

table.Get(rtxn, q)

returns the stored testObject, despite the object we used to generate the query had a Key value equal to "unrelated" (IOW: from a primary key POV it should not match).

Even if I don't think it is dangerous, it feels inconsistent. Getting this right seems hard, that's why I wonder if we should instead return a "no-match" representation instead of steering an empty index name toward the primary index.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that's a good point. I didn't consider this in my original change. I agree that getting this right is rather tricky, so I'll move the PR to draft for now and will discuss it with @joamaki after he's back.

tklauser added 3 commits June 30, 2026 10:01
Validate that the primary index is unique before starting to construct
any objects. Nothing in the check depends on these being present.

Signed-off-by: Tobias Klauser <tobias@cilium.io>
Suggested-by: Fabio Falzoi <fabio.falzoi@isovalent.com>
Signed-off-by: Tobias Klauser <tobias@cilium.io>
Querying a table with an empty index name currently panics with an
"index out of range" runtime error. This is because
(*genTable[Obj]).indexPos reads name[0] without a length check.

This is reachable from normal API usage: NetIPPrefixIndex.QueryFromObject
and LPMIndex.QueryFromObject return a zero-value Query (with an empty
index name) when the object yields no prefixes or keys, and passing such
a query to Get, List, Prefix or LowerBound panics.

Fix this by treating an empty index name as a reference to the primary
index in indexPos, following the existing behavior in getIndexer.

Signed-off-by: Tobias Klauser <tobias@cilium.io>
@tklauser
tklauser force-pushed the pr/tklauser/empty-index-name branch from fe1d212 to 7ded532 Compare June 30, 2026 08:01
@tklauser
tklauser requested a review from pippolo84 June 30, 2026 08:01
@tklauser
tklauser marked this pull request as draft June 30, 2026 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants