In our project, we got query like this:
TagItem.query(TagItem.template == template.key, TagItem.owner_type == owner_type).filter(TagItem.owner_uid._IN(owner_uids))
And the performance become worse when we owner_uids become bigger, after some profiling, we found that it is because we make datastore query sequentially for each owner_uid.
And from the ndb doc, we found that any of the IN operation would translate to OR
https://cloud.google.com/appengine/docs/standard/python/ndb/queries#neq_and_in
And I found this code in the current code:
Run the subqueries sequentially; there is no order to keep.
https://github.com/GoogleCloudPlatform/datastore-ndb-python/blob/master/ndb/query.py#L1957
It doesn't seem to be the most efficient way to filter with IN operation. Wondering if there is a way to change it to make subqueries concurrently.
In our project, we got query like this:
And the performance become worse when we
owner_uidsbecome bigger, after some profiling, we found that it is because we make datastore query sequentially for eachowner_uid.And from the ndb doc, we found that any of the
INoperation would translate toORhttps://cloud.google.com/appengine/docs/standard/python/ndb/queries#neq_and_in
And I found this code in the current code:
https://github.com/GoogleCloudPlatform/datastore-ndb-python/blob/master/ndb/query.py#L1957
It doesn't seem to be the most efficient way to filter with
INoperation. Wondering if there is a way to change it to make subqueries concurrently.