Conversation
Wraps the pylibmc client in _MemcachedWithFallback so that any connection error on get/set/delete/flush_all is caught and silently retried against the local diskcache instance. The server is still probed at startup and an error is logged, but the application no longer crashes when memcached is temporarily unreachable. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace zope.sqlalchemy register() / transaction.commit() / transaction.abort() with plain SQLAlchemy session calls (DBSession.commit() / DBSession.rollback()). Remove the zope.sqlalchemy package from requirements.txt. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rule.__json__: expose format, ref_key, network_id, template_id and types so callers receive a complete representation - Template.check_write_permission: new helper that gates writes on admin status (imported _is_admin from base) - get_network_rules: access template_id through rtype.templatetype rather than directly on rtype; fix get_hierarchy().reverse() (which returned None) by calling reverse() on the list separately; pass user_id to get_hierarchy() - update_rule: assign ref_id to rule_i.template_id (not network_id) for TEMPLATE ref_key - get_template: validate that a cached template has templatetypes before returning it; re-fetch from DB if the cached entry is empty Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Resource attributes (add_resource_attributes): - Replace sequential exists-check + individual ORM inserts with a single bulk_insert_ignore() call (new helper in db/__init__ using dialect- specific INSERT IGNORE / ON CONFLICT DO NOTHING). - Query back inserted IDs in one batch using tuple_() IN instead of iterating each object. - Invalidate the network RA cache immediately after the bulk insert. Scenario data (bulk_update_resourcedata / assign_value): - Pre-compute all dataset hashes in a single pass before entering the per-RS loop and batch-check for hash collisions with one query. - Skip the DB write entirely for unchanged datasets (hash match). - In the single-RS fast lane, mutate the dataset in place and consult the pre-built collision cache instead of issuing a per-dataset query. - Cache newly created datasets by hash within the batch to avoid duplicate inserts when the same value appears more than once. - Defer flush() to the end of the batch (flush=False per iteration). - Eager-load dataset metadata with joinedload to eliminate N+1 queries. Other: - Add thread-local permission cache in check_perm() to avoid repeated identical DB queries within the same request. - Remove unused user_id/source parameters from get_metadata_as_dict(). - Update test assertions to match the new return shape of add_resource_attributes (dict of id → [resource_id, attr_id]). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…olumns Adds nullable FK columns tNetwork.cloned_network_id and tProject.cloned_project_id so cloned networks and projects can track which original they were derived from. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…g out groups which were causing bad DB query performance
The cache previously stored positive permission checks forever with no invalidation, so revoking a user's role/permission wouldn't take effect for a long-lived worker thread until it happened to restart. Bound this by re-verifying against the DB after PERM_CACHE_TTL seconds (default 30, configurable via [permissions] cache_ttl).
Bundle the three batch-scoped caches (dataset_rs_map, new_dataset_cache, dataset_hash_cache) into a single _BulkAssignContext, document the connectivity invariant it relies on, and re-verify single-ownership at the point of the in-place dataset mutation rather than trusting an earlier check to still hold. Add a per-batch summary log line (unchanged / updated in place / created / collisions avoided). Also fixes a real bug surfaced by the new tests: when a hash collision was found but the calling user had no read permission on the existing dataset, the code fell through and tried to persist a second dataset with the same hash, which always raised IntegrityError against tDataset.hash's UNIQUE constraint (both in this fast lane and in the pre-existing data.add_dataset). Fixed via a new Dataset.set_unique_hash() that salts the hash computation only, without leaking the salt into the dataset's real (persisted) metadata.
b7f3e1a92c44 was branched off a1b2c3d4e5f6 before edf7bffb7b33 (this branch's merge of the divergent a1b2c3d4e5f6/580425ade2e4/877adf863b33 heads) existed, so merging both in left two heads again. Repoint it onto edf7bffb7b33 to restore a single alembic head.
The direct cache.get() call here bypassed get_cache()'s error handling, so a memcached timeout (e.g. from a pod restart) propagated as an unhandled 500 instead of degrading gracefully like the other cache accessors on this class.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
1: Fall back to diskcache when memchache is unavailable so it doesn't just blow up
2: Remove zope dependency
3: Make rules work correctly on templates
4: Speed up data insertion
5: speed up get_all_attributes_in_network