Skip to content

Commit bda06bf

Browse files
authored
Merge pull request #21 from Sankhya-AI/alpha
V-3.0.0
2 parents 459bef7 + 5128d8c commit bda06bf

27 files changed

Lines changed: 8590 additions & 135 deletions

CHANGELOG.md

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,91 @@ All notable changes to this project will be documented in this file.
44

55
The format is based on [Keep a Changelog](https://keepachangelog.com/), and this project adheres to [Semantic Versioning](https://semver.org/).
66

7+
## [3.0.0] - 2026-04-01 — Event-Sourced Cognition Substrate
8+
9+
Dhee v3 is a ground-up architectural overhaul that transforms the memory layer into an immutable-event, versioned cognition substrate. Raw memory is now immutable truth; derived cognition (beliefs, policies, insights, heuristics) is provisional, rebuildable, and auditable.
10+
11+
### New Architecture
12+
13+
- **Immutable raw events**`remember()` writes to `raw_memory_events`. Corrections create new events with `supersedes_event_id`, never mutate originals.
14+
- **Type-specific derived stores** — Beliefs, Policies, Anchors, Insights, Heuristics each have their own table with type-appropriate schemas, indexes, lifecycle rules, and invalidation behavior.
15+
- **Derived lineage** — Every derived object traces to its source raw events via `derived_lineage` table with contribution weights.
16+
- **Candidate promotion pipeline** — Consolidation no longer writes through `memory.add()`. Distillation produces candidates; promotion validates, dedupes, and transactionally promotes them to typed stores.
17+
18+
### Three-Tier Invalidation
19+
20+
- **Hard invalidation** — Source deleted: derived objects tombstoned, excluded from retrieval.
21+
- **Soft invalidation** — Source corrected: derived objects marked stale, repair job enqueued.
22+
- **Partial invalidation** — One of N sources changed with contribution weight < 30%: confidence penalty, not full re-derive. Weight >= 30% escalates to soft invalidation.
23+
24+
### 5-Stage RRF Fusion Retrieval
25+
26+
1. Per-index retrieval (raw, distilled, episodic — parallel, zero LLM)
27+
2. Min-max score normalization within each index
28+
3. Weighted Reciprocal Rank Fusion (k=60, distilled=1.0, episodic=0.7, raw=0.5)
29+
4. Post-fusion adjustments (recency boost, confidence normalization, staleness penalty, invalidation exclusion, contradiction penalty)
30+
5. Dedup + final ranking — zero LLM calls on the hot path
31+
32+
### Conflict Handling
33+
34+
- **Cognitive conflicts table** — Contradictions are explicit rows, not silent resolution.
35+
- **Auto-resolution** — When confidence gap >= 0.5 (one side >= 0.8, other <= 0.3), auto-resolve in favor of high-confidence side.
36+
37+
### Job Registry
38+
39+
- Replaces phantom `agi_loop.py` with real, observable maintenance jobs.
40+
- SQLite lease manager prevents concurrent execution of same job.
41+
- Jobs are named, idempotent, leasable, retryable, and independently testable.
42+
43+
### Anchor Resolution
44+
45+
- Per-field anchor candidates with individual confidence scores.
46+
- Re-anchoring: corrections re-resolve without touching raw events.
47+
48+
### Materialized Read Model
49+
50+
- `retrieval_view` materialized table for fast cold-path queries.
51+
- Delta overlay for hot-path freshness.
52+
53+
### Migration Bridge
54+
55+
- Dual-write: v2 path + v3 raw events in parallel (`DHEE_V3_WRITE=1`, default on).
56+
- Backfill: idempotent migration of v2 memories into v3 raw events via content-hash dedup.
57+
- Feature flag `DHEE_V3_READ` (default off) for gradual cutover.
58+
59+
### Observability
60+
61+
- `v3_health()` reports: raw event counts, derived invalidation counts per type, open conflicts, active leases, candidate stats, job health, retrieval view freshness, lineage coverage.
62+
63+
### Consolidation Safety
64+
65+
- Breaks the feedback loop: `_promote_to_passive()` uses `infer=False`, tags `source="consolidated"`.
66+
- `_should_promote()` rejects already-consolidated content.
67+
68+
### Other Changes
69+
70+
- `UniversalEngram.to_dict(sparse=True)` — omits None, empty strings, empty lists, empty dicts.
71+
- `agi_loop.py` cleaned: removed all phantom `engram_*` package imports.
72+
- 913 tests passing.
73+
74+
### New Files
75+
76+
- `dhee/core/storage.py` — Schema DDL for all v3 tables
77+
- `dhee/core/events.py` — RawEventStore
78+
- `dhee/core/derived_store.py` — BeliefStore, PolicyStore, AnchorStore, InsightStore, HeuristicStore, DerivedLineageStore, CognitionStore
79+
- `dhee/core/anchor_resolver.py` — AnchorCandidateStore, AnchorResolver
80+
- `dhee/core/invalidation.py` — Three-tier InvalidationEngine
81+
- `dhee/core/conflicts.py` — ConflictStore with auto-resolution
82+
- `dhee/core/read_model.py` — Materialized ReadModel
83+
- `dhee/core/fusion_v3.py` — 5-stage RRF fusion pipeline
84+
- `dhee/core/v3_health.py` — Observability metrics
85+
- `dhee/core/v3_migration.py` — Dual-write bridge + backfill
86+
- `dhee/core/lease_manager.py` — SQLite lease manager
87+
- `dhee/core/jobs.py` — JobRegistry + concrete jobs
88+
- `dhee/core/promotion.py` — PromotionEngine
89+
90+
---
91+
792
## [2.2.0b1] - 2026-03-31 — Architectural Cleanup
893

994
Beta release focused on internal discipline rather than new features.

dhee/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@
3131
# Default: CoreMemory (lightest, zero-config)
3232
Memory = CoreMemory
3333

34-
__version__ = "2.2.0b1"
34+
__version__ = "3.0.0"
3535
__all__ = [
3636
# Memory classes
3737
"CoreMemory",

dhee/core/agi_loop.py

Lines changed: 58 additions & 125 deletions
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,27 @@
1-
"""AGI Loopthe full cognitive cycle.
1+
"""Dhee v3Cognitive Maintenance Cycle.
22
3-
Orchestrates all memory subsystems in a single cycle:
4-
Perceive → Attend → Encode → Store → Consolidate → Retrieve →
5-
Evaluate → Learn → Plan → Act → Loop
3+
Replaces the phantom AGI loop with honest, real maintenance operations.
64
7-
This module provides the run_agi_cycle function called by the
8-
heartbeat behavior, plus system health reporting.
5+
v2.2 had 8 steps, 6 of which imported non-existent engram_* packages.
6+
v3 runs only what actually exists:
7+
1. Consolidation (active → passive, via safe consolidation engine)
8+
2. Decay (forgetting curves)
9+
10+
Planned but not yet implemented (will be added as real Job classes):
11+
- Anchor candidate resolution
12+
- Distillation promotion
13+
- Conflict scanning
14+
- Stale intention cleanup
15+
16+
The old API surface (run_agi_cycle, get_system_health) is preserved
17+
for backward compatibility with existing callers.
918
"""
1019

1120
from __future__ import annotations
1221

1322
import logging
1423
from datetime import datetime, timezone
15-
from typing import Any, Dict, List, Optional
24+
from typing import Any, Dict, Optional
1625

1726
logger = logging.getLogger(__name__)
1827

@@ -22,22 +31,20 @@ def run_agi_cycle(
2231
user_id: str = "default",
2332
context: Optional[str] = None,
2433
) -> Dict[str, Any]:
25-
"""Run one iteration of the AGI cognitive cycle.
26-
27-
Each step is optional — missing subsystems are gracefully skipped.
34+
"""Run one maintenance cycle. Only executes real subsystems.
2835
2936
Args:
30-
memory: Engram Memory instance
37+
memory: Dhee Memory instance
3138
user_id: User identifier for scoped operations
32-
context: Optional current context for reconsolidation
39+
context: Optional current context (reserved for future use)
3340
3441
Returns:
35-
Dict with status of each subsystem step
42+
Dict with status of each step
3643
"""
3744
now = datetime.now(timezone.utc).isoformat()
3845
results: Dict[str, Any] = {"timestamp": now, "user_id": user_id}
3946

40-
# 1. Consolidate — run distillation (episodic → semantic)
47+
# Step 1: Consolidation — run distillation (episodic → semantic)
4148
try:
4249
if hasattr(memory, "_kernel") and memory._kernel:
4350
consolidation = memory._kernel.sleep_cycle(user_id=user_id)
@@ -47,96 +54,19 @@ def run_agi_cycle(
4754
except Exception as e:
4855
results["consolidation"] = {"status": "error", "error": str(e)}
4956

50-
# 2. Decay — apply forgetting
57+
# Step 2: Decay — apply forgetting curves
5158
try:
5259
decay_result = memory.apply_decay(scope={"user_id": user_id})
5360
results["decay"] = {"status": "ok", "result": decay_result}
5461
except Exception as e:
5562
results["decay"] = {"status": "error", "error": str(e)}
5663

57-
# 3. Reconsolidation — auto-apply high-confidence proposals
58-
try:
59-
from engram_reconsolidation import Reconsolidation
60-
rc = Reconsolidation(memory, user_id=user_id)
61-
pending = rc.list_pending_proposals(limit=5)
62-
auto_applied = 0
63-
for p in pending:
64-
if p.get("confidence", 0) >= rc.config.min_confidence_for_auto_apply:
65-
rc.apply_update(p["id"])
66-
auto_applied += 1
67-
results["reconsolidation"] = {
68-
"status": "ok", "pending": len(pending), "auto_applied": auto_applied
69-
}
70-
except ImportError:
71-
results["reconsolidation"] = {"status": "skipped", "reason": "not installed"}
72-
except Exception as e:
73-
results["reconsolidation"] = {"status": "error", "error": str(e)}
74-
75-
# 4. Procedural — scan for extractable procedures
76-
try:
77-
from engram_procedural import Procedural
78-
proc = Procedural(memory, user_id=user_id)
79-
procedures = proc.list_procedures(status="active", limit=5)
80-
results["procedural"] = {
81-
"status": "ok", "active_procedures": len(procedures)
82-
}
83-
except ImportError:
84-
results["procedural"] = {"status": "skipped", "reason": "not installed"}
85-
except Exception as e:
86-
results["procedural"] = {"status": "error", "error": str(e)}
87-
88-
# 5. Metamemory — calibration check
89-
try:
90-
from engram_metamemory import Metamemory
91-
mm = Metamemory(memory, user_id=user_id)
92-
gaps = mm.list_knowledge_gaps(limit=5)
93-
results["metamemory"] = {
94-
"status": "ok", "open_gaps": len(gaps)
95-
}
96-
except ImportError:
97-
results["metamemory"] = {"status": "skipped", "reason": "not installed"}
98-
except Exception as e:
99-
results["metamemory"] = {"status": "error", "error": str(e)}
100-
101-
# 6. Prospective — check intention triggers
102-
try:
103-
from engram_prospective import Prospective
104-
pm = Prospective(memory, user_id=user_id)
105-
triggered = pm.check_triggers()
106-
results["prospective"] = {
107-
"status": "ok", "triggered": len(triggered)
108-
}
109-
except ImportError:
110-
results["prospective"] = {"status": "skipped", "reason": "not installed"}
111-
except Exception as e:
112-
results["prospective"] = {"status": "error", "error": str(e)}
113-
114-
# 7. Working memory — decay stale items
115-
try:
116-
from engram_working import WorkingMemory
117-
wm = WorkingMemory(memory, user_id=user_id)
118-
items = wm.list()
119-
results["working_memory"] = {
120-
"status": "ok", "active_items": len(items)
121-
}
122-
except ImportError:
123-
results["working_memory"] = {"status": "skipped", "reason": "not installed"}
124-
except Exception as e:
125-
results["working_memory"] = {"status": "error", "error": str(e)}
126-
127-
# 8. Failure — check for extractable anti-patterns
128-
try:
129-
from engram_failure import FailureLearning
130-
fl = FailureLearning(memory, user_id=user_id)
131-
stats = fl.get_failure_stats()
132-
results["failure_learning"] = {"status": "ok", **stats}
133-
except ImportError:
134-
results["failure_learning"] = {"status": "skipped", "reason": "not installed"}
135-
except Exception as e:
136-
results["failure_learning"] = {"status": "error", "error": str(e)}
137-
138-
# Compute overall status
139-
statuses = [v.get("status", "unknown") for v in results.values() if isinstance(v, dict)]
64+
# Compute summary
65+
statuses = [
66+
v.get("status", "unknown")
67+
for v in results.values()
68+
if isinstance(v, dict) and "status" in v
69+
]
14070
ok_count = statuses.count("ok")
14171
error_count = statuses.count("error")
14272
skipped_count = statuses.count("skipped")
@@ -152,53 +82,56 @@ def run_agi_cycle(
15282

15383

15484
def get_system_health(memory: Any, user_id: str = "default") -> Dict[str, Any]:
155-
"""Report health status across all cognitive subsystems.
85+
"""Report health status across real cognitive subsystems.
15686
157-
Returns a dict with each subsystem's availability and basic stats.
87+
Only reports subsystems that actually exist — no phantom package checks.
15888
"""
15989
now = datetime.now(timezone.utc).isoformat()
16090
systems: Dict[str, Dict] = {}
16191

162-
# Core systems (always available)
92+
# Core memory
16393
try:
16494
stats = memory.get_stats(user_id=user_id)
16595
systems["core_memory"] = {"available": True, "stats": stats}
16696
except Exception as e:
16797
systems["core_memory"] = {"available": False, "error": str(e)}
16898

169-
# Knowledge Graph
99+
# Knowledge graph
170100
systems["knowledge_graph"] = {
171-
"available": hasattr(memory, "knowledge_graph") and memory.knowledge_graph is not None,
101+
"available": (
102+
hasattr(memory, "knowledge_graph")
103+
and memory.knowledge_graph is not None
104+
),
172105
}
173106
if systems["knowledge_graph"]["available"]:
174107
try:
175108
systems["knowledge_graph"]["stats"] = memory.knowledge_graph.stats()
176109
except Exception:
177110
pass
178111

179-
# Power packages
180-
_optional_packages = [
181-
("engram_router", "router"),
182-
("engram_identity", "identity"),
183-
("engram_heartbeat", "heartbeat"),
184-
("engram_policy", "policy"),
185-
("engram_skills", "skills"),
186-
("engram_spawn", "spawn"),
187-
("engram_resilience", "resilience"),
188-
("engram_metamemory", "metamemory"),
189-
("engram_prospective", "prospective"),
190-
("engram_procedural", "procedural"),
191-
("engram_reconsolidation", "reconsolidation"),
192-
("engram_failure", "failure_learning"),
193-
("engram_working", "working_memory"),
194-
]
195-
196-
for pkg_name, system_name in _optional_packages:
112+
# Cognition kernel
113+
has_kernel = hasattr(memory, "_kernel") and memory._kernel is not None
114+
systems["cognition_kernel"] = {"available": has_kernel}
115+
if has_kernel:
197116
try:
198-
__import__(pkg_name)
199-
systems[system_name] = {"available": True}
200-
except ImportError:
201-
systems[system_name] = {"available": False}
117+
systems["cognition_kernel"]["stats"] = memory._kernel.cognition_health(
118+
user_id=user_id
119+
)
120+
except Exception:
121+
pass
122+
123+
# Active memory / consolidation
124+
systems["consolidation"] = {
125+
"available": (
126+
hasattr(memory, "_consolidation_engine")
127+
and memory._consolidation_engine is not None
128+
),
129+
}
130+
131+
# v3 stores (if wired)
132+
systems["v3_event_store"] = {
133+
"available": hasattr(memory, "_event_store") and memory._event_store is not None,
134+
}
202135

203136
available = sum(1 for s in systems.values() if s.get("available"))
204137
total = len(systems)

0 commit comments

Comments
 (0)