diff --git a/docs/rvm/instruction-set.md b/docs/rvm/instruction-set.md index bb252a470..1fe967838 100644 --- a/docs/rvm/instruction-set.md +++ b/docs/rvm/instruction-set.md @@ -68,6 +68,7 @@ comprehension contexts. Outside loops it aborts the current rule. | Mnemonic | Operands / Params | Behaviour | | :------------------------- | :---------------------------- | :---------------------------------------------------------- | | `ObjectSet` | `obj, key, value` | Mutates object in `obj` with key/value from registers. | +| `ObjectSetAdd` | `obj, key, value` | Adds a defined value to the set at `obj[key]`, creating it if absent. Undefined keys and values are omitted. | | `ObjectCreate` | `params_index=pN` | Builds object from literal template and register entries. | | `ArrayNew` | `dest` | Creates empty array. | | `ArrayPush` | `arr, value` | Appends to array. | @@ -84,6 +85,48 @@ comprehension contexts. Outside loops it aborts the current rule. Parameter structures: +- `ObjectSetAdd` uses three register operands without a parameter table. It + accumulates `rule[key] contains value` into an object of sets, preserving + existing members and omitting undefined keys and values. The target must be + an object, and an existing leaf must be a set even when the incoming value is + undefined. An undefined contribution does not create a missing leaf. Static + string prefixes remain part of the rule path; when a parent rule also has dynamic keys, its literal + final keys accumulate into the same object. Head-key loops are hoisted even + for bodyless rules and independently for each body. Undefined and scalar + collections contribute nothing. Failed keyed + rule bodies do not create empty leaf sets. The compiler + rejects conflicting `contains` and single-valued rule kinds. Multiple dynamic + key levels remain unsupported. Dynamic prefixes followed by dotted keys and + ancestor evaluation requiring intervening static prefixes are also explicitly + rejected; a static prefix is supported when it is the compiled rule path. + Static string leaf declarations are seeded as empty sets once before body + evaluation, without overwriting accumulated members. Key and value output + expressions share index bindings; named indices pair entries, while independent + wildcards retain Cartesian-product semantics. Output-loop and binding metadata + is selected by body query as well as module and expression, so differing body + bindings cannot overwrite each other's head plans. Both execution modes + accumulate all independent partial-rule bodies, while `else` bodies retain + first-success semantics. Static child references and entrypoints include + dynamic keyed-set definitions indexed once per parent and merged only when a + child is looked up. A known constant child key is bound and checked before + unrelated body statements, avoiding false recursion through nonmatching + dynamic definitions. The compiler also filters those definitions by the + requested key before set insertion, rather than calling the entire parent + and introducing cycles through sibling rules. + Both evaluators check ready input/reference keys before body evaluation, + between hoisted loops, and before output evaluation. Early filters defer + declared-but-undefined locals until binding and run after hoisted assignments + as well as ordinary statements. Scalar key bindings using `:=` or `=` (in + either operand order for `=`) can precede unrelated dependent statements. + Calls, comprehensions, and unresolved references conservatively defer early + filtering. Both evaluators always check the actual output key before + evaluating its value or accumulating a projected contribution, preventing + sibling writes even when early filtering is deferred. Interpreter recursion + tracking distinguishes parent and child projections of the same definition, + while repeated evaluation of the same projection remains a cycle. + Programs containing child projections enable runtime recursion checking in + both VM modes; genuine cycles propagate an error rather than becoming + undefined results. - `ObjectCreateParams` reuses arrays of literal key/value pairs and register pairs. Literal keys must be sorted to match template order. - `ArrayCreateParams` and `SetCreateParams` store register lists. The VM checks @@ -304,4 +347,3 @@ artifacts. Some contain `Vec` fields; the compiler is responsible for ensuring indices remain valid and stable across serialization boundaries. --- - diff --git a/src/compiled_policy.rs b/src/compiled_policy.rs index 468426e4f..e10d5c698 100644 --- a/src/compiled_policy.rs +++ b/src/compiled_policy.rs @@ -15,7 +15,8 @@ use crate::scheduler::*; use crate::utils::*; use crate::*; -use alloc::collections::BTreeMap; +use alloc::borrow::Cow; +use alloc::collections::{BTreeMap, BTreeSet}; use anyhow::Result; #[cfg(feature = "azure_policy")] @@ -216,6 +217,9 @@ pub(crate) struct CompiledPolicyData { pub(crate) parser_contexts: Rc>, pub(crate) schedule: Option>, pub(crate) rules: Map>>, + // Dynamic keyed-set definitions are indexed once per parent for child lookups. + pub(crate) keyed_set_parent_rules: Map, + pub(crate) keyed_set_child_parents: Map, pub(crate) default_rules: Map>, pub(crate) rule_path_components: Map>, pub(crate) imports: BTreeMap>, @@ -241,3 +245,57 @@ pub(crate) struct CompiledPolicyData { // Pre-computed loop hoisting information pub(crate) loop_hoisting_table: HoistedLoopsLookup, } + +#[derive(Debug, Clone, Default)] +pub(crate) struct KeyedSetParentRules { + pub(crate) definitions: Vec>, + rule_set: BTreeSet>, +} + +impl KeyedSetParentRules { + pub(crate) fn insert(&mut self, rule: Ref) { + if self.rule_set.insert(rule.clone()) { + self.definitions.push(rule); + } + } + + pub(crate) fn contains(&self, rule: &Ref) -> bool { + self.rule_set.contains(rule) + } +} + +impl CompiledPolicyData { + pub(crate) fn rules_for_lookup(&self, path: &str) -> Option]>> { + let static_rules = self.rules.get(path)?; + let Some(parent) = self.keyed_set_child_parents.get(path) else { + return Some(Cow::Borrowed(static_rules)); + }; + let Some(dynamic_rules) = self.keyed_set_parent_rules.get(parent) else { + return Some(Cow::Borrowed(static_rules)); + }; + + let mut rules = static_rules.clone(); + rules.extend(dynamic_rules.definitions.iter().cloned()); + Some(Cow::Owned(rules)) + } + + pub(crate) fn keyed_set_child_filter(&self, path: &str, rule: &Ref) -> Option { + let parent = self.keyed_set_child_parents.get(path)?; + if !self.keyed_set_parent_rules.get(parent)?.contains(rule) { + return None; + } + self.rule_path_components + .get(path)? + .last() + .map(|key| Value::from(key.clone())) + } + + pub(crate) fn is_projected_keyed_set_rule(&self, path: &str, rule: &Ref) -> bool { + let Some(parent) = self.keyed_set_child_parents.get(path) else { + return false; + }; + self.keyed_set_parent_rules + .get(parent) + .is_some_and(|rules| rules.contains(rule)) + } +} diff --git a/src/compiler/hoist.rs b/src/compiler/hoist.rs index 5f251b157..95fd9159a 100644 --- a/src/compiler/hoist.rs +++ b/src/compiler/hoist.rs @@ -16,7 +16,9 @@ use super::destructuring_planner::{ map_binding_error, BindingPlan, ScopingMode, VariableBindingContext, }; -use crate::ast::{Expr, ExprRef, Literal, LiteralStmt, Module, Query, Ref, Rule, RuleHead}; +use crate::ast::{ + AssignOp, Expr, ExprRef, Literal, LiteralStmt, Module, Query, Ref, Rule, RuleHead, +}; use crate::compiler::context::{ContextType, ScopeContext}; use crate::lookup::Lookup; use crate::lookup::LookupResult; @@ -24,9 +26,267 @@ use crate::scheduler::compute_module_globals; use crate::*; use anyhow::{anyhow, Result}; -use alloc::collections::BTreeSet; +use alloc::collections::{BTreeMap, BTreeSet}; use alloc::vec::Vec; +pub fn key_expr_is_ready( + expr: &ExprRef, + mut is_bound: impl FnMut(&ExprRef) -> Result, + mut is_var_bound: impl FnMut(&crate::lexer::Span) -> bool, + mut is_safe_call: impl FnMut(&ExprRef) -> Result, +) -> Result { + let mut ready = true; + crate::query::traversal::traverse(expr, &mut |node| { + if is_bound(node)? { + return Ok(false); + } + match node.as_ref() { + Expr::Var { span, .. } => { + ready &= is_var_bound(span); + Ok(false) + } + Expr::ArrayCompr { .. } + | Expr::SetCompr { .. } + | Expr::ObjectCompr { .. } + | Expr::BinExpr { .. } + | Expr::UnaryExpr { .. } + | Expr::ArithExpr { .. } + | Expr::AssignExpr { .. } => { + ready = false; + Ok(false) + } + Expr::Call { fcn, params, .. } => { + ready &= is_safe_literal_key_call(fcn, params)? && is_safe_call(node)?; + Ok(false) + } + _ => Ok(true), + } + })?; + Ok(ready) +} + +/// Whether `key_expr` reads a variable that one of `loops` binds as its index. +pub fn key_reads_loop_index_vars(key_expr: &ExprRef, loops: &[HoistedLoop]) -> Result { + fn any_var(expr: &ExprRef, mut f: impl FnMut(&str) -> Result) -> Result { + let mut found = false; + crate::query::traversal::traverse(expr, &mut |node| { + if let Expr::Var { span, .. } = node.as_ref() { + found = found || f(span.text())?; + } + Ok(!found) + })?; + Ok(found) + } + any_var(key_expr, |name| { + for key in loops.iter().filter_map(|l| l.key.as_ref()) { + if any_var(key, |index_var| Ok(index_var == name))? { + return Ok(true); + } + } + Ok(false) + }) +} + +pub fn is_safe_literal_key_call(fcn: &ExprRef, params: &[ExprRef]) -> Result { + let name = crate::utils::get_path_string(fcn, None)?; + let is_string = + |expr: &ExprRef| matches!(expr.as_ref(), Expr::String { .. } | Expr::RawString { .. }); + Ok(match (name.as_str(), params) { + ("lower", [value]) => is_string(value), + ("concat", [separator, values]) => { + is_string(separator) + && matches!( + values.as_ref(), + Expr::Array { items, .. } if items.iter().all(is_string) + ) + } + _ => false, + }) +} + +fn is_safe_key_binding_value( + expr: &ExprRef, + is_safe_call: &mut impl FnMut(&ExprRef) -> Result, +) -> Result { + match expr.as_ref() { + Expr::Call { fcn, params, .. } => { + Ok(is_safe_literal_key_call(fcn, params)? && is_safe_call(expr)?) + } + other => Ok(is_scalar_literal(other) || is_literal_input_ref(other)), + } +} + +/// `input`, or a path into it whose segments are all literals. Such a ref +/// reads the same value anywhere in a body, so its binding may move earlier. +fn is_literal_input_ref(expr: &Expr) -> bool { + match expr { + Expr::Var { span, .. } => span.text() == "input", + Expr::RefDot { refr, .. } => is_literal_input_ref(refr), + Expr::RefBrack { refr, index, .. } => { + is_scalar_literal(index) && is_literal_input_ref(refr) + } + _ => false, + } +} + +pub fn prioritize_key_binding( + statements: &mut Vec<&LiteralStmt>, + key_expr: Option<&ExprRef>, + key_filter: Option<&Value>, + mut is_safe_call: impl FnMut(&ExprRef) -> Result, +) -> Result<()> { + if key_filter.is_none() { + return Ok(()); + } + let Some(key_expr) = key_expr else { + return Ok(()); + }; + let mut key_names = BTreeSet::new(); + crate::query::traversal::traverse(key_expr, &mut |node| { + if let Expr::Var { span, .. } = node.as_ref() { + let name = span.text(); + if name != "_" && name != "input" { + key_names.insert(name.to_string()); + } + } + Ok(true) + })?; + for key_name in key_names { + let mut binding = None; + for (idx, stmt) in statements.iter().enumerate() { + if !stmt.with_mods.is_empty() { + continue; + } + let Literal::Expr { expr, .. } = &stmt.literal else { + continue; + }; + let Expr::AssignExpr { op, lhs, rhs, .. } = expr.as_ref() else { + continue; + }; + let value = if matches!(lhs.as_ref(), Expr::Var { span, .. } if span.text() == key_name) + { + rhs + } else if *op == AssignOp::Eq + && matches!(rhs.as_ref(), Expr::Var { span, .. } if span.text() == key_name) + { + lhs + } else { + continue; + }; + if is_safe_key_binding_value(value, &mut is_safe_call)? { + binding = Some((idx, is_literal_input_ref(value))); + } + } + if let Some((idx, reads_input)) = binding { + // A `some` declaration must stay ahead of the binding; any other + // earlier use of the key variable blocks reordering. Bindings that + // read `input` also stay behind earlier statements mentioning + // `input`, which may shadow it (Rego v0) or override it. + let mut insert_at = Some(0); + for (pos, stmt) in statements.iter().take(idx).enumerate() { + if matches!(&stmt.literal, Literal::SomeVars { vars, .. } if vars.iter().any(|var| var.text() == key_name)) + { + insert_at = Some(pos + 1); + } else if stmt_references_var(stmt, &key_name)? { + insert_at = None; + break; + } else if reads_input && stmt_references_var(stmt, "input")? { + insert_at = Some(pos + 1); + } + } + if let Some(insert_at) = insert_at.filter(|&at| at < idx) { + let stmt = statements.remove(idx); + statements.insert(insert_at, stmt); + } + } + } + Ok(()) +} + +/// Whether a body binds a local `input` (Rego v0 shadowing), in which case +/// `input` in a projected key is unready until that binding runs. +pub fn query_shadows_input(stmts: &[LiteralStmt]) -> Result { + let is_input = + |expr: &ExprRef| matches!(expr.as_ref(), Expr::Var { span, .. } if span.text() == "input"); + for stmt in stmts { + match &stmt.literal { + Literal::SomeVars { vars, .. } if vars.iter().any(|var| var.text() == "input") => { + return Ok(true) + } + Literal::SomeIn { key, value, .. } + if key.as_ref().is_some_and(&is_input) || is_input(value) => + { + return Ok(true) + } + Literal::Expr { expr, .. } => { + if let Expr::AssignExpr { + op: AssignOp::ColEq, + lhs, + .. + } = expr.as_ref() + { + let mut found = false; + crate::query::traversal::traverse(lhs, &mut |node| { + found |= is_input(node); + Ok(!found) + })?; + if found { + return Ok(true); + } + } + } + _ => {} + } + } + Ok(false) +} + +fn stmt_references_var(stmt: &LiteralStmt, name: &str) -> Result { + let mut found = false; + let mut check_expr = |expr: &ExprRef| -> Result { + if matches!(expr.as_ref(), Expr::Var { span, .. } if span.text() == name) { + found = true; + Ok(false) + } else { + Ok(true) + } + }; + match &stmt.literal { + Literal::Expr { expr, .. } | Literal::NotExpr { expr, .. } => { + crate::query::traversal::traverse(expr, &mut check_expr)?; + } + Literal::SomeIn { + key, + value, + collection, + .. + } => { + if let Some(key) = key { + crate::query::traversal::traverse(key, &mut check_expr)?; + } + crate::query::traversal::traverse(value, &mut check_expr)?; + crate::query::traversal::traverse(collection, &mut check_expr)?; + } + Literal::SomeVars { vars, .. } => { + found = vars.iter().any(|var| var.text() == name); + } + Literal::Every { .. } => return Ok(true), + } + Ok(found) +} + +fn is_scalar_literal(expr: &Expr) -> bool { + match expr { + Expr::String { .. } + | Expr::RawString { .. } + | Expr::Number { .. } + | Expr::Bool { .. } + | Expr::Null { .. } => true, + Expr::UnaryExpr { expr, .. } => matches!(expr.as_ref(), Expr::Number { .. }), + _ => false, + } +} + /// Implementation of VariableBindingContext for ScopeContext impl VariableBindingContext for ScopeContext { fn is_var_unbound(&self, var_name: &str, scoping: ScopingMode) -> bool { @@ -120,6 +380,10 @@ pub struct HoistedLoopsLookup { /// Maps (module_index, query_index) -> ScopeContext /// Stores compilation contexts for queries (rules, comprehensions, every) query_contexts: Lookup, + + // Rule heads reuse expression IDs across bodies with different bindings. + output_loops: BTreeMap<(u32, u32, u32), Vec>, + output_binding_plans: BTreeMap<(u32, u32, u32), BindingPlan>, } impl HoistedLoopsLookup { @@ -195,6 +459,37 @@ impl HoistedLoopsLookup { self.expr_loops.get_checked(module_idx, expr_idx) } + pub fn get_output_loops( + &self, + module_idx: u32, + query_idx: Option, + expr_idx: u32, + ) -> LookupResult>> { + if let Some(query_idx) = query_idx { + if let Some(loops) = self.output_loops.get(&(module_idx, query_idx, expr_idx)) { + return Ok(Some(loops)); + } + } + self.get_expr_loops(module_idx, expr_idx) + } + + pub fn get_output_binding_plan( + &self, + module_idx: u32, + query_idx: Option, + expr_idx: u32, + ) -> LookupResult> { + if let Some(query_idx) = query_idx { + if let Some(plan) = self + .output_binding_plans + .get(&(module_idx, query_idx, expr_idx)) + { + return Ok(Some(plan)); + } + } + self.get_expr_binding_plan(module_idx, expr_idx) + } + /// Store the compilation context for a query pub fn set_query_context( &mut self, @@ -246,6 +541,7 @@ pub struct LoopHoister { lookup: HoistedLoopsLookup, schedule: Option>, module_globals: Lookup>>, + output_query: Option, } impl LoopHoister { @@ -255,6 +551,7 @@ impl LoopHoister { lookup: HoistedLoopsLookup::new(), schedule: None, module_globals: Lookup::new(), + output_query: None, } } @@ -264,6 +561,7 @@ impl LoopHoister { lookup: HoistedLoopsLookup::new(), schedule: Some(schedule), module_globals: Lookup::new(), + output_query: None, } } @@ -432,9 +730,20 @@ impl LoopHoister { }; (key_expr, output_expr) } - RuleHead::Set { key, .. } => { - // For set rules, no separate key_expr, output_expr is the key - (None, key.clone()) + RuleHead::Set { refr, key, .. } => { + let key_expr = match refr.as_ref() { + Expr::RefBrack { index, .. } + if key.is_some() + && !matches!( + index.as_ref(), + Expr::String { .. } | Expr::RawString { .. } + ) => + { + Some(index.clone()) + } + _ => None, + }; + (key_expr, key.clone()) } RuleHead::Func { assign, .. } => { // Function rules return the assignment value (if any) @@ -454,7 +763,7 @@ impl LoopHoister { context.current_scope_bound_vars.clone(); // Store the context for this query - let populated_body_context = + let mut populated_body_context = self.populate_query(module_idx, &body.query, &body_context)?; self.lookup .ensure_query_capacity(module_idx, body.query.qidx); @@ -466,20 +775,26 @@ impl LoopHoister { // Process the key expression if present if let Some(ref key) = key_expr { - self.populate_output_expr(module_idx, key, &populated_body_context)?; + self.populate_body_output_expr( + module_idx, + body.query.qidx, + key, + &mut populated_body_context, + )?; } // Process the head value expression (if present) - if let Some(ref head_value) = value_expr { - self.populate_output_expr(module_idx, head_value, &populated_body_context)?; - } - - // Process the value expression if present in the assign - if let Some(ref assign) = body.assign { - self.populate_output_expr( + if let Some(output) = body + .assign + .as_ref() + .map(|assign| &assign.value) + .or(value_expr.as_ref()) + { + self.populate_body_output_expr( module_idx, - &assign.value, - &populated_body_context, + body.query.qidx, + output, + &mut populated_body_context, )?; } } @@ -495,18 +810,18 @@ impl LoopHoister { context.current_scope_bound_vars.clone(); if let Some(ref key) = key_expr { - self.populate_output_expr(module_idx, key, &body_context)?; + self.populate_output_expr(module_idx, key, &mut body_context)?; } if let Some(ref value) = value_expr { - self.populate_output_expr(module_idx, value, &body_context)?; + self.populate_output_expr(module_idx, value, &mut body_context)?; } } } Rule::Default { value, .. } => { // For default rules, just process the value expression - let context = self.create_scope_context(module_idx)?; - self.populate_output_expr(module_idx, value, &context)?; + let mut context = self.create_scope_context(module_idx)?; + self.populate_output_expr(module_idx, value, &mut context)?; } } @@ -519,6 +834,18 @@ impl LoopHoister { module_idx: u32, query: &Query, parent_context: &ScopeContext, + ) -> Result { + let output_query = self.output_query.take(); + let result = self.populate_query_impl(module_idx, query, parent_context); + self.output_query = output_query; + result + } + + fn populate_query_impl( + &mut self, + module_idx: u32, + query: &Query, + parent_context: &ScopeContext, ) -> Result { self.lookup.ensure_query_capacity(module_idx, query.qidx); let mut context = parent_context.clone(); @@ -672,12 +999,17 @@ impl LoopHoister { None, Some(term.clone()), ); - let populated_context = + let mut populated_context = self.populate_query(module_idx, query.as_ref(), &compr_context)?; self.lookup.ensure_query_capacity(module_idx, query.qidx); self.lookup .set_query_context(module_idx, query.qidx, populated_context.clone())?; - self.populate_output_expr_with_context(module_idx, term, &populated_context)?; + self.populate_body_output_expr( + module_idx, + query.qidx, + term, + &mut populated_context, + )?; } E::ObjectCompr { key, value, query, .. @@ -687,13 +1019,23 @@ impl LoopHoister { Some(key.clone()), Some(value.clone()), ); - let populated_context = + let mut populated_context = self.populate_query(module_idx, query.as_ref(), &compr_context)?; self.lookup.ensure_query_capacity(module_idx, query.qidx); self.lookup .set_query_context(module_idx, query.qidx, populated_context.clone())?; - self.populate_output_expr_with_context(module_idx, key, &populated_context)?; - self.populate_output_expr_with_context(module_idx, value, &populated_context)?; + self.populate_body_output_expr( + module_idx, + query.qidx, + key, + &mut populated_context, + )?; + self.populate_body_output_expr( + module_idx, + query.qidx, + value, + &mut populated_context, + )?; } E::Call { fcn, params, .. } => { self.analyze_expr(module_idx, fcn, context, loops)?; @@ -734,11 +1076,7 @@ impl LoopHoister { // Immediately bind variables from the plan to context Self::bind_vars_from_plan_to_context(&binding_plan, context); - self.lookup.set_expr_binding_plan( - module_idx, - expr_idx, - binding_plan, - )?; + self.set_output_binding_plan(module_idx, expr_idx, binding_plan)?; } Err(err) => return Err(map_binding_error(err)), } @@ -762,11 +1100,7 @@ impl LoopHoister { let expr_idx = index.as_ref().eidx(); self.lookup.ensure_expr_capacity(module_idx, expr_idx); Self::bind_vars_from_plan_to_context(&binding_plan, context); - self.lookup.set_expr_binding_plan( - module_idx, - expr_idx, - binding_plan, - )?; + self.set_output_binding_plan(module_idx, expr_idx, binding_plan)?; } Err(err) => return Err(map_binding_error(err)), } @@ -798,8 +1132,7 @@ impl LoopHoister { let expr_idx = expr.as_ref().eidx(); self.lookup.ensure_expr_capacity(module_idx, expr_idx); Self::bind_vars_from_plan_to_context(&binding_plan, context); - self.lookup - .set_expr_binding_plan(module_idx, expr_idx, binding_plan)?; + self.set_output_binding_plan(module_idx, expr_idx, binding_plan)?; self.analyze_expr(module_idx, lhs, context, loops)?; self.analyze_expr(module_idx, rhs, context, loops)?; @@ -855,28 +1188,54 @@ impl LoopHoister { } /// Populate loop information for output expressions (rule values, comprehension terms) - fn populate_output_expr( + fn populate_body_output_expr( &mut self, module_idx: u32, + query_idx: u32, expr: &ExprRef, - context: &ScopeContext, + context: &mut ScopeContext, + ) -> Result<()> { + let previous = self.output_query.replace(query_idx); + let result = self.populate_output_expr(module_idx, expr, context); + self.output_query = previous; + result + } + + fn set_output_binding_plan( + &mut self, + module_idx: u32, + expr_idx: u32, + plan: BindingPlan, ) -> Result<()> { - self.populate_output_expr_with_context(module_idx, expr, context) + if let Some(query_idx) = self.output_query { + self.lookup + .output_binding_plans + .insert((module_idx, query_idx, expr_idx), plan); + Ok(()) + } else { + self.lookup + .set_expr_binding_plan(module_idx, expr_idx, plan) + } } - fn populate_output_expr_with_context( + fn populate_output_expr( &mut self, module_idx: u32, expr: &ExprRef, - context: &ScopeContext, + context: &mut ScopeContext, ) -> Result<()> { let mut loops = Vec::new(); - let mut expr_context = context.clone(); - self.analyze_expr(module_idx, expr, &mut expr_context, &mut loops)?; + self.analyze_expr(module_idx, expr, context, &mut loops)?; let expr_idx = expr.as_ref().eidx(); self.lookup.ensure_expr_capacity(module_idx, expr_idx); - self.lookup.set_expr_loops(module_idx, expr_idx, loops)?; + if let Some(query_idx) = self.output_query { + self.lookup + .output_loops + .insert((module_idx, query_idx, expr_idx), loops); + } else { + self.lookup.set_expr_loops(module_idx, expr_idx, loops)?; + } Ok(()) } diff --git a/src/interpreter.rs b/src/interpreter.rs index eea1a4d6e..7a74263ef 100644 --- a/src/interpreter.rs +++ b/src/interpreter.rs @@ -38,7 +38,10 @@ use anyhow::{anyhow, bail, Result}; use core::ops::Bound::*; type Scope = BTreeMap; -type ExprLookup = Lookup; +// Each slot is tagged with the rule-invocation frame that wrote it; other +// frames treat it as unbound. +type LoopVarSlot = (u64, Option); +type ExprLookup = Lookup; #[cfg(feature = "azure_policy")] pub mod error; @@ -114,12 +117,17 @@ pub struct Interpreter { scopes: Vec, // TODO: handle recursive calls where same expr could have different values. loop_var_values: ExprLookup, + loop_var_frame: u64, + loop_var_frame_counter: u64, + // Caller slots overwritten by nested frames, restored when they return. + loop_var_undo: Vec<(u32, u32, Option)>, contexts: Vec, processed: BTreeSet>, processed_paths: Value, rule_values: RuleValues, - active_rules: Vec>, + // A parent invocation and its filtered child projections have distinct identities. + active_rules: Vec<(Ref, Option)>, builtins_cache: BTreeMap<(&'static str, Vec), Value>, no_rules_lookup: bool, execution_timer: ExecutionTimer, @@ -158,6 +166,9 @@ impl Clone for Interpreter { processed: BTreeSet::default(), processed_paths: Value::new_object(), loop_var_values: ExprLookup::new(), + loop_var_frame: 0, + loop_var_frame_counter: 0, + loop_var_undo: Vec::new(), scopes: Vec::default(), rule_values: BTreeMap::default(), @@ -190,6 +201,13 @@ struct Context { is_old_style_set: bool, output_constness_determined: bool, early_return: bool, + output_query: Option, + key_filter: Option, + // Head-key loops could not be checked before the body ran because a + // collection depends on a body binding; recheck as bindings appear. + head_key_loops_pending: bool, + // The current body binds a local `input` (Rego v0 shadowing). + input_shadowed: bool, } impl Default for Context { @@ -207,6 +225,10 @@ impl Default for Context { is_old_style_set: false, output_constness_determined: false, early_return: false, + output_query: None, + head_key_loops_pending: false, + input_shadowed: false, + key_filter: None, } } } @@ -236,6 +258,9 @@ impl Interpreter { scopes: Vec::default(), contexts: Vec::default(), loop_var_values: Lookup::default(), + loop_var_frame: 0, + loop_var_frame_counter: 0, + loop_var_undo: Vec::new(), processed: BTreeSet::default(), processed_paths: Value::new_object(), @@ -275,6 +300,9 @@ impl Interpreter { scopes: Vec::default(), contexts: Vec::default(), loop_var_values: Lookup::default(), + loop_var_frame: 0, + loop_var_frame_counter: 0, + loop_var_undo: Vec::new(), processed: BTreeSet::default(), processed_paths: Value::new_object(), @@ -369,6 +397,20 @@ impl Interpreter { } } + fn expression_binding_plan( + &self, + expr_idx: u32, + ) -> crate::lookup::LookupResult> { + self.loop_hoisting_table(self.current_module_index) + .get_output_binding_plan( + self.current_module_index, + self.contexts + .last() + .and_then(|context| context.output_query), + expr_idx, + ) + } + pub const fn get_data_mut(&mut self) -> &mut Value { &mut self.data } @@ -454,23 +496,42 @@ impl Interpreter { fn set_loop_var_value(&mut self, expr: &ExprRef, value: Value) -> Result<()> { let module_idx = self.current_module_index; let expr_idx = expr.eidx(); - self.loop_var_values - .set_checked(module_idx, expr_idx, value) - .map_err(|err| anyhow!("internal error: loop var indices out of bounds: {err}")) + let frame = self.loop_var_frame; + let old = self + .loop_var_values + .replace_checked(module_idx, expr_idx, Some((frame, Some(value)))) + .map_err(|err| anyhow!("internal error: loop var indices out of bounds: {err}"))?; + if frame != 0 && !matches!(old, Some((owner, _)) if owner == frame) { + self.loop_var_undo.push((module_idx, expr_idx, old)); + } + Ok(()) } fn get_loop_var_value(&self, expr: &ExprRef) -> Result> { let module_idx = self.current_module_index; let expr_idx = expr.eidx(); - self.loop_var_values + let slot = self + .loop_var_values .get_checked(module_idx, expr_idx) - .map_err(|err| anyhow!("internal error: loop var indices out of bounds: {err}")) + .map_err(|err| anyhow!("internal error: loop var indices out of bounds: {err}"))?; + Ok(match slot { + Some((owner, value)) if *owner == self.loop_var_frame => value.as_ref(), + _ => None, + }) } fn remove_loop_var_value(&mut self, expr: &ExprRef) { let module_idx = self.current_module_index; let expr_idx = expr.eidx(); - self.loop_var_values.clear(module_idx, expr_idx); + let frame = self.loop_var_frame; + if matches!( + self.loop_var_values.get_checked(module_idx, expr_idx), + Ok(Some((owner, _))) if *owner == frame + ) { + let _ = self + .loop_var_values + .set_checked(module_idx, expr_idx, (frame, None)); + } } #[inline] @@ -500,11 +561,9 @@ impl Interpreter { if let Expr::Call { params, .. } = Self::loop_assignment_expr(loop_info).as_ref() { if let Some(last_param) = params.last() { - let module_idx = self.current_module_index; let expr_idx = last_param.as_ref().eidx(); let binding_plan = self - .loop_hoisting_table(module_idx) - .get_expr_binding_plan(module_idx, expr_idx) + .expression_binding_plan(expr_idx) .map_err(|err| anyhow!("loop hoisting table out of bounds: {err}"))?; return match binding_plan.cloned() { @@ -1018,15 +1077,14 @@ impl Interpreter { ) -> Result { self.check_execution_time()?; let scope_saved = self.current_scope()?.clone(); + let pending_saved = self.head_key_loops_pending(); let mut count: usize = 0; // Fetch the binding plan for this some..in expression - let module_idx = self.current_module_index; let expr_idx = collection.as_ref().eidx(); let binding_plan = self - .loop_hoisting_table(module_idx) - .get_expr_binding_plan(module_idx, expr_idx) + .expression_binding_plan(expr_idx) .map_err(|err| anyhow!("loop hoisting table out of bounds: {err}"))?; let Some(BindingPlan::SomeIn { @@ -1042,6 +1100,7 @@ impl Interpreter { Value::Array(a) => { for (idx, value) in a.iter().enumerate() { *self.current_scope_mut()? = scope_saved.clone(); + self.set_head_key_loops_pending(pending_saved); let mut success = if let Some(key_plan) = &key_plan { self.execute_destructuring_plan(key_plan, &Value::from(idx))? @@ -1060,6 +1119,11 @@ impl Interpreter { continue; } + if self.reject_nonmatching_child_key()? { + *self.current_scope_mut()? = scope_saved.clone(); + continue; + } + let mut should_break = false; if self.eval_stmts(stmts)? { count = count.saturating_add(1); @@ -1079,6 +1143,7 @@ impl Interpreter { Value::Set(s) => { for value in s.iter() { *self.current_scope_mut()? = scope_saved.clone(); + self.set_head_key_loops_pending(pending_saved); let mut success = if let Some(key_plan) = &key_plan { self.execute_destructuring_plan(key_plan, value)? == Value::from(true) @@ -1096,6 +1161,11 @@ impl Interpreter { continue; } + if self.reject_nonmatching_child_key()? { + *self.current_scope_mut()? = scope_saved.clone(); + continue; + } + let mut should_break = false; if self.eval_stmts(stmts)? { count = count.saturating_add(1); @@ -1116,6 +1186,7 @@ impl Interpreter { Value::Object(o) => { for (key, value) in o.iter() { *self.current_scope_mut()? = scope_saved.clone(); + self.set_head_key_loops_pending(pending_saved); let mut success = if let Some(key_plan) = &key_plan { self.execute_destructuring_plan(key_plan, key)? == Value::from(true) @@ -1133,6 +1204,11 @@ impl Interpreter { continue; } + if self.reject_nonmatching_child_key()? { + *self.current_scope_mut()? = scope_saved.clone(); + continue; + } + let mut should_break = false; if self.eval_stmts(stmts)? { count = count.saturating_add(1); @@ -1157,6 +1233,7 @@ impl Interpreter { } } + self.set_head_key_loops_pending(pending_saved); Ok(count > 0) } @@ -1593,17 +1670,47 @@ impl Interpreter { } } + fn head_key_loops_pending(&self) -> bool { + self.contexts + .last() + .is_some_and(|context| context.head_key_loops_pending) + } + + fn set_head_key_loops_pending(&mut self, pending: bool) { + if let Some(context) = self.contexts.last_mut() { + context.head_key_loops_pending = pending; + } + } + + // A resolved head-key recheck only holds for the current bindings, so + // each loop iteration restores the pending state on exit. fn eval_stmts_in_loop( &mut self, stmts: &[&LiteralStmt], loops: &[HoistedLoop], + ) -> Result { + let pending = self.head_key_loops_pending(); + let result = self.eval_stmts_in_loop_impl(stmts, loops); + self.set_head_key_loops_pending(pending); + result + } + + fn eval_stmts_in_loop_impl( + &mut self, + stmts: &[&LiteralStmt], + loops: &[HoistedLoop], ) -> Result { self.memory_check()?; self.check_execution_time()?; if loops.is_empty() { + if self.reject_nonmatching_child_key()? { + return Ok(false); + } if let Some((first_stmt, tail_stmts)) = stmts.split_first() { // Evaluate the current statement whose loop expressions have been hoisted. - if self.eval_stmt(first_stmt, tail_stmts)? { + if self.eval_stmt(first_stmt, tail_stmts)? + && !self.reject_nonmatching_child_key()? + { if !matches!(&first_stmt.literal, Literal::SomeIn { .. }) { self.eval_stmts(tail_stmts) } else { @@ -1717,28 +1824,7 @@ impl Interpreter { } } - let index_plan = if let Some(index) = index_expr { - let module_idx = self.current_module_index; - let expr_idx = index.as_ref().eidx(); - let plan = self - .loop_hoisting_table(module_idx) - .get_expr_binding_plan(module_idx, expr_idx) - .map_err(|err| anyhow!("loop hoisting table out of bounds: {err}"))?; - - match plan.cloned() { - Some(BindingPlan::LoopIndex { - destructuring_plan, .. - }) => destructuring_plan, - Some(other_plan) => { - bail!("internal error: expected LoopIndex for loop index expression, got {:?}", other_plan); - } - None => { - bail!("internal error: no binding plan found for loop index expression"); - } - } - } else { - bail!("internal error: no binding plan found for loop index expression"); - }; + let index_plan = self.get_loop_index_plan(loop_info)?; // Create a new scope. self.scopes.push(Scope::default()); @@ -1957,8 +2043,69 @@ impl Interpreter { Ok(is_const && Self::is_simple_literal(output_expr)?) } - fn eval_output_expr_in_loop(&mut self, loops: &[HoistedLoop]) -> Result { + fn get_loop_index_plan(&self, loop_info: &HoistedLoop) -> Result { + let index = Self::loop_index_expr(loop_info) + .ok_or_else(|| anyhow!("internal error: no loop index expression"))?; + let plan = self + .expression_binding_plan(index.as_ref().eidx()) + .map_err(|err| anyhow!("loop hoisting table out of bounds: {err}"))?; + match plan { + Some(BindingPlan::LoopIndex { + destructuring_plan, .. + }) => Ok(destructuring_plan.clone()), + Some(other_plan) => { + bail!( + "internal error: expected LoopIndex for loop index expression, got {:?}", + other_plan + ) + } + None => bail!("internal error: no binding plan found for loop index expression"), + } + } + + fn eval_output_loop_iteration( + &mut self, + loop_info: &HoistedLoop, + index_plan: &DestructuringPlan, + index: &Value, + value: &Value, + loops: &[HoistedLoop], + value_loops: usize, + ) -> Result { + self.memory_check()?; + self.set_loop_var_value(Self::loop_assignment_expr(loop_info), value.clone())?; + let result = if self.execute_destructuring_plan(index_plan, index)? == Value::Bool(true) { + self.eval_output_expr_in_loop(loops, value_loops) + } else { + Ok(false) + }; + Self::clear_scope(self.current_scope_mut()?); + result + } + + /// `loops` holds the head key loops followed by `value_loops` head value + /// loops. + fn eval_output_expr_in_loop( + &mut self, + loops: &[HoistedLoop], + value_loops: usize, + ) -> Result { self.check_execution_time()?; + if self.reject_nonmatching_child_key()? { + return Ok(false); + } + // Once the key is fully bound, reject a nonmatching projected key + // before value loops evaluate collections that may read the child. + if value_loops > 0 && loops.len() == value_loops { + let ctx = self.get_current_context()?; + if let (Some(filter), Some(key_expr)) = (ctx.key_filter.clone(), ctx.key_expr.clone()) { + if !crate::compiler::hoist::key_reads_loop_index_vars(&key_expr, loops)? + && self.eval_expr(&key_expr)? != filter + { + return Ok(false); + } + } + } if loops.is_empty() { let (key_expr, output_expr) = self.get_exprs_from_context()?; @@ -1982,7 +2129,16 @@ impl Interpreter { let mut comps = self.eval_rule_ref(&rule_ref)?; if let Some(ke) = &key_expr { is_const_rule = is_const_rule && Self::is_simple_literal(ke)?; - comps.push(self.eval_expr(ke)?); + let key = self.eval_expr(ke)?; + if self + .get_current_context()? + .key_filter + .as_ref() + .is_some_and(|filter| filter != &key) + { + return Ok(false); + } + comps.push(key); } let output = if let Some(oe) = &output_expr { // Rule is constant only if its ref, key and output are constant. @@ -2012,7 +2168,9 @@ impl Interpreter { } if output == Value::Undefined || !comps_defined { - ctx_mut.rule_value = Value::Undefined; + if !is_set || key_expr.is_none() { + ctx_mut.rule_value = Value::Undefined; + } return Ok(false); } @@ -2133,36 +2291,60 @@ impl Interpreter { let (loop_info, loop_tail) = loops .split_first() .ok_or_else(|| anyhow!("internal error: expected loop info"))?; - let mut result = false; let loop_target_expr = Self::loop_assignment_expr(loop_info); - match self.eval_expr(Self::loop_collection_expr(loop_info))? { - Value::Array(items) => { - for v in items.iter() { - self.set_loop_var_value(loop_target_expr, v.clone())?; - result = self.eval_output_expr_in_loop(loop_tail)? || result; + let collection = self.eval_expr(Self::loop_collection_expr(loop_info))?; + let index_plan = if Self::loop_index_expr(loop_info).is_some() { + self.get_loop_index_plan(loop_info)? + } else { + DestructuringPlan::Ignore + }; + self.scopes.push(Scope::default()); + let result = (|| { + let mut result = false; + match collection { + Value::Array(items) => { + for (idx, value) in items.iter().enumerate() { + result = self.eval_output_loop_iteration( + loop_info, + &index_plan, + &Value::from(idx), + value, + loop_tail, + value_loops, + )? || result; + } } - } - Value::Set(items) => { - for v in items.iter() { - self.set_loop_var_value(loop_target_expr, v.clone())?; - result = self.eval_output_expr_in_loop(loop_tail)? || result; + Value::Set(items) => { + for value in items.iter() { + result = self.eval_output_loop_iteration( + loop_info, + &index_plan, + value, + value, + loop_tail, + value_loops, + )? || result; + } } - } - Value::Object(obj) => { - for (_, v) in obj.iter() { - self.set_loop_var_value(loop_target_expr, v.clone())?; - result = self.eval_output_expr_in_loop(loop_tail)? || result; + Value::Object(obj) => { + for (key, value) in obj.iter() { + result = self.eval_output_loop_iteration( + loop_info, + &index_plan, + key, + value, + loop_tail, + value_loops, + )? || result; + } } + _ => {} } - _ => { - let span = Self::loop_span(loop_info); - return Err(span - .source - .error(span.line, span.col, "item cannot be indexed")); - } - } + Ok(result) + })(); + self.scopes.pop(); self.remove_loop_var_value(loop_target_expr); - Ok(result) + result } fn get_current_context(&self) -> Result<&Context> { @@ -2186,6 +2368,9 @@ impl Interpreter { fn eval_output_expr(&mut self) -> Result { self.check_execution_time()?; + if self.reject_nonmatching_child_key()? { + return Ok(false); + } // Evaluate output expression after all the statements have been executed. let (key_expr, output_expr) = self.get_exprs_from_context()?; @@ -2195,7 +2380,13 @@ impl Interpreter { if let Some(ke) = &key_expr { match self .loop_hoisting_table(self.current_module_index) - .get_expr_loops(self.current_module_index, ke.as_ref().eidx()) + .get_output_loops( + self.current_module_index, + self.contexts + .last() + .and_then(|context| context.output_query), + ke.as_ref().eidx(), + ) .map_err(|err| anyhow!("loop hoisting table out of bounds: {err}"))? { Some(hoisted_loops) => { @@ -2207,11 +2398,18 @@ impl Interpreter { } } + let key_loops = loops.len(); // Get pre-computed loops for output expression if let Some(oe) = &output_expr { match self .loop_hoisting_table(self.current_module_index) - .get_expr_loops(self.current_module_index, oe.as_ref().eidx()) + .get_output_loops( + self.current_module_index, + self.contexts + .last() + .and_then(|context| context.output_query), + oe.as_ref().eidx(), + ) .map_err(|err| anyhow!("loop hoisting table out of bounds: {err}"))? { Some(hoisted_loops) => { @@ -2223,7 +2421,8 @@ impl Interpreter { } } - let r = self.eval_output_expr_in_loop(&loops[..])?; + let value_loops = loops.len().saturating_sub(key_loops); + let r = self.eval_output_expr_in_loop(&loops[..], value_loops)?; let ctx = self.get_current_context()?; ctx.output_expr @@ -2232,6 +2431,9 @@ impl Interpreter { } fn eval_stmts(&mut self, stmts: &[&LiteralStmt]) -> Result { + if self.reject_nonmatching_child_key()? { + return Ok(false); + } let mut eval_success = true; for (idx, stmt) in stmts.iter().enumerate() { @@ -2271,6 +2473,9 @@ impl Interpreter { .unwrap_or(&[]); eval_success = self.eval_stmt(stmt, tail)?; + if eval_success && self.reject_nonmatching_child_key()? { + eval_success = false; + } if matches!(&stmt.literal, Literal::SomeIn { .. }) { return Ok(eval_success); @@ -2312,8 +2517,235 @@ impl Interpreter { Ok(eval_success) } + fn is_safe_projected_key_call(&self, expr: &ExprRef) -> Result { + let Expr::Call { fcn, params, .. } = expr.as_ref() else { + return Ok(false); + }; + if !crate::compiler::hoist::is_safe_literal_key_call(fcn, params)? { + return Ok(false); + } + let original_path = crate::utils::get_path_string(fcn, None)?; + let full_path = self + .resolve_fcn_path_through_imports(&original_path) + .unwrap_or_else(|| format!("{}.{}", self.current_module_path, original_path)); + Ok(!self.extensions.contains_key(&original_path) + && !self.with_functions.contains_key(&original_path) + && !self.with_functions.contains_key(&full_path) + && self.lookup_function_by_name(&original_path).is_none() + && self.lookup_function_by_name(&full_path).is_none() + && Self::is_builtin(fcn.span(), &original_path)) + } + + fn projected_key_var_is_bound(&self, span: &Span) -> bool { + self.lookup_local_var(&span.source_str()) + .is_some_and(|value| value != Value::Undefined) + || span.text() == "input" + && !self + .contexts + .last() + .is_some_and(|context| context.input_shadowed) + } + + fn reject_nonmatching_child_key(&mut self) -> Result { + let Some((key_expr, filter)) = self.contexts.last().and_then(|ctx| { + ctx.key_filter + .as_ref() + .zip(ctx.key_expr.as_ref()) + .map(|(filter, expr)| (expr.clone(), filter.clone())) + }) else { + return Ok(false); + }; + if !crate::compiler::hoist::key_expr_is_ready( + &key_expr, + |expr| self.get_loop_var_value(expr).map(|value| value.is_some()), + |span| self.projected_key_var_is_bound(span), + |call| self.is_safe_projected_key_call(call), + )? { + if self + .contexts + .last() + .is_some_and(|context| context.head_key_loops_pending) + { + return Ok(self.projected_head_key_loops_match()? == Some(false)); + } + return Ok(false); + } + Ok(self.eval_expr(&key_expr)? != filter) + } + + fn projected_head_key_loops_match(&mut self) -> Result> { + let Some((key_expr, filter, query_idx)) = self.contexts.last().and_then(|context| { + context + .key_filter + .as_ref() + .zip(context.key_expr.as_ref()) + .zip(context.output_query) + .map(|((filter, expr), query_idx)| (expr.clone(), filter.clone(), query_idx)) + }) else { + return Ok(None); + }; + let loops = self + .loop_hoisting_table(self.current_module_index) + .get_output_loops(self.current_module_index, Some(query_idx), key_expr.eidx()) + .map_err(|err| anyhow!("loop hoisting table out of bounds: {err}"))? + .cloned() + .unwrap_or_default(); + if loops.is_empty() + || loops + .iter() + .any(|loop_info| loop_info.loop_type != LoopType::IndexIteration) + { + return Ok(None); + } + self.scopes.push(Scope::default()); + let result = self.key_matches_in_hoisted_loops(&loops, &key_expr, &filter); + self.scopes.pop(); + if let (Ok(resolved), Some(context)) = (&result, self.contexts.last_mut()) { + context.head_key_loops_pending = resolved.is_none(); + } + result + } + + fn key_matches_in_hoisted_loops( + &mut self, + loops: &[HoistedLoop], + key_expr: &ExprRef, + filter: &Value, + ) -> Result> { + let Some((loop_info, remaining)) = loops.split_first() else { + if !crate::compiler::hoist::key_expr_is_ready( + key_expr, + |expr| self.get_loop_var_value(expr).map(|value| value.is_some()), + |span| self.projected_key_var_is_bound(span), + |call| self.is_safe_projected_key_call(call), + )? { + return Ok(None); + } + return Ok(Some(self.eval_expr(key_expr)? == *filter)); + }; + // An enclosing output loop already bound this iteration; reuse it + // instead of overwriting and then clearing the live binding. + if self + .get_loop_var_value(Self::loop_assignment_expr(loop_info))? + .is_some() + { + return self.key_matches_in_hoisted_loops(remaining, key_expr, filter); + } + if !crate::compiler::hoist::key_expr_is_ready( + &loop_info.collection, + |expr| self.get_loop_var_value(expr).map(|value| value.is_some()), + |span| self.projected_key_var_is_bound(span), + |call| self.is_safe_projected_key_call(call), + )? { + return Ok(None); + } + let collection = self.eval_expr(Self::loop_collection_expr(loop_info))?; + let index_plan = if Self::loop_index_expr(loop_info).is_some() { + self.get_loop_index_plan(loop_info)? + } else { + DestructuringPlan::Ignore + }; + let loop_target = Self::loop_assignment_expr(loop_info).clone(); + self.scopes.push(Scope::default()); + let result = (|| { + match collection { + Value::Array(values) => { + for (index, value) in values.iter().enumerate() { + self.memory_check()?; + self.check_execution_time()?; + match self.key_matches_for_hoisted_loop_item( + &loop_target, + &index_plan, + &Value::from(index), + value, + remaining, + key_expr, + filter, + )? { + Some(true) => return Ok(Some(true)), + Some(false) => {} + // Readiness depends on bindings, not item values, + // so the remaining items are unresolved too. + None => return Ok(None), + } + } + } + Value::Set(values) => { + for value in values.iter() { + self.memory_check()?; + self.check_execution_time()?; + match self.key_matches_for_hoisted_loop_item( + &loop_target, + &index_plan, + value, + value, + remaining, + key_expr, + filter, + )? { + Some(true) => return Ok(Some(true)), + Some(false) => {} + // Readiness depends on bindings, not item values, + // so the remaining items are unresolved too. + None => return Ok(None), + } + } + } + Value::Object(values) => { + for (key, value) in values.iter() { + self.memory_check()?; + self.check_execution_time()?; + match self.key_matches_for_hoisted_loop_item( + &loop_target, + &index_plan, + key, + value, + remaining, + key_expr, + filter, + )? { + Some(true) => return Ok(Some(true)), + Some(false) => {} + // Readiness depends on bindings, not item values, + // so the remaining items are unresolved too. + None => return Ok(None), + } + } + } + Value::Undefined => {} + _ => {} + } + Ok(Some(false)) + })(); + self.scopes.pop(); + self.remove_loop_var_value(&loop_target); + result + } + + #[allow(clippy::too_many_arguments)] + fn key_matches_for_hoisted_loop_item( + &mut self, + loop_target: &ExprRef, + index_plan: &DestructuringPlan, + index: &Value, + value: &Value, + remaining: &[HoistedLoop], + key_expr: &ExprRef, + filter: &Value, + ) -> Result> { + self.set_loop_var_value(loop_target, value.clone())?; + let matches = if self.execute_destructuring_plan(index_plan, index)? == Value::Bool(true) { + self.key_matches_in_hoisted_loops(remaining, key_expr, filter)? + } else { + Some(false) + }; + Self::clear_scope(self.current_scope_mut()?); + Ok(matches) + } + fn eval_query(&mut self, query: &Ref) -> Result { self.check_execution_time()?; + self.get_current_context_mut()?.output_query = Some(query.qidx); // Execute the query in a new scope self.scopes.push(Scope::new()); let order_indices = { @@ -2363,7 +2795,7 @@ impl Interpreter { } }; - let ordered_stmts: Vec<&LiteralStmt> = match order_indices { + let mut ordered_stmts: Vec<&LiteralStmt> = match order_indices { Some(order) => { let stmts_len = query.stmts.len(); if order.len() != stmts_len { @@ -2390,6 +2822,14 @@ impl Interpreter { } None => query.stmts.iter().collect(), }; + if let Some(context) = self.contexts.last() { + crate::compiler::hoist::prioritize_key_binding( + &mut ordered_stmts, + context.key_expr.as_ref(), + context.key_filter.as_ref(), + |call| self.is_safe_projected_key_call(call), + )?; + } let r = self.eval_stmts(&ordered_stmts); self.scopes.pop(); @@ -3118,13 +3558,11 @@ impl Interpreter { let value = self.eval_call_impl(span, expr, fcn, arg_prefix)?; if allow_return_arg { - let module_idx = self.current_module_index; let expr_idx = last_param.as_ref().eidx(); if let Some(BindingPlan::Parameter { destructuring_plan, .. }) = self - .loop_hoisting_table(module_idx) - .get_expr_binding_plan(module_idx, expr_idx) + .expression_binding_plan(expr_idx) .map_err(|err| anyhow!("loop hoisting table out of bounds: {err}"))? .cloned() { @@ -3239,12 +3677,20 @@ impl Interpreter { fn ensure_rule_evaluated(&mut self, path: String) -> Result<()> { self.check_execution_time()?; let mut matched = false; - if let Some(rules) = self.compiled_policy.rules.get(&path) { + let lookup_rules = self + .compiled_policy + .rules_for_lookup(&path) + .map(|rules| rules.into_owned()); + if let Some(rules) = lookup_rules { matched = true; - for r in rules.clone() { + for r in rules { if !self.processed.contains(&r) { let module = self.get_rule_module(&r)?; - self.eval_rule(&module, &r)?; + let key_filter = self.compiled_policy.keyed_set_child_filter(&path, &r); + self.eval_rule_with_key_filter(&module, &r, key_filter)?; + if self.compiled_policy.is_projected_keyed_set_rule(&path, &r) { + self.processed.remove(&r); + } } } } @@ -3521,8 +3967,7 @@ impl Interpreter { let expr_idx = expr.as_ref().eidx(); let expr_text = expr.span().text().to_string(); let binding_plan = self - .loop_hoisting_table(module_idx) - .get_expr_binding_plan(module_idx, expr_idx) + .expression_binding_plan(expr_idx) .map_err(|err| anyhow!("loop hoisting table out of bounds: {err}"))? .cloned() .ok_or_else(|| { @@ -3619,11 +4064,24 @@ impl Interpreter { RuleHead::Set { refr, key, .. } => { Parser::get_path_ref_components_into(refr, &mut path)?; let is_old_style_set = key.is_none(); + let (rule_ref, key_expr) = match refr.as_ref() { + Expr::RefBrack { refr, index, .. } + if !is_old_style_set + && !matches!( + index.as_ref(), + Expr::String { .. } | Expr::RawString { .. } + ) => + { + (refr.clone(), Some(index.clone())) + } + _ => (refr.clone(), None), + }; Ok(( Context { + key_expr, output_expr: key.clone(), value: Value::new_set(), - rule_ref: Some(refr.clone()), + rule_ref: Some(rule_ref), is_set: true, is_old_style_set, ..Context::default() @@ -3697,7 +4155,17 @@ impl Interpreter { } self.contexts.push(next_ctx); } - result = self.eval_query(&body.query); + if let Some(context) = self.contexts.last_mut() { + context.output_query = Some(body.query.qidx); + context.head_key_loops_pending = false; + context.input_shadowed = context.key_filter.is_some() + && crate::compiler::hoist::query_shadows_input(&body.query.stmts)?; + } + result = match self.projected_head_key_loops_match() { + Ok(Some(false)) => Ok(false), + Ok(_) => self.eval_query(&body.query), + Err(error) => Err(error), + }; match &result { Ok(true) => { any_success = true; @@ -4123,7 +4591,12 @@ impl Interpreter { Ok(()) } - fn eval_rule_impl(&mut self, module: &Ref, rule: &Ref) -> Result<()> { + fn eval_rule_impl( + &mut self, + module: &Ref, + rule: &Ref, + key_filter: Option, + ) -> Result<()> { self.check_execution_time()?; match rule.as_ref() { Rule::Spec { @@ -4133,12 +4606,23 @@ impl Interpreter { } => { match rule_head { RuleHead::Compr { refr, .. } | RuleHead::Set { refr, .. } => { - let (ctx, _) = self.make_rule_context(rule_head)?; + let (mut ctx, _) = self.make_rule_context(rule_head)?; + ctx.key_filter = key_filter; let is_set = ctx.is_set; + let is_keyed_set = is_set && ctx.key_expr.is_some(); let is_object = ctx.key_expr.is_some() && !is_set; - let value = self.eval_rule_bodies(ctx, span, rule_body)?; let package_components = self.eval_rule_ref(&module.package.refr)?; + if is_keyed_set { + let mut prefix = package_components.clone(); + prefix.extend( + Self::get_rule_path_components(refr)? + .into_iter() + .map(Value::String), + ); + self.update_rule_value(span, prefix, Value::Undefined, false)?; + } + let value = self.eval_rule_bodies(ctx, span, rule_body)?; if value != Value::Undefined { for (path, value_in_map) in value.as_object()? { @@ -4152,7 +4636,7 @@ impl Interpreter { is_set, )?; } - } else if is_set { + } else if is_set && !is_keyed_set { if let Ok(mut comps) = self.eval_rule_ref(refr) { let mut full_path = package_components; full_path.append(&mut comps); @@ -4223,6 +4707,15 @@ impl Interpreter { } pub fn eval_rule(&mut self, module: &Ref, rule: &Ref) -> Result<()> { + self.eval_rule_with_key_filter(module, rule, None) + } + + fn eval_rule_with_key_filter( + &mut self, + module: &Ref, + rule: &Ref, + key_filter: Option, + ) -> Result<()> { self.check_execution_time()?; // Set current module index self.current_module_index = self.find_module_index(module); @@ -4237,10 +4730,17 @@ impl Interpreter { return Ok(()); } - self.active_rules.push(rule.clone()); - if self.active_rules.iter().filter(|&r| r == rule).count() == 2 { + let invocation = (rule.clone(), key_filter.clone()); + self.active_rules.push(invocation.clone()); + if self + .active_rules + .iter() + .filter(|&r| r == &invocation) + .count() + == 2 + { let mut msg = String::default(); - for r in &self.active_rules { + for (r, _) in &self.active_rules { let refr = Self::get_rule_refr(r); let span = refr.span(); msg.push_str( @@ -4264,13 +4764,26 @@ impl Interpreter { // the local variables of callee function. let scopes = core::mem::take(&mut self.scopes); let prev_module = self.set_current_module(Some(module.clone()))?; - - let res = self.eval_rule_impl(module, rule); - - self.set_current_module(prev_module)?; + // Give the callee an empty view of hoisted-expression bindings without + // copying the table; only slots it overwrites are restored afterwards. + let parent_frame = self.loop_var_frame; + let undo_len = self.loop_var_undo.len(); + self.loop_var_frame_counter = self.loop_var_frame_counter.wrapping_add(1).max(1); + self.loop_var_frame = self.loop_var_frame_counter; + + let res = self.eval_rule_impl(module, rule, key_filter); + + for (module_idx, expr_idx, old) in self.loop_var_undo.drain(undo_len..).rev() { + let _ = self + .loop_var_values + .replace_checked(module_idx, expr_idx, old); + } + self.loop_var_frame = parent_frame; + let restore_module = self.set_current_module(prev_module); self.scopes = scopes; + restore_module?; match self.active_rules.pop() { - Some(ref r) if r == rule => res, + Some(ref r) if r == &invocation => res, _ => bail!("internal error: current rule not active"), } } @@ -4556,6 +5069,8 @@ impl Interpreter { fn clear_rule_registrations(&mut self) { let compiled_policy = self.compiled_policy_mut(); compiled_policy.rules.clear(); + compiled_policy.keyed_set_parent_rules.clear(); + compiled_policy.keyed_set_child_parents.clear(); compiled_policy.default_rules.clear(); compiled_policy.rule_paths.clear(); compiled_policy.rule_path_components.clear(); @@ -4656,7 +5171,7 @@ impl Interpreter { self.set_current_module(prev_module)?; registration_result?; } - Ok(()) + self.record_keyed_set_children() })(); if result.is_err() { self.clear_rule_registrations(); @@ -4664,6 +5179,71 @@ impl Interpreter { result } + // Include dynamic contributions in child lookups without evaluating unrelated siblings. + fn record_keyed_set_children(&mut self) -> Result<()> { + let mut dynamic_sets = Map::::new(); + for module in self.compiled_policy.modules.clone().iter() { + let package = get_path_string(&module.package.refr, Some("data"))?; + for rule in &module.policy { + let Rule::Spec { + head: + RuleHead::Set { + refr, key: Some(_), .. + }, + .. + } = rule.as_ref() + else { + continue; + }; + let Expr::RefBrack { + refr: prefix, + index, + .. + } = refr.as_ref() + else { + continue; + }; + if matches!(index.as_ref(), Expr::String { .. } | Expr::RawString { .. }) { + continue; + } + if get_rule_path_components(prefix)? + .iter() + .any(|component| !matches!(component, PathComponent::String(_))) + { + continue; + } + let parent = get_path_string(prefix, Some(&package))?; + dynamic_sets.entry(parent).or_default().insert(rule.clone()); + self.memory_check()?; + } + } + if dynamic_sets.is_empty() { + return Ok(()); + } + + let mut child_parents = Map::new(); + let policy = self.compiled_policy.clone(); + for path in policy.rules.keys() { + let components = policy + .rule_path_components + .get(path) + .ok_or_else(|| anyhow!("missing components for registered rule path {path}"))?; + if let Some((_, parent)) = components.split_last() { + let parent: Vec<&str> = parent.iter().map(String::as_str).collect(); + let parent = format_string_path(&parent)?; + if dynamic_sets.contains_key(&parent) { + child_parents.insert(path.clone(), parent); + self.memory_check()?; + } + } + } + drop(policy); + let compiled_policy = self.compiled_policy_mut(); + compiled_policy.keyed_set_parent_rules = dynamic_sets; + compiled_policy.keyed_set_child_parents = child_parents; + Ok(()) + } + pub fn add_extension( &mut self, path: String, diff --git a/src/languages/rego/compiler/comprehensions.rs b/src/languages/rego/compiler/comprehensions.rs index 5f9964736..290a5cdba 100644 --- a/src/languages/rego/compiler/comprehensions.rs +++ b/src/languages/rego/compiler/comprehensions.rs @@ -42,9 +42,15 @@ impl<'a> Compiler<'a> { context_type: ContextType::Comprehension(context_type), dest_register: result_reg, key_expr: key_expr.cloned(), + key_filter: None, + key_filter_guarded: false, + multi_value: false, value_expr: value_expr.cloned(), span: span.clone(), + hoisted_key_loops: 0, key_value_loops_hoisted: false, + value_loops: 0, + output_query: Some(query.qidx), }; self.push_context(context); self.compile_query(query)?; diff --git a/src/languages/rego/compiler/core.rs b/src/languages/rego/compiler/core.rs index 314a01b2f..22cbffb3a 100644 --- a/src/languages/rego/compiler/core.rs +++ b/src/languages/rego/compiler/core.rs @@ -116,6 +116,7 @@ impl<'a> Compiler<'a> { pub fn reset_rule_definition_registers(&mut self) { self.current_input_register = None; self.current_data_register = None; + self.loop_expr_register_map.clear(); } /// Push a new compilation context onto the context stack @@ -198,7 +199,13 @@ impl<'a> Compiler<'a> { self.policy .inner .loop_hoisting_table - .get_expr_binding_plan(module_idx, expr_idx) + .get_output_binding_plan( + module_idx, + self.context_stack + .last() + .and_then(|context| context.output_query), + expr_idx, + ) .map_err(|err| { CompilerError::General { message: format!("loop hoisting table out of bounds: {err}"), diff --git a/src/languages/rego/compiler/function_calls.rs b/src/languages/rego/compiler/function_calls.rs index ef93804a7..b210bbb6e 100644 --- a/src/languages/rego/compiler/function_calls.rs +++ b/src/languages/rego/compiler/function_calls.rs @@ -232,7 +232,7 @@ impl<'a> Compiler<'a> { /// keep their prior meaning (e.g. a builtin call). OPA instead rewrites /// aliases unconditionally and rejects calls to a missing target at /// compile time. - fn resolve_fcn_path_through_imports(&self, path: &str) -> Option { + pub(super) fn resolve_fcn_path_through_imports(&self, path: &str) -> Option { if self.policy.inner.imports.is_empty() { return None; } diff --git a/src/languages/rego/compiler/loops.rs b/src/languages/rego/compiler/loops.rs index 8c17f1596..26a84f9f5 100644 --- a/src/languages/rego/compiler/loops.rs +++ b/src/languages/rego/compiler/loops.rs @@ -52,7 +52,13 @@ impl<'a> Compiler<'a> { .policy .inner .loop_hoisting_table - .get_expr_loops(module_idx, expr_idx) + .get_output_loops( + module_idx, + self.context_stack + .last() + .and_then(|context| context.output_query), + expr_idx, + ) .map_err(|err| { CompilerError::General { message: format!("loop hoisting table out of bounds: {err}"), @@ -72,20 +78,32 @@ impl<'a> Compiler<'a> { }) } + /// Compiles `loops` around `stmts`. When `first_stmt_owns_loops` is set, + /// the loops were hoisted from `stmts[0]`, which is compiled directly in + /// the innermost loop; otherwise every statement is compiled normally. pub(super) fn compile_hoisted_loops( &mut self, stmts: &[&LiteralStmt], loops: &[HoistedLoop], + first_stmt_owns_loops: bool, ) -> Result<()> { if loops.is_empty() { + if !first_stmt_owns_loops { + return self.hoist_loops_and_compile_statements(stmts); + } if !stmts.is_empty() { self.compile_single_statement(stmts[0])?; + self.emit_key_filter_guard(&stmts[0].span)?; return self.hoist_loops_and_compile_statements(&stmts[1..]); } else { return self.hoist_loops_and_emit_context_yield(); } } + if first_stmt_owns_loops && stmts.is_empty() { + self.emit_projected_key_guard_before_value_loops(loops)?; + } + let current_loop = &loops[0]; let remaining_loops = &loops[1..]; @@ -94,10 +112,10 @@ impl<'a> Compiler<'a> { self.compile_index_iteration_loop( ¤t_loop.loop_expr, ¤t_loop.key, - ¤t_loop.value, ¤t_loop.collection, stmts, remaining_loops, + first_stmt_owns_loops, )?; Ok(()) } @@ -156,9 +174,15 @@ impl<'a> Compiler<'a> { context_type: ContextType::Every, dest_register: result_reg, key_expr: None, + key_filter: None, + key_filter_guarded: false, + multi_value: false, value_expr: None, span: span.clone(), + hoisted_key_loops: 0, key_value_loops_hoisted: false, + value_loops: 0, + output_query: Some(query.qidx), }; self.push_context(every_context); @@ -217,10 +241,10 @@ impl<'a> Compiler<'a> { &mut self, loop_expr: &Option, key_var: &Option, - _value_var: &ExprRef, collection: &ExprRef, remaining_stmts: &[&LiteralStmt], remaining_loops: &[HoistedLoop], + first_stmt_owns_loops: bool, ) -> Result<()> { let collection_reg = self.compile_rego_expr(collection)?; @@ -296,9 +320,10 @@ impl<'a> Compiler<'a> { .apply_binding_plan(binding_plan, key_reg, plan_span) .map_err(|e| CompilerError::from(e).at(plan_span))?; } + self.emit_key_filter_guard(collection.span())?; let body_stmts = &remaining_stmts[0..]; - self.compile_hoisted_loops(body_stmts, remaining_loops)?; + self.compile_hoisted_loops(body_stmts, remaining_loops, first_stmt_owns_loops)?; self.emit_instruction( Instruction::LoopNext { @@ -466,6 +491,7 @@ impl<'a> Compiler<'a> { } } + self.emit_key_filter_guard(collection.span())?; self.hoist_loops_and_compile_statements(loop_body_stmts)?; self.emit_instruction( diff --git a/src/languages/rego/compiler/mod.rs b/src/languages/rego/compiler/mod.rs index 60121a60f..dec332d3d 100644 --- a/src/languages/rego/compiler/mod.rs +++ b/src/languages/rego/compiler/mod.rs @@ -61,9 +61,18 @@ pub struct CompilationContext { pub(super) context_type: ContextType, pub(super) dest_register: Register, pub(super) key_expr: Option, + /// Requested static child when projecting a dynamic keyed-set definition. + pub(super) key_filter: Option, + pub(super) key_filter_guarded: bool, + pub(super) multi_value: bool, pub(super) value_expr: Option, pub(super) span: Span, + pub(super) hoisted_key_loops: usize, pub(super) key_value_loops_hoisted: bool, + /// Head value loops trailing the hoisted key/value loops; the projected + /// key is checked before entering them. + pub(super) value_loops: usize, + pub(super) output_query: Option, } /// Entry in the rule compilation worklist that tracks both rule path and full call stack for recursion detection diff --git a/src/languages/rego/compiler/queries.rs b/src/languages/rego/compiler/queries.rs index 8ba78e2d8..99812baa5 100644 --- a/src/languages/rego/compiler/queries.rs +++ b/src/languages/rego/compiler/queries.rs @@ -7,14 +7,40 @@ )] use super::{Compiler, CompilerError, ComprehensionType, ContextType, Result}; -use crate::ast::{self, LiteralStmt, Query}; +use crate::ast::{self, Expr, ExprRef, LiteralStmt, Query}; +use crate::compiler::hoist::{ + is_safe_literal_key_call, key_reads_loop_index_vars, prioritize_key_binding, HoistedLoop, +}; use crate::rvm::instructions::GuardMode; use crate::rvm::program::RuleType; use crate::rvm::Instruction; use alloc::format; +use alloc::string::ToString as _; use alloc::vec::Vec; impl<'a> Compiler<'a> { + fn is_safe_projected_key_call(&self, expr: &ExprRef) -> Result { + let Expr::Call { fcn, params, .. } = expr.as_ref() else { + return Ok(false); + }; + if !is_safe_literal_key_call(fcn, params).map_err(CompilerError::from)? { + return Ok(false); + } + let original_path = crate::utils::get_path_string(fcn, None) + .map_err(|_| CompilerError::InvalidFunctionExpression)?; + let full_path = if self.policy.inner.rules.contains_key(&original_path) { + original_path.clone() + } else if let Some(resolved) = self.resolve_fcn_path_through_imports(&original_path) { + resolved + } else { + crate::utils::get_path_string(fcn, Some(&self.current_package)) + .map_err(|_| CompilerError::InvalidFunctionExpressionWithPackage)? + }; + Ok(self.is_builtin(&original_path) + && !self.is_user_defined_function(&full_path) + && !self.host_await_builtins.contains_key(&original_path)) + } + pub(super) fn compile_query(&mut self, query: &Query) -> Result<()> { self.push_scope(); @@ -32,7 +58,7 @@ impl<'a> Compiler<'a> { None => None, }; - let ordered_stmts: Vec<&LiteralStmt> = match schedule { + let mut ordered_stmts: Vec<&LiteralStmt> = match schedule { Some(schedule) => schedule .order .iter() @@ -40,6 +66,23 @@ impl<'a> Compiler<'a> { .collect(), None => query.stmts.iter().collect(), }; + if let Some(context) = self.context_stack.last() { + prioritize_key_binding( + &mut ordered_stmts, + context.key_expr.as_ref(), + context.key_filter.as_ref(), + |expr| { + self.is_safe_projected_key_call(expr) + .map_err(|err| anyhow::anyhow!("{err}")) + }, + ) + .map_err(|err| { + CompilerError::General { + message: err.to_string(), + } + .at(&query.span) + })?; + } self.hoist_loops_and_compile_statements(&ordered_stmts) }; @@ -52,6 +95,13 @@ impl<'a> Compiler<'a> { &mut self, stmts: &[&LiteralStmt], ) -> Result<()> { + if self.hoist_ready_key_loops(stmts)? { + return Ok(()); + } + if let Some(context) = self.context_stack.last() { + let span = context.span.clone(); + self.emit_key_filter_guard(&span)?; + } for (idx, stmt) in stmts.iter().enumerate() { if !stmt.with_mods.is_empty() { return Err(CompilerError::WithKeywordUnsupported.at(&stmt.span)); @@ -59,7 +109,9 @@ impl<'a> Compiler<'a> { let loop_exprs = self.get_statement_loops(stmt)?; if !loop_exprs.is_empty() { - return self.compile_hoisted_loops(&stmts[idx..], &loop_exprs); + self.compile_hoisted_loops(&stmts[idx..], &loop_exprs, true)?; + self.emit_key_filter_guard(&stmt.span)?; + return Ok(()); } if matches!(&stmt.literal, ast::Literal::SomeIn { .. }) { @@ -95,17 +147,180 @@ impl<'a> Compiler<'a> { &stmt.span, ); } + self.emit_key_filter_guard(&stmt.span)?; return Ok(()); } } self.compile_single_statement(stmt)?; + self.emit_key_filter_guard(&stmt.span)?; + if self.hoist_ready_key_loops( + stmts + .get(idx..) + .and_then(|rest| rest.get(1..)) + .unwrap_or_default(), + )? { + return Ok(()); + } } self.hoist_loops_and_emit_context_yield() } + /// Hoists projected head-key loops as soon as their collections are bound, + /// so nonmatching keys are rejected before dependent body statements run. + /// Loops whose collections need later body bindings stay pending. + fn hoist_ready_key_loops(&mut self, stmts: &[&LiteralStmt]) -> Result { + let (key_expr, start) = match self.context_stack.last() { + Some(context) + if matches!(context.context_type, ContextType::Rule(_)) + && context.key_filter.is_some() + && !context.key_value_loops_hoisted => + { + (context.key_expr.clone(), context.hoisted_key_loops) + } + _ => (None, 0), + }; + let Some(key_expr) = key_expr else { + return Ok(false); + }; + let loops = self.get_expr_loops(&key_expr)?; + let mut ready = loops.len(); + for (idx, loop_info) in loops.iter().enumerate().skip(start) { + let prefix = loops.get(..idx).unwrap_or_default(); + let is_ready = crate::compiler::hoist::key_expr_is_ready( + &loop_info.collection, + |node| { + Ok(self.loop_expr_register_map.contains_key(node) + || prefix.iter().any(|l| { + l.loop_expr.as_ref() == Some(node) || l.key.as_ref() == Some(node) + })) + }, + |span| { + span.text() == "input" + || self.lookup_variable(span.text()).is_some() + || prefix.iter().any(|l| { + matches!(l.key.as_deref(), Some(Expr::Var { span: key, .. }) if key.text() == span.text()) + }) + }, + |call| self.is_safe_projected_key_call(call).map_err(Into::into), + ) + .map_err(CompilerError::from)?; + if !is_ready { + ready = idx; + break; + } + } + if ready <= start { + return Ok(false); + } + if let Some(context) = self.context_stack.last_mut() { + context.hoisted_key_loops = ready; + } + self.compile_hoisted_loops(stmts, loops.get(start..ready).unwrap_or_default(), false)?; + Ok(true) + } + + pub(super) fn emit_key_filter_guard(&mut self, span: &crate::lexer::Span) -> Result<()> { + let Some((expr, filter)) = self.context_stack.last().and_then(|context| { + if context.key_filter_guarded { + return None; + } + let (ContextType::Rule(_), Some(filter), Some(expr)) = ( + &context.context_type, + context.key_filter.as_ref(), + context.key_expr.as_ref(), + ) else { + return None; + }; + Some((expr.clone(), filter.clone())) + }) else { + return Ok(()); + }; + if !crate::compiler::hoist::key_expr_is_ready( + &expr, + |node| Ok(self.loop_expr_register_map.contains_key(node)), + |key_span| { + key_span.text() == "input" || self.lookup_variable(key_span.text()).is_some() + }, + |call| self.is_safe_projected_key_call(call).map_err(Into::into), + ) + .map_err(CompilerError::from)? + { + return Ok(()); + } + let key_register = self.compile_rego_expr(&expr)?; + self.emit_key_match_guard(key_register, filter, span); + if let Some(context) = self.context_stack.last_mut() { + context.key_filter_guarded = true; + } + Ok(()) + } + + fn emit_key_match_guard( + &mut self, + key_register: super::Register, + filter: crate::Value, + span: &crate::lexer::Span, + ) { + let literal_idx = self.add_literal(filter); + let expected = self.alloc_register(); + self.emit_instruction( + Instruction::Load { + dest: expected, + literal_idx, + }, + span, + ); + let matches = self.alloc_register(); + self.emit_instruction( + Instruction::Eq { + dest: matches, + left: key_register, + right: expected, + }, + span, + ); + self.emit_instruction( + Instruction::Guard { + register: matches, + mode: GuardMode::Condition, + }, + span, + ); + } + + /// Rejects a nonmatching projected key once key loops are bound and + /// before head value loops evaluate collections that may read the child. + pub(super) fn emit_projected_key_guard_before_value_loops( + &mut self, + remaining_loops: &[HoistedLoop], + ) -> Result<()> { + let Some(context) = self.context_stack.last() else { + return Ok(()); + }; + if context.value_loops == 0 || remaining_loops.len() != context.value_loops { + return Ok(()); + } + let (Some(filter), Some(key_expr)) = (context.key_filter.clone(), context.key_expr.clone()) + else { + return Ok(()); + }; + // A key bound by a value loop is checked once that loop binds it. + if key_reads_loop_index_vars(&key_expr, remaining_loops).map_err(CompilerError::from)? { + return Ok(()); + } + let span = context.span.clone(); + let key_register = self.compile_rego_expr(&key_expr)?; + self.emit_key_match_guard(key_register, filter, &span); + Ok(()) + } + pub(super) fn hoist_loops_and_emit_context_yield(&mut self) -> Result<()> { + if let Some(context) = self.context_stack.last() { + let span = context.span.clone(); + self.emit_key_filter_guard(&span)?; + } if let Some(context) = self.context_stack.last() { match &context.context_type { ContextType::Every => { @@ -128,18 +343,37 @@ impl<'a> Compiler<'a> { let mut key_value_loops = Vec::new(); if let Some(expr) = key_expr.as_ref() { - key_value_loops.extend(self.get_expr_loops(expr)?); + let hoisted_key_loops = self + .context_stack + .last() + .map_or(0, |context| context.hoisted_key_loops); + if !Self::is_simple_literal(expr) { + key_value_loops.extend( + self.get_expr_loops(expr)? + .into_iter() + .skip(hoisted_key_loops), + ); + } } - if let Some(expr) = value_expr.as_ref() { - key_value_loops.extend(self.get_expr_loops(expr)?); + let value_loops = match value_expr.as_ref() { + Some(expr) => self.get_expr_loops(expr)?, + None => Vec::new(), + }; + let value_loop_count = value_loops.len(); + key_value_loops.extend(value_loops); + + if let Some(context) = self.context_stack.last() { + let span = context.span.clone(); + self.emit_key_filter_guard(&span)?; } if !key_value_loops.is_empty() { if let Some(context) = self.context_stack.last_mut() { context.key_value_loops_hoisted = true; + context.value_loops = value_loop_count; } - self.compile_hoisted_loops(&[], &key_value_loops) + self.compile_hoisted_loops(&[], &key_value_loops, true) } else { self.emit_context_yield() } @@ -149,6 +383,20 @@ impl<'a> Compiler<'a> { if let Some(context) = self.context_stack.last().cloned() { let dest_register = context.dest_register; let span = &context.span; + let projected_key_register = if context.key_filter.is_some() { + context + .key_expr + .as_ref() + .map(|expr| self.compile_rego_expr(expr)) + .transpose()? + } else { + None + }; + if let (Some(key_register), Some(key)) = + (projected_key_register, context.key_filter.clone()) + { + self.emit_key_match_guard(key_register, key, span); + } let value_register = match context.value_expr { Some(expr) => self.compile_rego_expr(&expr)?, None => { @@ -164,10 +412,13 @@ impl<'a> Compiler<'a> { } }; - let key_register = context - .key_expr - .map(|key_expr| self.compile_rego_expr(&key_expr)) - .unwrap_or(Ok(value_register))?; + let key_register = match projected_key_register { + Some(register) => register, + None => context + .key_expr + .map(|key_expr| self.compile_rego_expr(&key_expr)) + .unwrap_or(Ok(value_register))?, + }; match context.context_type { ContextType::Comprehension(ComprehensionType::Array) => { @@ -208,10 +459,18 @@ impl<'a> Compiler<'a> { } ContextType::Rule(RuleType::PartialObject) => { self.emit_instruction( - Instruction::ObjectSet { - obj: dest_register, - key: key_register, - value: value_register, + if context.multi_value { + Instruction::ObjectSetAdd { + obj: dest_register, + key: key_register, + value: value_register, + } + } else { + Instruction::ObjectSet { + obj: dest_register, + key: key_register, + value: value_register, + } }, span, ); diff --git a/src/languages/rego/compiler/references.rs b/src/languages/rego/compiler/references.rs index ca8d8c5bd..e3d0aee96 100644 --- a/src/languages/rego/compiler/references.rs +++ b/src/languages/rego/compiler/references.rs @@ -169,7 +169,14 @@ impl<'a> Compiler<'a> { let rule_candidate = format_string_path(&static_prefix[0..=i]).map_err(CompilerError::from)?; - if let Ok(rule_index) = self.get_or_assign_rule_index(&rule_candidate) { + if self.policy.inner.rules.contains_key(&rule_candidate) + || self + .policy + .inner + .default_rules + .contains_key(&rule_candidate) + { + let rule_index = self.get_or_assign_rule_index(&rule_candidate)?; // Found a rule match! Call the rule let rule_result_reg = self.alloc_register(); self.emit_instruction( @@ -309,7 +316,14 @@ impl<'a> Compiler<'a> { append_path_component(&rule_candidate, field).map_err(CompilerError::from)?; } - if let Ok(rule_index) = self.get_or_assign_rule_index(&rule_candidate) { + if self.policy.inner.rules.contains_key(&rule_candidate) + || self + .policy + .inner + .default_rules + .contains_key(&rule_candidate) + { + let rule_index = self.get_or_assign_rule_index(&rule_candidate)?; let rule_result_reg = self.alloc_register(); self.emit_instruction( Instruction::CallRule { diff --git a/src/languages/rego/compiler/rules.rs b/src/languages/rego/compiler/rules.rs index abd6b0641..265a29e58 100644 --- a/src/languages/rego/compiler/rules.rs +++ b/src/languages/rego/compiler/rules.rs @@ -38,7 +38,7 @@ impl<'a> Compiler<'a> { } pub(super) fn compute_rule_type(&self, rule_path: &str) -> Result { - let Some(definitions) = self.policy.inner.rules.get(rule_path) else { + let Some(definitions) = self.policy.inner.rules_for_lookup(rule_path) else { // Default-only rules (e.g., `default deny := true`) have no regular definitions // in the `rules` map — they only exist in `default_rules`. Treat them as Complete. if self.policy.inner.default_rules.contains_key(rule_path) { @@ -50,11 +50,25 @@ impl<'a> Compiler<'a> { .into()); }; + let mut has_keyed_set = false; + let mut has_single_value = false; let rule_types: BTreeSet = definitions .iter() .map(|def| { - if let Rule::Spec { head, .. } = def.as_ref() { + Ok(if let Rule::Spec { head, .. } = def.as_ref() { + has_single_value |= !matches!(head, RuleHead::Set { .. }); match head { + RuleHead::Set { + refr, key: Some(_), .. + } => { + let (key, filter) = self.keyed_set_key(rule_path, def, refr)?; + has_keyed_set |= key.is_some(); + if key.is_some() && filter.is_none() { + RuleType::PartialObject + } else { + RuleType::PartialSet + } + } RuleHead::Set { .. } => RuleType::PartialSet, RuleHead::Compr { refr, assign, .. } => match refr.as_ref() { crate::ast::Expr::RefBrack { .. } if assign.is_some() => { @@ -67,9 +81,18 @@ impl<'a> Compiler<'a> { } } else { RuleType::Complete - } + }) }) - .collect(); + .collect::>()?; + + if has_keyed_set && has_single_value { + return Err(CompilerError::General { + message: format!( + "conflicting rule kinds for '{rule_path}': contains and single-valued heads" + ), + } + .into()); + } if rule_types.len() > 1 { return Err(CompilerError::General { @@ -89,6 +112,60 @@ impl<'a> Compiler<'a> { }) } + fn keyed_set_key( + &self, + rule_path: &str, + rule: &crate::ast::NodeRef, + refr: &ExprRef, + ) -> Result<(Option, Option)> { + let (prefix, key) = match refr.as_ref() { + Expr::RefBrack { refr, index, .. } => (refr, index.clone()), + Expr::RefDot { + refr, field, eidx, .. + } => ( + refr, + ExprRef::new(Expr::String { + span: field.0.clone(), + value: field.1.clone(), + eidx: *eidx, + }), + ), + _ => return Ok((None, None)), + }; + if Self::has_unsupported_bracket_prefix(prefix) { + return Err(CompilerError::PartialObjectNestedKeyUnsupported.at(refr.span())); + } + + let module_index = self.find_module_index_for_rule(rule)?; + let module = &self.policy.get_modules()[module_index as usize]; + let package = + get_path_string(&module.package.refr, Some("data")).map_err(CompilerError::from)?; + if Self::is_string_literal(&key) + && get_path_string(refr, Some(&package)).map_err(CompilerError::from)? == rule_path + { + return Ok((None, None)); + } + let prefix_path = get_path_string(prefix, Some(&package)).map_err(CompilerError::from)?; + if prefix_path != rule_path { + if !Self::is_string_literal(&key) { + if let Some((leaf, parent)) = self + .policy + .inner + .rule_path_components + .get(rule_path) + .and_then(|components| components.split_last()) + { + let parent: Vec<&str> = parent.iter().map(String::as_str).collect(); + if format_string_path(&parent).map_err(CompilerError::from)? == prefix_path { + return Ok((Some(key), Some(Value::from(leaf.clone())))); + } + } + } + return Err(CompilerError::PartialObjectNestedKeyUnsupported.at(refr.span())); + } + Ok((Some(key), None)) + } + fn validate_partial_object_shape(&self, refr: &ExprRef) -> Result<()> { let Expr::RefBrack { refr: prefix, @@ -124,7 +201,7 @@ impl<'a> Compiler<'a> { matches!(expr.as_ref(), Expr::String { .. } | Expr::RawString { .. }) } - fn is_simple_literal(expr: &ExprRef) -> bool { + pub(super) fn is_simple_literal(expr: &ExprRef) -> bool { match expr.as_ref() { Expr::String { .. } | Expr::RawString { .. } @@ -401,7 +478,7 @@ impl<'a> Compiler<'a> { self.current_module_index = module_index; let saved_register_counter = self.register_counter; - if let Some(rule_definitions) = rules.get(rule_path) { + if let Some(rule_definitions) = self.policy.inner.rules_for_lookup(rule_path) { let Some(rule_index) = self.rule_index_map.get(rule_path).copied() else { return Err(CompilerError::General { message: format!( @@ -440,6 +517,41 @@ impl<'a> Compiler<'a> { self.rule_definition_static_values.push(Vec::new()); } + // Seed all static leaves once, before any body can fail or accumulate members. + let mut initial_object = Value::new_object(); + if rule_type == RuleType::PartialObject { + for rule in rule_definitions.iter() { + if let Rule::Spec { + head: + RuleHead::Set { + refr, key: Some(_), .. + }, + .. + } = rule.as_ref() + { + if let (Some(key), _) = self.keyed_set_key(rule_path, rule, refr)? { + if let Expr::String { value, .. } | Expr::RawString { value, .. } = + key.as_ref() + { + initial_object + .as_object_mut() + .map_err(CompilerError::from)? + .insert(value.clone(), Value::new_set()); + } + } + } + } + } + let initial_literal = if initial_object + .as_object() + .map_err(CompilerError::from)? + .is_empty() + { + None + } else { + Some(self.add_literal(initial_object)) + }; + let mut num_registers_used = 0; let mut rule_param_count: Option = None; @@ -452,7 +564,15 @@ impl<'a> Compiler<'a> { let result_register = self.alloc_register(); self.current_module_index = self.find_module_index_for_rule(rule_ref)?; - + self.current_package = get_path_string( + &self.policy.get_modules()[self.current_module_index as usize] + .package + .refr, + Some("data"), + ) + .map_err(CompilerError::from)?; + + let mut key_filter = None; let (key_expr, value_expr) = match head { RuleHead::Compr { refr, assign, .. } => { if rule_type == RuleType::PartialObject { @@ -470,12 +590,23 @@ impl<'a> Compiler<'a> { }; (key_expr, output_expr) } - RuleHead::Set { key, .. } => { + RuleHead::Set { refr, key, .. } => { self.rule_definition_function_params[rule_index as usize].push(None); self.rule_definition_destructuring_patterns[rule_index as usize] .push(None); - (None, key.clone()) + let key_expr = if key.is_some() { + let (key_expr, filter) = + self.keyed_set_key(rule_path, rule_ref, refr)?; + key_filter = filter; + if key_filter.is_some() { + self.program.needs_runtime_recursion_check = true; + } + key_expr + } else { + None + }; + (key_expr, key.clone()) } RuleHead::Func { assign, args, .. } => { let mut param_names = Vec::new(); @@ -572,9 +703,15 @@ impl<'a> Compiler<'a> { dest_register: result_register, context_type: ContextType::Rule(rule_type.clone()), key_expr, + key_filter, + key_filter_guarded: false, + multi_value: matches!(head, RuleHead::Set { .. }), value_expr, span, + hoisted_key_loops: 0, key_value_loops_hoisted: false, + value_loops: 0, + output_query: None, }; self.push_context(context); let mut body_entry_points = Vec::new(); @@ -596,8 +733,19 @@ impl<'a> Compiler<'a> { }, value_expr.span(), ); + if def_idx == 0 { + if let Some(literal_idx) = initial_literal { + self.emit_instruction( + Instruction::Load { + dest: result_register, + literal_idx, + }, + value_expr.span(), + ); + } + } - self.emit_context_yield()?; + self.hoist_loops_and_emit_context_yield()?; self.emit_instruction(Instruction::RuleReturn {}, value_expr.span()); self.pop_scope(); @@ -631,6 +779,11 @@ impl<'a> Compiler<'a> { if let Some(context) = self.context_stack.last_mut() { context.value_expr = body_value_expr.clone(); + context.hoisted_key_loops = 0; + context.key_value_loops_hoisted = false; + context.value_loops = 0; + context.key_filter_guarded = false; + context.output_query = Some(body.query.qidx); } self.emit_instruction( @@ -640,18 +793,22 @@ impl<'a> Compiler<'a> { }, &body.span, ); + if def_idx == 0 && body_idx == 0 { + if let Some(literal_idx) = initial_literal { + self.emit_instruction( + Instruction::Load { + dest: result_register, + literal_idx, + }, + &body.span, + ); + } + } if !body.query.stmts.is_empty() { self.compile_query(&body.query)?; - } else if let Some(value_expr) = body_value_expr.clone() { - let value_reg = self.compile_rego_expr(&value_expr)?; - self.emit_instruction( - Instruction::Move { - dest: result_register, - src: value_reg, - }, - value_expr.span(), - ); + } else { + self.hoist_loops_and_emit_context_yield()?; } self.emit_instruction(Instruction::RuleReturn {}, &body.span); diff --git a/src/lookup.rs b/src/lookup.rs index eb5b80b48..1bf4bf1c8 100644 --- a/src/lookup.rs +++ b/src/lookup.rs @@ -114,19 +114,23 @@ impl Lookup { Ok(()) } + /// Replace the slot contents, returning the previous contents. + pub(crate) fn replace_checked( + &mut self, + module_idx: u32, + node_idx: u32, + value: Option, + ) -> LookupResult> { + let slot = self.slot_mut(module_idx, node_idx)?; + Ok(core::mem::replace(slot, value)) + } + /// Get data using direct indices with bounds checking. /// Returns Ok(None) if the entry is unset, Err if indices are out of range. pub(crate) fn get_checked(&self, module_idx: u32, node_idx: u32) -> LookupResult> { self.slot_ref(module_idx, node_idx) } - /// Clear data at the given indices by setting it to None. - pub(crate) fn clear(&mut self, module_idx: u32, node_idx: u32) { - if let Ok(slot) = self.slot_mut(module_idx, node_idx) { - *slot = None; - } - } - /// Validate indices and return them as usize on success. fn validate_indices(&self, module_idx: u32, node_idx: u32) -> LookupResult<(usize, usize)> { let m = usize_from_u32(module_idx).ok_or(LookupIndexError::ModuleOutOfBounds { diff --git a/src/rvm/instructions/display.rs b/src/rvm/instructions/display.rs index 1b8a00fd2..9131a27ca 100644 --- a/src/rvm/instructions/display.rs +++ b/src/rvm/instructions/display.rs @@ -201,6 +201,9 @@ impl core::fmt::Display for Instruction { Instruction::ObjectSet { obj, key, value } => { format!("OBJECT_SET R({}) R({}) R({})", obj, key, value) } + Instruction::ObjectSetAdd { obj, key, value } => { + format!("OBJECT_SET_ADD R({}) R({}) R({})", obj, key, value) + } Instruction::ObjectCreate { params_index } => { format!("OBJECT_CREATE P({})", params_index) } diff --git a/src/rvm/instructions/mod.rs b/src/rvm/instructions/mod.rs index 82fe0f70a..2d46632b8 100644 --- a/src/rvm/instructions/mod.rs +++ b/src/rvm/instructions/mod.rs @@ -17,6 +17,7 @@ pub use types::{ use serde::{Deserialize, Serialize}; /// RVM Instructions - simplified enum-based design +#[non_exhaustive] #[derive(Debug, Clone, Copy, Serialize, Deserialize)] pub enum Instruction { /// Load literal value from literal table into register @@ -416,6 +417,14 @@ pub enum Instruction { /// Register that accumulates the result. result: u8, }, + + /// Add a defined value to the set at `obj[key]`, creating the set if absent. + /// Used by keyed partial-set rules (`rule[key] contains value`). + ObjectSetAdd { + obj: u8, + key: u8, + value: u8, + }, } impl Instruction { diff --git a/src/rvm/program/listing.rs b/src/rvm/program/listing.rs index 9b825e819..27153c9a7 100644 --- a/src/rvm/program/listing.rs +++ b/src/rvm/program/listing.rs @@ -507,6 +507,17 @@ fn format_instruction_readable( let comment = format!("Set field r{}[r{}] = r{}", obj, key, value); align_comment(&base, &comment, config.comment_column) } + Instruction::ObjectSetAdd { obj, key, value } => { + let base = format!( + "{}ObjectSetAdd r{}[r{}].insert(r{})", + indent, obj, key, value + ); + align_comment( + &base, + "Accumulate a keyed partial set", + config.comment_column, + ) + } Instruction::ObjectCreate { params_index } => { let params = program .instruction_data @@ -1022,6 +1033,7 @@ const fn get_instruction_name(instruction: &Instruction) -> &'static str { Instruction::HostAwait { .. } => "HOST_AWAIT", Instruction::Return { .. } => "RETURN", Instruction::ObjectSet { .. } => "OBJ_SET", + Instruction::ObjectSetAdd { .. } => "OBJ_SET_ADD", Instruction::ObjectCreate { .. } => "OBJ_CREATE", Instruction::Index { .. } => "INDEX", Instruction::IndexLiteral { .. } => "INDEX_LIT", diff --git a/src/rvm/tests/instruction_parser.rs b/src/rvm/tests/instruction_parser.rs index 5d3237524..e956f13da 100644 --- a/src/rvm/tests/instruction_parser.rs +++ b/src/rvm/tests/instruction_parser.rs @@ -53,6 +53,14 @@ pub fn parse_instruction(text: &str) -> Result { "RuleReturn" => parse_rule_return(params_text), "DestructuringSuccess" => parse_destructuring_success(params_text), "ObjectSet" => parse_object_set(params_text), + "ObjectSetAdd" => { + let params = parse_params(params_text)?; + Ok(Instruction::ObjectSetAdd { + obj: get_param_u16(¶ms, "obj")?.try_into()?, + key: get_param_u16(¶ms, "key")?.try_into()?, + value: get_param_u16(¶ms, "value")?.try_into()?, + }) + } "ObjectCreate" => parse_object_create(params_text), "Index" => parse_index(params_text), "IndexLiteral" => parse_index_literal(params_text), diff --git a/src/rvm/vm/dispatch.rs b/src/rvm/vm/dispatch.rs index b1913fbb8..5d5fac0f7 100644 --- a/src/rvm/vm/dispatch.rs +++ b/src/rvm/vm/dispatch.rs @@ -444,6 +444,46 @@ impl RegoVM { ) -> Result { use Instruction::*; match instruction { + ObjectSetAdd { obj, key, value } => { + let key_value = self.get_register(key)?.clone(); + let value_value = self.get_register(value)?.clone(); + let mut obj_value = self.take_register(obj)?; + let result = (|| { + let Value::Object(ref mut fields) = obj_value else { + return Err(VmError::RegisterNotObject { + register: obj, + value: obj_value.clone(), + pc: self.pc, + }); + }; + if key_value != Value::Undefined && value_value == Value::Undefined { + if let Some(set) = fields.get(&key_value) { + if !matches!(set, Value::Set(_)) { + return Err(VmError::RegisterNotSet { + register: obj, + value: set.clone(), + pc: self.pc, + }); + } + } + } + if key_value != Value::Undefined && value_value != Value::Undefined { + let set = crate::Rc::make_mut(fields) + .get_or_insert_with(key_value, Value::new_set); + let Value::Set(ref mut elements) = *set else { + return Err(VmError::RegisterNotSet { + register: obj, + value: set.clone(), + pc: self.pc, + }); + }; + crate::Rc::make_mut(elements).insert(value_value); + } + Ok(InstructionOutcome::Continue) + })(); + self.set_register(obj, obj_value)?; + result + } ObjectSet { obj, key, value } => { let key_value = self.get_register(key)?.clone(); let value_value = self.get_register(value)?.clone(); diff --git a/src/rvm/vm/errors.rs b/src/rvm/vm/errors.rs index c856c1029..8494f27db 100644 --- a/src/rvm/vm/errors.rs +++ b/src/rvm/vm/errors.rs @@ -9,6 +9,7 @@ use core::time::Duration; use thiserror::Error; /// VM execution errors +#[non_exhaustive] #[derive(Error, Debug, Clone, PartialEq)] pub enum VmError { #[error("Execution stopped: exceeded maximum instruction limit of {limit} after {executed} instructions (pc={pc})")] @@ -87,6 +88,9 @@ pub enum VmError { available: usize, }, + #[error("Runtime recursion detected in rule {rule_index} (pc={pc})")] + RuleRecursionDetected { rule_index: u16, pc: usize }, + #[error("Invalid object create params index: {index} (pc={pc}, available={available})")] InvalidObjectCreateParams { index: u16, diff --git a/src/rvm/vm/rules.rs b/src/rvm/vm/rules.rs index 4e27f685c..6b365ba99 100644 --- a/src/rvm/vm/rules.rs +++ b/src/rvm/vm/rules.rs @@ -17,7 +17,7 @@ use super::execution_model::{ use super::machine::RegoVM; impl RegoVM { - /// Returns true if the error represents a resource-limit violation that + /// Returns true if the error represents a fatal evaluation failure that /// must never be silently absorbed by rule evaluation. pub(super) const fn is_fatal_vm_error(err: &VmError) -> bool { matches!( @@ -27,10 +27,26 @@ impl RegoVM { | VmError::MemoryBudgetExceeded { .. } | VmError::RegexSizeLimitExceeded { .. } | VmError::InstructionLimitExceeded { .. } + | VmError::RuleRecursionDetected { .. } | VmError::NumericOverflow { .. } ) } + fn check_rule_recursion(&self, rule_index: u16) -> Result<()> { + if self.program.needs_runtime_recursion_check + && self + .call_rule_stack + .iter() + .any(|context| context.rule_index == rule_index) + { + return Err(VmError::RuleRecursionDetected { + rule_index, + pc: self.pc, + }); + } + Ok(()) + } + /// Restore VM state that was swapped out for rule execution. /// Must be called before returning an error from `execute_rule_definitions_common` /// to avoid leaving the VM in an inconsistent state. @@ -250,6 +266,8 @@ impl RegoVM { } } + self.check_rule_recursion(rule_index)?; + let rule_type = rule_info.rule_type.clone(); let rule_definitions = rule_info.definitions.clone(); @@ -417,6 +435,8 @@ impl RegoVM { } } + self.check_rule_recursion(rule_index)?; + if rule_info.definitions.is_empty() { // No compiled definitions — check for a default value before returning Undefined. let result = rule_info @@ -709,11 +729,30 @@ impl RegoVM { } } - if let Some(definition_bodies) = rule_info - .definitions + let next_is_else = rule_info + .else_bodies .get(frame_data.current_definition_index) + .and_then(|bodies| { + frame_data + .current_body_index + .checked_add(1) + .and_then(|next| bodies.get(next)) + }) + .copied() + .unwrap_or(false); + if !matches!( + frame_data.rule_type, + RuleType::PartialSet | RuleType::PartialObject + ) || next_is_else { - frame_data.current_body_index = definition_bodies.len(); + if let Some(definition_bodies) = rule_info + .definitions + .get(frame_data.current_definition_index) + { + frame_data.current_body_index = definition_bodies.len(); + } else { + self.increment_counter(&mut frame_data.current_body_index, "rule body index")?; + } } else { self.increment_counter(&mut frame_data.current_body_index, "rule body index")?; } diff --git a/src/tests/engine.rs b/src/tests/engine.rs index 37fd82b4b..00b1e7df3 100644 --- a/src/tests/engine.rs +++ b/src/tests/engine.rs @@ -981,3 +981,122 @@ fn has_declared_rule_rooted_at_handles_32_component_heads_and_cloned_load_contex Ok(()) } + +fn eval_v0_shadowed_input_alias(local_key: &str, input_key: &str) -> Result { + let mut engine = Engine::new(); + engine.set_rego_v0(true); + engine.add_policy( + "test.rego".to_string(), + format!( + "package test\nimport future.keywords.contains\nimport future.keywords.if\n\ + values.a contains 1\n\ + values[z] contains n if {{ input := {{\"key\": \"{local_key}\"}}; n := count(values.a); z := input.key }}" + ), + )?; + engine.set_input(Value::from_json_str(&format!( + "{{\"key\": \"{input_key}\"}}" + ))?); + engine.eval_rule("data.test.values.a".to_string()) +} + +// Key bindings that read `input` must not move above a Rego v0 local +// `input` shadow; the RVM does not support such shadowing yet. +#[test] +fn keyed_set_input_alias_respects_v0_input_shadowing() -> Result<()> { + let result = eval_v0_shadowed_input_alias("b", "a")?; + anyhow::ensure!( + result == Value::from(alloc::collections::BTreeSet::from([Value::from(1_u64)])), + "nonmatching shadowed key must be filtered, got {result}" + ); + let err = eval_v0_shadowed_input_alias("a", "b") + .err() + .ok_or_else(|| anyhow::anyhow!("matching shadowed key must still recurse"))?; + anyhow::ensure!( + err.to_string().contains("recursion detected"), + "unexpected error: {err}" + ); + Ok(()) +} + +fn eval_v0_shadowed_input_head(rule: &str, input: &str, query: &str) -> Result { + let mut engine = Engine::new(); + engine.set_rego_v0(true); + engine.add_policy( + "test.rego".to_string(), + format!( + "package test\nimport future.keywords.contains\nimport future.keywords.if\n\ + import future.keywords.in\nvalues.a contains 1\n{rule}" + ), + )?; + engine.set_input(Value::from_json_str(input)?); + engine.eval_rule(query.to_string()) +} + +// A projected head key that reads `input` must not be filtered against the +// global input before a Rego v0 body binds its local `input` shadow. The +// projected query must agree with unfiltered evaluation of the whole rule. +#[test] +fn keyed_set_direct_input_head_respects_v0_input_shadowing() -> Result<()> { + let set = |items: &[u64]| { + Value::from( + items + .iter() + .map(|item| Value::from(*item)) + .collect::>(), + ) + }; + let cases = [ + ( + "values[input.key] contains 2 if { input := {\"key\": \"a\"} }", + "{\"key\": \"b\"}", + Some(set(&[1, 2])), + ), + ( + "values[input.key] contains 2 if { input := {\"key\": \"b\"} }", + "{\"key\": \"a\"}", + Some(set(&[1])), + ), + ( + "values[input.groups[_]] contains 2 if { input := {\"groups\": [\"c\", \"a\"]} }", + "{\"groups\": [\"b\"]}", + Some(set(&[1, 2])), + ), + ( + "values[input.groups[_]] contains 2 if { input := {\"groups\": [\"c\"]} }", + "{\"groups\": [\"a\"]}", + Some(set(&[1])), + ), + ( + "values[input.key] contains 2 if { some input in [{\"key\": \"a\"}] }", + "{\"key\": \"b\"}", + None, + ), + ( + "values[input.key] contains 2 if { some input; input := {\"key\": \"a\"} }", + "{\"key\": \"b\"}", + None, + ), + // Without a shadow, the global input still drives filtering. + ( + "values[input.key] contains 2 if { true }", + "{\"key\": \"a\"}", + Some(set(&[1, 2])), + ), + ]; + for (rule, input, expected) in cases { + let projected = eval_v0_shadowed_input_head(rule, input, "data.test.values.a")?; + let full = eval_v0_shadowed_input_head(rule, input, "data.test.values")?; + let full = full["a"].clone(); + anyhow::ensure!( + projected == full, + "{rule} with input {input}: projected {projected} != unfiltered {full}" + ); + if let Some(expected) = expected { + anyhow::ensure!( + projected == expected, + "{rule} with input {input}: expected {expected}, got {projected}" + ); + } + } + Ok(()) +} diff --git a/tests/engine/mod.rs b/tests/engine/mod.rs index 9c71e982f..b5e5633b7 100644 --- a/tests/engine/mod.rs +++ b/tests/engine/mod.rs @@ -3,6 +3,234 @@ use anyhow::{bail, Result}; use regorus::*; +use std::collections::BTreeSet; + +#[test] +fn keyed_set_expression_projection_preserves_sibling_override() -> Result<()> { + for head in [ + "values[input.key] contains 2", + "values[input.keys[_]] contains 2", + "values[input.keys[_]] contains 2 if { true }", + ] { + let mut engine = Engine::new(); + engine.add_policy( + "test.rego".into(), + format!("package test\nvalues.a contains 1\n{head}"), + )?; + engine.set_input(Value::from_json_str(r#"{"key":"b","keys":["a","b"]}"#)?); + let result = engine.eval_query( + "[data.test.values.a, data.test.values.b] with data.test.values.b as {9}".into(), + false, + )?; + let first = if head.contains("keys") { + vec![1, 2] + } else { + vec![1] + }; + let expected = Value::from(vec![ + Value::from(first.into_iter().map(Value::from).collect::>()), + Value::from(BTreeSet::from([Value::from(9)])), + ]); + assert_eq!(result.result[0].expressions[0].value, expected, "{head}"); + } + Ok(()) +} + +#[test] +fn keyed_set_projection_preserves_cycle_detection() -> Result<()> { + for (policy, entrypoint) in [ + ( + r#"package test + values.a contains 1 + values[k] contains n if { k := "a"; n := count(values.a) }"#, + "data.test.values.a", + ), + ( + r#"package test + values.a contains 1 + values[k] contains n if { k := "a"; n := count(values.a) }"#, + "data.test.values", + ), + ( + r#"package test + values.a contains 1 + values.b contains 2 + values[k] contains n if { k := "a"; n := count(values.b) } + values[k] contains n if { k := "b"; n := count(values.a) }"#, + "data.test.values.a", + ), + ] { + let mut engine = Engine::new(); + engine.add_policy("test.rego".into(), policy.into())?; + let error = engine + .eval_rule(entrypoint.into()) + .expect_err("a genuine keyed-set cycle must be rejected"); + assert!( + error.to_string().contains("recursion detected"), + "{entrypoint}: {error}" + ); + } + Ok(()) +} + +#[test] +fn keyed_set_parent_loop_projection_state_is_invocation_local() -> Result<()> { + let mut engine = Engine::new(); + engine.add_policy( + "test.rego".into(), + r#"package test + values.a contains 1 + values[input.keys[_]] contains count(data.test.values.a)"# + .into(), + )?; + + for keys in [r#"["b","a"]"#, r#"["a","b"]"#] { + engine.set_input(Value::from_json_str(&format!(r#"{{"keys":{keys}}}"#))?); + for path in ["data.test.values", "data.test.values.a"] { + let error = engine + .eval_rule(path.into()) + .expect_err("matching projected recursion must be rejected"); + assert!( + error.to_string().contains("recursion detected"), + "{path}: {error}" + ); + } + + engine.set_input(Value::from_json_str(r#"{"keys":["b"]}"#)?); + assert_eq!( + engine.eval_rule("data.test.values.a".into())?, + Value::from(BTreeSet::from([Value::from(1)])) + ); + } + Ok(()) +} + +#[test] +fn keyed_set_guarded_head_arithmetic_runs_after_body_success() -> Result<()> { + let mut engine = Engine::new(); + engine.add_policy( + "test.rego".into(), + r#"package test + values.a contains 1 + values[k] contains 2 if { is_number(input.port); k := input.port + 1 }"# + .into(), + )?; + + engine.set_input(Value::from_json_str(r#"{"port":"invalid"}"#)?); + assert_eq!( + engine.eval_rule("data.test.values.a".into())?, + Value::from(BTreeSet::from([Value::from(1)])) + ); + + engine.set_input(Value::from_json_str(r#"{"port":80}"#)?); + let result = engine.eval_rule("data.test.values".into())?; + assert_eq!( + result, + Value::from(std::collections::BTreeMap::from([ + ( + Value::from("a"), + Value::from(BTreeSet::from([Value::from(1)])), + ), + ( + Value::from(81), + Value::from(BTreeSet::from([Value::from(2)])), + ), + ])) + ); + + let mut guarded_engine = Engine::new(); + guarded_engine.add_policy( + "guard.rego".into(), + r#"package guard + values[k] contains 2 if { input.enabled; k := input.port + 1 }"# + .into(), + )?; + guarded_engine.set_input(Value::from_json_str( + r#"{"port":"invalid","enabled":true}"#, + )?); + let error = guarded_engine + .eval_rule("data.guard.values".into()) + .expect_err("a successful body must still report a failing head expression"); + assert!(!error.to_string().contains("recursion detected"), "{error}"); + guarded_engine.set_input(Value::from_json_str(r#"{"port":80,"enabled":true}"#)?); + assert_eq!( + guarded_engine.eval_rule("data.guard.values".into())?, + Value::from(std::collections::BTreeMap::from([( + Value::from(81), + Value::from(BTreeSet::from([Value::from(2)])), + )])) + ); + Ok(()) +} + +#[test] +fn keyed_set_does_not_speculate_overridden_builtin_key_calls() -> Result<()> { + let mut engine = Engine::new(); + engine.add_extension( + "lower".to_string(), + 1, + Box::new(|_| Err(anyhow::anyhow!("overridden lower must not run"))), + )?; + engine.add_policy( + "test.rego".into(), + r#"package test + values.a contains 1 + values[k] contains 2 if { false; k := lower("B") }"# + .into(), + )?; + + assert_eq!( + engine.eval_rule("data.test.values.a".into())?, + Value::from(BTreeSet::from([Value::from(1)])) + ); + Ok(()) +} + +#[test] +fn keyed_set_multiple_bodies_remain_rejected() { + let mut engine = Engine::new(); + let error = engine + .add_policy( + "test.rego".into(), + r#"package test + values.a contains 1 + values[k] contains n if { k := "b"; n := 2 } + { k := "b"; n := count(values.a) }"# + .into(), + ) + .expect_err("multiple set bodies are unsupported"); + assert!( + error + .to_string() + .contains("else cannot be used with set rules"), + "{error}" + ); +} + +#[test] +fn keyed_set_child_override_preserves_dynamic_sibling_contributions() -> Result<()> { + let mut engine = Engine::new(); + engine.add_policy( + "test.rego".into(), + r#"package test + values.a contains 1 + values[k] contains 2 if { k := "b" } + main := result if { + result := values.b with data.test.values.a as {9} + }"# + .into(), + )?; + + assert_eq!( + engine.eval_rule("data.test.main".into())?, + Value::from(BTreeSet::from([Value::from(2)])) + ); + assert_eq!( + engine.eval_rule("data.test.values.a".into())?, + Value::from(BTreeSet::from([Value::from(1)])) + ); + Ok(()) +} #[test] fn package_components_distinguish_literal_dots_from_nested_namespaces() -> Result<()> { diff --git a/tests/rvm/rego/cases/keyed_partial_sets.yaml b/tests/rvm/rego/cases/keyed_partial_sets.yaml new file mode 100644 index 000000000..818c1ad8a --- /dev/null +++ b/tests/rvm/rego/cases/keyed_partial_sets.yaml @@ -0,0 +1,2321 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +cases: + - note: keyed_set_compound_key_binding_before_dependency + input: {keys: {chosen: b}} + modules: + - | + package test + values.a contains 1 + values[input.keys[z]] contains n if { n := count(values.a); z := "chosen" } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_compound_key_binding_before_dependency_suspendable + execution_mode: suspendable + input: {keys: {chosen: b}} + modules: + - | + package test + values.a contains 1 + values[input.keys[z]] contains n if { n := count(values.a); z := "chosen" } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_multiple_compound_key_bindings_before_dependency + input: {map: {foo: {bar: b}}} + modules: + - | + package test + values.a contains 1 + values[input.map[x][y]] contains n if { + n := count(values.a) + x := "foo" + y := "bar" + } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_call_key_body_dependency + modules: + - | + package test + values.a contains 1 + values[k] contains n if { n := count(values.a); k := lower("B") } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_call_key_body_dependency_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains n if { n := count(values.a); k := lower("B") } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_call_key_body_matching_cycle + modules: + - | + package test + values.a contains 1 + values[k] contains n if { n := count(values.a); k := lower("A") } + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: keyed_set_call_key_body_matching_cycle_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains n if { n := count(values.a); k := lower("A") } + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: keyed_set_call_key_value_loop + modules: + - | + package test + values.a contains 1 + values[k] contains [count(data.test.values.a)][_] if { k := concat("", ["b"]) } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_call_key_value_loop_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains [count(data.test.values.a)][_] if { k := concat("", ["b"]) } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_call_key_value_loop_matching_cycle + modules: + - | + package test + values.a contains 1 + values[k] contains [count(data.test.values.a)][_] if { k := concat("", ["a"]) } + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: keyed_set_call_key_value_loop_matching_cycle_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains [count(data.test.values.a)][_] if { k := concat("", ["a"]) } + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: keyed_set_head_loop_before_dependent_body + input: {keys: [b]} + modules: + - | + package test + values.a contains 1 + values[input.keys[_]] contains count(data.test.values.a) + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_head_loop_before_dependent_body_suspendable + execution_mode: suspendable + input: {keys: [b]} + modules: + - | + package test + values.a contains 1 + values[input.keys[_]] contains count(data.test.values.a) + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_head_loop_matching_cycle + input: {keys: [a]} + modules: + - | + package test + values.a contains 1 + values[input.keys[_]] contains count(data.test.values.a) + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: keyed_set_head_loop_matching_cycle_suspendable + execution_mode: suspendable + input: {keys: [a]} + modules: + - | + package test + values.a contains 1 + values[input.keys[_]] contains count(data.test.values.a) + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: keyed_set_guarded_arithmetic_invalid + input: {port: "invalid"} + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { is_number(input.port); k := input.port + 1 } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_guarded_arithmetic_invalid_suspendable + execution_mode: suspendable + input: {port: "invalid"} + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { is_number(input.port); k := input.port + 1 } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_guarded_arithmetic_missing_input + input: {} + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { is_number(input.port); k := input.port + 1 } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_guarded_arithmetic_missing_input_suspendable + execution_mode: suspendable + input: {} + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { is_number(input.port); k := input.port + 1 } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_guarded_arithmetic_valid + input: {port: 80} + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { is_number(input.port); k := input.port + 1 } + query: data.test.values + want_result: + a: + set!: [1] + 81: + set!: [2] + + - note: keyed_set_guarded_arithmetic_valid_suspendable + execution_mode: suspendable + input: {port: 80} + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { is_number(input.port); k := input.port + 1 } + query: data.test.values + want_result: + a: + set!: [1] + 81: + set!: [2] + + - note: overlapping_keyed_set_head_value_loop_dependency + input: {keys: [b, c]} + modules: + - | + package test + values.a contains 1 + values[input.keys[_]] contains [count(values.a)][_] + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_head_value_loop_dependency_suspendable + execution_mode: suspendable + input: {keys: [b, c]} + modules: + - | + package test + values.a contains 1 + values[input.keys[_]] contains [count(values.a)][_] + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_input_key_dependency + input: {key: b} + modules: + - | + package test + values.a contains 1 + values[input.key] contains n if { n := count(values.a) } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_input_key_dependency_suspendable + execution_mode: suspendable + input: {key: b} + modules: + - | + package test + values.a contains 1 + values[input.key] contains n if { n := count(values.a) } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_bodyless_input_key_dependency + input: {key: b} + modules: + - | + package test + values.a contains 1 + values[input.key] contains count(values.a) + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_bodyless_input_key_dependency_suspendable + execution_mode: suspendable + input: {key: b} + modules: + - | + package test + values.a contains 1 + values[input.key] contains count(values.a) + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_missing_input_key_dependency + input: {} + modules: + - | + package test + values.a contains 1 + values[input.key] contains count(values.a) + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_missing_input_key_dependency_suspendable + execution_mode: suspendable + input: {} + modules: + - | + package test + values.a contains 1 + values[input.key] contains count(values.a) + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_unification_before_dependency + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); z = "b" } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_unification_before_dependency_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); z = "b" } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_reversed_unification_before_dependency + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); "b" = z } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_reversed_unification_before_dependency_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); "b" = z } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_input_head_loop_dependency + input: {keys: [b, c]} + modules: + - | + package test + values.a contains 1 + values[input.keys[_]] contains count(values.a) + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_input_head_loop_dependency_suspendable + execution_mode: suspendable + input: {keys: [b, c]} + modules: + - | + package test + values.a contains 1 + values[input.keys[_]] contains count(values.a) + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_matching_input_key + input: {key: a} + modules: + - | + package test + values.a contains 1 + values[input.key] contains 2 + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_matching_input_key_suspendable + execution_mode: suspendable + input: {key: a} + modules: + - | + package test + values.a contains 1 + values[input.key] contains 2 + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_later_bound_expression_key + input: {keys: {chosen: a}} + modules: + - | + package test + values.a contains 1 + values[input.keys[k]] contains 2 if { k := "chosen" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_later_bound_expression_key_suspendable + execution_mode: suspendable + input: {keys: {chosen: a}} + modules: + - | + package test + values.a contains 1 + values[input.keys[k]] contains 2 if { k := "chosen" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_matching_cycle + modules: + - | + package test + values.a contains 1 + values[k] contains n if { k := "a"; n := count(values.a) } + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: overlapping_keyed_set_matching_cycle_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains n if { k := "a"; n := count(values.a) } + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: overlapping_keyed_set_sibling_cycle + modules: + - | + package test + values.a contains 1 + values.b contains 2 + values[k] contains n if { k := "a"; n := count(values.b) } + values[k] contains n if { k := "b"; n := count(values.a) } + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: overlapping_keyed_set_sibling_cycle_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values.b contains 2 + values[k] contains n if { k := "a"; n := count(values.b) } + values[k] contains n if { k := "b"; n := count(values.a) } + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: overlapping_keyed_set_index_bound_before_dependency + modules: + - | + package test + values.a contains 1 + values[k] contains n if { n := {"b": 2}[k] + count(values.a) } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_index_bound_before_dependency_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains n if { n := {"b": 2}[k] + count(values.a) } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_dependent_lookup_order + modules: + - | + package test + values.a contains 0 + values.b contains 4 + values[k] contains n if { k := "a"; n := count(values.b) } + main := [values.a, values, values.b] + query: data.test.main + want_result: + - set!: [0, 1] + - a: + set!: [0, 1] + b: + set!: [4] + - set!: [4] + + - note: overlapping_keyed_set_dependent_lookup_order_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 0 + values.b contains 4 + values[k] contains n if { k := "a"; n := count(values.b) } + main := [values, values.b, values.a] + query: data.test.main + want_result: + - a: + set!: [0, 1] + b: + set!: [4] + - set!: [4] + - set!: [0, 1] + + - note: overlapping_keyed_set_declared_key_unification + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { some k; k = "a" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_declared_key_unification_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { some k; k = "a" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_parent_dependent_child + modules: + - | + package test + values.a contains 1 + values[k] contains n if { k := "b"; n := count(values.a) } + query: data.test.values + want_result: + a: + set!: [1] + b: + set!: [1] + + - note: overlapping_keyed_set_parent_dependent_child_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains n if { k := "b"; n := count(values.a) } + query: data.test.values + want_result: + a: + set!: [1] + b: + set!: [1] + + - note: overlapping_keyed_set_sibling_projection + modules: + - | + package test + values.a contains 0 + values.b contains 4 + values[k] contains n if { k := "a"; n := count(values.b) } + query: data.test.values.a + want_result: + set!: [0, 1] + + - note: overlapping_keyed_set_sibling_projection_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 0 + values.b contains 4 + values[k] contains n if { k := "a"; n := count(values.b) } + query: data.test.values.a + want_result: + set!: [0, 1] + + - note: overlapping_keyed_set_indexed_assignment + modules: + - | + package test + values.a contains 1 + values[k] contains n if { k := ["b"][_]; n := count(values.a) } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_indexed_assignment_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains n if { k := ["b"][_]; n := count(values.a) } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_indexed_matching_assignment + modules: + - | + package test + values.a contains 1 + values[k] contains n if { k := ["b", "a"][_]; n := 2 } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_indexed_matching_assignment_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains n if { k := ["b", "a"][_]; n := 2 } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_nonmatching_dependent_child + modules: + - | + package test + values.a contains 1 + values[k] contains n if { + k := "b" + n := count(values.a) + } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_nonmatching_dependent_child_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains n if { + k := "b" + n := count(values.a) + } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_nonmatching_dependency_before_key + modules: + - | + package test + values.a contains 1 + values[k] contains n if { + n := count(values.a) + k := "b" + } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_nonmatching_dependency_before_key_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains n if { + n := count(values.a) + k := "b" + } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_nonmatching_some_in_dependency + modules: + - | + package test + values.a contains 1 + values[k] contains n if { + some k in ["b"] + n := count(values.a) + } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_nonmatching_some_in_dependency_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains n if { + some k in ["b"] + n := count(values.a) + } + query: data.test.values.a + want_result: + set!: [1] + + - note: overlapping_keyed_set_local_child + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { k := "a" } + main := values.a + query: data.test.main + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_local_child_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { k := "a" } + main := values.a + query: data.test.main + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_child_entrypoint + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { k := "a" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_child_entrypoint_suspendable + execution_mode: suspendable + modules: + - | + package test + values[k] contains 2 if { k := "a" } + values["a"] contains 1 + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_external_child + modules: + - | + package test + main := data.other.values.a + - | + package other + values.a contains 1 + values[k] contains 2 if { k := "a" } + query: data.test.main + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_imported_child_suspendable + execution_mode: suspendable + modules: + - | + package test + import data.other.values.a as child + main := child + - | + package other + values.a contains 1 + - | + package other + values[k] contains 2 if { k := "a" } + query: data.test.main + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_parent_then_child + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { k := "a" } + main := [values, values.a] + query: data.test.main + want_result: + - a: + set!: [1, 2] + - set!: [1, 2] + + - note: overlapping_keyed_set_child_nested_package_suspendable + execution_mode: suspendable + modules: + - | + package test + extra := 2 + values[k] contains extra if { k := "a" } + - | + package test.values + a contains 1 + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_child_then_parent_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { k := "a" } + main := [values.a, values] + query: data.test.main + want_result: + - set!: [1, 2] + - a: + set!: [1, 2] + + - note: overlapping_keyed_set_child_head_loop + input: {keys: ["a", "b", "a"], vals: [2, 99, 3]} + modules: + - | + package test + values.a contains 1 + values[input.keys[i]] contains input.vals[i] + query: data.test.values.a + want_result: + set!: [1, 2, 3] + + - note: overlapping_keyed_set_child_failed_dynamic_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 if { false } + values[k] contains 2 if { k := "a"; false } + query: data.test.values.a + want_result: + set!: [] + + - note: overlapping_keyed_set_child_head_loop_suspendable + execution_mode: suspendable + input: {keys: ["a", "b", "a"], vals: [false, 99, null]} + modules: + - | + package test + values.a contains 1 + values[input.keys[i]] contains input.vals[i] if { true } + query: data.test.values.a + want_result: + set!: [1, false, null] + + - note: overlapping_keyed_set_child_sibling_dependency + modules: + - | + package test + values.a contains count(values.b) + values.b contains 4 + values[k] contains 2 if { k := "a" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_child_sibling_dependency_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains count(values.b) + values.b contains 4 + values[k] contains 2 if { k := "a" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: overlapping_keyed_set_escaped_child_suspendable + execution_mode: suspendable + modules: + - | + package test + values["a.b"] contains 1 + values[k] contains 2 if { k := "a.b" } + query: data.test.values["a.b"] + want_result: + set!: [1, 2] + + - note: output_head_body_specific_unbound_then_bound + input: {keys: ["a", "b"]} + modules: + - | + package test + values[input.keys[i]] := i if { true } { i := 1 } + query: data.test.values + want_result: {a: 0, b: 1} + + - note: output_head_body_specific_unbound_then_bound_suspendable + execution_mode: suspendable + input: {keys: ["a", "b"]} + modules: + - | + package test + values[input.keys[i]] := i if { true } { i := 1 } + query: data.test.values + want_result: {a: 0, b: 1} + + - note: output_head_body_specific_bound_then_unbound + input: {keys: ["a", "b"]} + modules: + - | + package test + values[input.keys[i]] := i if { i := 0 } { true } + query: data.test.values + want_result: {a: 0, b: 1} + + - note: output_head_body_specific_bound_then_unbound_suspendable + execution_mode: suspendable + input: {keys: ["a", "b"]} + modules: + - | + package test + values[input.keys[i]] := i if { i := 0 } { true } + query: data.test.values + want_result: {a: 0, b: 1} + + - note: output_head_body_specific_bound_then_failed + input: {keys: ["a", "b"]} + modules: + - | + package test + values[input.keys[i]] := i if { i := 0 } { false } + query: data.test.values + want_result: {a: 0} + + - note: output_head_body_specific_bound_then_failed_suspendable + execution_mode: suspendable + input: {keys: ["a", "b"]} + modules: + - | + package test + values[input.keys[i]] := i if { i := 0 } { false } + query: data.test.values + want_result: {a: 0} + + - note: output_head_body_specific_paired_values + input: {keys: ["a", "b"], vals: [10, 20]} + modules: + - | + package test + values[input.keys[i]] := input.vals[i] if { true } { i := 1 } + query: data.test.values + want_result: {a: 10, b: 20} + + - note: output_head_body_specific_paired_values_suspendable + execution_mode: suspendable + input: {keys: ["a", "b"], vals: [10, 20]} + modules: + - | + package test + values[input.keys[i]] := input.vals[i] if { i := 0 } { true } + query: data.test.values + want_result: {a: 10, b: 20} + + - note: output_head_body_specific_builtin_values + execution_mode: suspendable + input: {keys: ["a", "b"], vals: ["A", "B"]} + modules: + - | + package test + values[input.keys[i]] := lower(input.vals[i]) if { true } { i := 1 } + query: data.test.values + want_result: {a: "a", b: "b"} + + - note: keyed_partial_set_shared_head_index + input: {keys: ["a", "b"], vals: [10, 20]} + modules: + - | + package test + values[input.keys[i]] contains input.vals[i] if { true } + query: data.test.values + want_result: + a: + set!: [10] + b: + set!: [20] + + - note: keyed_partial_set_shared_head_index_bodyless + execution_mode: suspendable + input: {keys: ["a", "b"], vals: [10, 20]} + modules: + - | + package test + values[input.keys[i]] contains input.vals[i] + query: data.test.values + want_result: + a: + set!: [10] + b: + set!: [20] + + - note: keyed_partial_set_shared_head_index_suspendable + execution_mode: suspendable + input: {keys: ["a", "b"], vals: [10, 20]} + modules: + - | + package test + values[input.keys[i]] contains input.vals[i] if { true } + query: data.test.values + want_result: + a: + set!: [10] + b: + set!: [20] + + - note: keyed_partial_set_shared_head_index_bodyless_completion + input: {keys: ["a", "b"], vals: [10, 20]} + modules: + - | + package test + values[input.keys[i]] contains input.vals[i] + query: data.test.values + want_result: + a: + set!: [10] + b: + set!: [20] + + - note: keyed_partial_set_shared_index_with_nested_value_loop + execution_mode: suspendable + input: {keys: ["a", "b"], vals: [[10, 11], [20, 21]]} + modules: + - | + package test + values[input.keys[i]] contains input.vals[i][j] if { true } + query: data.test.values + want_result: + a: + set!: [10, 11] + b: + set!: [20, 21] + + - note: keyed_partial_set_nested_head_key_loops + input: + groups: + - keys: ["a", "b"] + vals: [10, 20] + modules: + - | + package test + values.b contains 99 + values[input.groups[i].keys[j]] contains input.groups[i].vals[j] if { true } + query: data.test.values.b + want_result: + set!: [20, 99] + + - note: keyed_partial_set_body_bound_head_index + input: {keys: ["a", "b"], vals: [10, 20]} + modules: + - | + package test + values[input.keys[i]] contains input.vals[i] if { i := 1 } + query: data.test.values + want_result: + b: + set!: [20] + + - note: keyed_partial_set_shared_comprehension_head_index + execution_mode: suspendable + input: {keys: ["a", "b"], vals: [10, 20]} + modules: + - | + package test + main := {input.keys[i]: input.vals[i] | true} + query: data.test.main + want_result: {a: 10, b: 20} + + - note: keyed_partial_set_head_index_value + input: {keys: ["a", "b"]} + modules: + - | + package test + values[input.keys[i]] contains i if { true } + query: data.test.values + want_result: + a: + set!: [0] + b: + set!: [1] + + - note: keyed_partial_set_shared_object_head_index + execution_mode: suspendable + input: {keys: {x: "a", y: "b"}, vals: {x: 10, y: 20}} + modules: + - | + package test + values[input.keys[i]] contains input.vals[i] if { true } + query: data.test.values + want_result: + a: + set!: [10] + b: + set!: [20] + + - note: keyed_partial_set_shared_set_head_index + modules: + - | + package test + values[keys[i]] contains i if { keys := {"a", "b"} } + query: data.test.values + want_result: + a: + set!: ["a"] + b: + set!: ["b"] + + - note: keyed_partial_set_short_shared_value_collection + execution_mode: suspendable + input: {keys: ["a", "b", "c"], vals: [10, 20]} + modules: + - | + package test + values[input.keys[i]] contains input.vals[i] if { true } + query: data.test.values + want_result: + a: + set!: [10] + b: + set!: [20] + + - note: keyed_partial_set_independent_head_wildcards + input: {keys: ["a", "b"], vals: [10, 20]} + modules: + - | + package test + values[input.keys[_]] contains input.vals[_] if { true } + query: data.test.values + want_result: + a: + set!: [10, 20] + b: + set!: [10, 20] + + - note: keyed_partial_set_static_empty_leaf + modules: + - | + package test + values.a contains 1 if { false } + main := values + query: data.test.main + want_result: + a: + set!: [] + + - note: keyed_partial_set_static_empty_leaf_suspendable + execution_mode: suspendable + modules: + - | + package test + values["a"] contains 1 if { false } + main := values + query: data.test.main + want_result: + a: + set!: [] + + - note: keyed_partial_set_static_empty_with_dynamic + modules: + - | + package test + values.a contains 1 if { false } + values[k] contains 2 if { k := "b" } + values[k] contains 3 if { k := "c"; false } + query: data.test.values + want_result: + a: + set!: [] + b: + set!: [2] + + - note: keyed_partial_set_static_empty_after_dynamic + execution_mode: suspendable + modules: + - | + package test + values[k] contains 2 if { k := "a" } + values.a contains 1 if { false } + query: data.test.values + want_result: + a: + set!: [2] + + - note: keyed_partial_set_static_empty_with_bodyless_first + execution_mode: suspendable + modules: + - | + package test + values.b contains 2 + values.a contains 1 if { false } + values.c contains 3 if { false } + main := values + query: data.test.main + want_result: + a: + set!: [] + b: + set!: [2] + c: + set!: [] + + - note: keyed_partial_set_static_empty_before_dynamic + modules: + - | + package test + values.a contains 1 if { false } + values[k] contains 2 if { k := "a" } + values.a contains 3 if { false } + query: data.test.values + want_result: + a: + set!: [2] + + - note: keyed_partial_set_rejects_dynamic_dotted_prefix + modules: + - | + package test + values[group].fixed contains 1 if { group := "g" } + main := values + query: data.test.main + want_error: "partial object rules with nested bracket keys are not yet supported by the RVM compiler" + allow_interpreter_success: true + + - note: keyed_partial_set_rejects_dynamic_dotted_prefix_suspendable + execution_mode: suspendable + modules: + - | + package test + values[group].fixed contains 1 if { group := "g" } + main := data.test.values + query: data.test.main + want_error: "partial object rules with nested bracket keys are not yet supported by the RVM compiler" + allow_interpreter_success: true + + - note: keyed_partial_set_rejects_flattened_static_prefix + modules: + - | + package test + values.fixed[k] contains 1 if { k := "a" } + main := values + query: data.test.main + want_error: "partial object rules with nested bracket keys are not yet supported by the RVM compiler" + allow_interpreter_success: true + + - note: keyed_partial_set_rejects_flattened_static_prefix_suspendable + execution_mode: suspendable + modules: + - | + package test + values.fixed[k] contains 1 if { k := "a" } + main := data.test.values + query: data.test.main + want_error: "partial object rules with nested bracket keys are not yet supported by the RVM compiler" + allow_interpreter_success: true + + - note: keyed_partial_set_multiple_body_head_loops + input: {keys: ["a", "b"]} + modules: + - | + package test + values[input.keys[_]] := 1 if { false } { true } + query: data.test.values + want_result: + a: 1 + b: 1 + + - note: keyed_partial_set_multiple_body_head_loops_suspendable + execution_mode: suspendable + input: {keys: ["a", "b"]} + modules: + - | + package test + values[input.keys[_]] := 1 if { false } { true } + query: data.test.values + want_result: + a: 1 + b: 1 + + - note: keyed_partial_set_scalar_head_collections + input: {collections: [null, false, true, 42, "abc"]} + modules: + - | + package test + values[collection[_]] contains 1 if { + some collection in input.collections + } + query: data.test.values + want_result: {} + + - note: keyed_partial_set_scalar_head_collections_suspendable + execution_mode: suspendable + input: {collections: [null, false, true, 42, "abc"]} + modules: + - | + package test + values[collection[_]] contains 1 if { + some collection in input.collections + } + query: data.test.values + want_result: {} + + - note: keyed_partial_set_resolvable_key_failing_body + input: {key: "a", enabled: false} + modules: + - | + package test + values[input.key] contains 1 if { input.enabled } + query: data.test.values + want_result: {} + + - note: keyed_partial_set_resolvable_key_failing_body_suspendable + execution_mode: suspendable + input: {key: "a", enabled: false} + modules: + - | + package test + values[input.key] contains 1 if { input.enabled } + query: data.test.values + want_result: {} + + - note: keyed_partial_set_failing_body_preserves_other_keys + input: {key: "a", enabled: false} + modules: + - | + package test + values[input.key] contains 1 if { input.enabled } + values[k] contains 2 if { k := "b" } + query: data.test.values + want_result: + b: + set!: [2] + + - note: keyed_partial_set_resolvable_key_undefined_body + input: {key: "a"} + modules: + - | + package test + values[input.key] contains 1 if { input.enabled } + query: data.test.values + want_result: {} + + - note: review_missing_head_key_collection + input: {} + modules: + - | + package test + values[input.keys[_]] contains 1 if { true } + query: data.test.values + want_result: {} + + - note: review_missing_head_key_collection_suspendable + execution_mode: suspendable + input: {} + modules: + - | + package test + values[input.keys[_]] contains 1 if { true } + query: data.test.values + want_result: {} + + - note: review_mixed_keyed_rule_kinds + modules: + - | + package test + values[k] contains 1 if { k := "a" } + values[k] := 2 if { k := "a" } + query: data.test.values + want_error: "conflicting rule kinds" + + - note: review_mixed_keyed_rule_kinds_reverse + execution_mode: suspendable + modules: + - | + package test + values[k] := 2 if { k := "a" } + values[k] contains 1 if { k := "a" } + query: data.test.values + want_error: "conflicting rule kinds" + + - note: review_mixed_keyed_rule_kinds_equal_set + modules: + - | + package test + values[k] contains 1 if { k := "a" } + values[k] := {1} if { k := "a" } + query: data.test.values + want_error: "conflicting rule kinds" + allow_interpreter_success: true + + - note: review_mixed_literal_dynamic_keys + modules: + - | + package test + values["a"] contains 1 + values[k] contains 2 if { k := "a" } + query: data.test.values + want_result: + a: + set!: [1, 2] + + - note: review_mixed_literal_dynamic_keys_suspendable + execution_mode: suspendable + modules: + - | + package test + values[k] contains 2 if { k := "a" } + values["a"] contains 1 + query: data.test.values + want_result: + a: + set!: [1, 2] + + - note: review_bodyless_head_key_loop + input: + keys: ["a", "b"] + modules: + - | + package test + values[input.keys[_]] contains 1 + query: data.test.values + want_result: + a: + set!: [1] + b: + set!: [1] + + - note: review_mixed_dot_dynamic_keys + modules: + - | + package test + values.a contains 1 + values[k] contains 2 if { k := "a" } + query: data.test.values + want_result: + a: + set!: [1, 2] + + - note: review_mixed_literal_dynamic_keys_across_modules + execution_mode: suspendable + modules: + - | + package test.nested + values["a"] contains 1 if { true } + - | + package test.nested + values[k] contains 2 if { k := "a" } + query: data.test.nested.values + want_result: + a: + set!: [1, 2] + + - note: review_bodyless_missing_head_key_collection + input: {} + modules: + - | + package test + values[input.keys[_]] contains 1 + query: data.test.values + want_result: {} + + - note: review_mixed_literal_dynamic_child_entrypoint + modules: + - | + package test + values["a"] contains 1 + values[k] contains 2 if { k := "b" } + query: data.test.values.a + want_result: + set!: [1] + + - note: review_empty_query_head_key_loop + input: + keys: ["a", "b"] + modules: + - | + package test + values[input.keys[_]] contains 1 if {} + query: data.test.values + want_result: + a: + set!: [1] + b: + set!: [1] + + - note: review_bodyless_head_key_loop_suspendable + execution_mode: suspendable + input: + keys: ["a", "b"] + modules: + - | + package test + values[input.keys[_]] contains 1 + query: data.test.values + want_result: + a: + set!: [1] + b: + set!: [1] + + - note: keyed_partial_set_accumulates_per_key + modules: + - | + package test + ports[host] contains port if { + some host, values in {"web": [80, 443, 80], "db": [5432]} + some port in values + } + query: data.test.ports + want_result: + web: + set!: [80, 443] + db: + set!: [5432] + + - note: keyed_partial_set_multiple_definitions + modules: + - | + package test + values[key] contains value if { + key := "a" + value := 1 + } + values[key] contains value if { + key := "a" + value := 2 + } + values[key] contains value if { + key := "b" + value := 1 + } + query: data.test.values + want_result: + a: + set!: [1, 2] + b: + set!: [1] + + - note: keyed_partial_set_preserves_defined_values + modules: + - | + package test + values[key] contains value if { + some key, value in {"false": false, "null": null, "zero": 0} + } + query: data.test.values + want_result: + "false": + set!: [false] + "null": + set!: [null] + zero: + set!: [0] + + - note: keyed_partial_set_undefined_key + input: {} + modules: + - | + package test + values[input.missing] contains 1 if { true } + query: data.test.values + want_result: {} + + - note: keyed_partial_set_undefined_value + input: {} + modules: + - | + package test + values[key] contains input.missing if { key := "a" } + query: data.test.values + want_result: {} + + - note: keyed_partial_set_empty_body + modules: + - | + package test + values["a"] contains 1 + values["a"] contains 2 + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: keyed_partial_set_unmatched_body + modules: + - | + package test + values[key] contains 1 if { + key := "a" + false + } + query: data.test.values + want_result: {} + + - note: keyed_partial_set_head_key_loop + input: + keys: ["a", "b", "a"] + modules: + - | + package test + values[input.keys[_]] contains 1 if { true } + query: data.test.values + want_result: + a: + set!: [1] + b: + set!: [1] + + - note: keyed_partial_set_static_prefix + modules: + - | + package test + values.fixed[key] contains value if { + some key, value in {"a": 1, "b": 2} + } + query: data.test.values.fixed + want_result: + a: + set!: [1] + b: + set!: [2] + + - note: keyed_partial_set_mixed_missing_values + input: + values: + a: 1 + b: false + modules: + - | + package test + values[key] contains input.values[key] if { + some key in ["a", "missing", "b"] + } + query: data.test.values + want_result: + a: + set!: [1] + b: + set!: [false] + + - note: keyed_partial_set_multiple_definitions_suspendable + execution_mode: suspendable + modules: + - | + package test + values[key] contains value if { + some key, value in {"a": 1, "b": 2} + } + values[key] contains 3 if { key := "a" } + query: data.test.values + want_result: + a: + set!: [1, 3] + b: + set!: [2] + + - note: keyed_partial_set_head_key_loop_suspendable + execution_mode: suspendable + input: + keys: ["a", "b", "a"] + modules: + - | + package test + values[input.keys[_]] contains 1 if { true } + query: data.test.values + want_result: + a: + set!: [1] + b: + set!: [1] + + - note: keyed_partial_set_mixed_missing_values_suspendable + execution_mode: suspendable + input: + values: + a: 1 + b: false + modules: + - | + package test + values[key] contains input.values[key] if { + some key in ["a", "missing", "b"] + } + query: data.test.values + want_result: + a: + set!: [1] + b: + set!: [false] + + - note: keyed_partial_set_undefined_value_suspendable + execution_mode: suspendable + input: {} + modules: + - | + package test + values[key] contains input.missing if { key := "a" } + query: data.test.values + want_result: {} + + - note: keyed_partial_set_unmatched_body_suspendable + execution_mode: suspendable + modules: + - | + package test + values[key] contains 1 if { + key := "a" + false + } + query: data.test.values + want_result: {} + + - note: keyed_partial_set_numeric_key + modules: + - | + package test + values[1] contains value if { some value in [false, null] } + query: data.test.values + want_result: + 1: + set!: [false, null] + + - note: keyed_partial_set_preserves_legacy_set_head + rego_v0: true + modules: + - | + package test + values[value] { value := [1, 2, 1][_] } + query: data.test.values + want_result: + set!: [1, 2] + + - note: keyed_partial_set_rejects_multiple_dynamic_key_levels + modules: + - | + package test + values[group][key] contains 1 if { + group := "g" + key := "k" + } + main := values + query: data.test.main + want_error: "partial object rules with nested bracket keys are not yet supported by the RVM compiler" + allow_interpreter_success: true + + - note: keyed_set_some_declared_key_before_dependency + modules: + - | + package test + values.a contains 1 + values[z] contains n if { some z; n := count(values.a); z = "b" } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_some_declared_key_before_dependency_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[z] contains n if { some z; n := count(values.a); z = "b" } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_some_declared_matching_key_still_recurses + modules: + - | + package test + values.a contains 1 + values[z] contains n if { some z; n := count(values.a); z = "a" } + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: keyed_set_some_declared_matching_key_still_recurses_suspendable + execution_mode: suspendable + modules: + - | + package test + values.a contains 1 + values[z] contains n if { some z; n := count(values.a); z = "a" } + query: data.test.values.a + want_error: "Runtime recursion detected" + + - note: keyed_set_inner_head_loop_needs_body_binding + input: {groups: [{keys: [a]}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[i][field][j]] contains 2 if { field := "keys" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: keyed_set_inner_head_loop_needs_body_binding_suspendable + execution_mode: suspendable + input: {groups: [{keys: [a]}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[i][field][j]] contains 2 if { field := "keys" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: keyed_set_inner_head_loop_body_local_array + input: {groups: [{x: a}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[i][arr[k]]] contains 2 if { arr := ["x", "y"] } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: keyed_set_inner_head_loop_body_local_array_suspendable + execution_mode: suspendable + input: {groups: [{x: a}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[i][arr[k]]] contains 2 if { arr := ["x", "y"] } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: keyed_set_deferred_inner_head_loop_filters_before_dependency + input: {groups: [{keys: [b]}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[i][field][j]] contains n if { field := "keys"; n := count(values.a) } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_deferred_inner_head_loop_filters_before_dependency_suspendable + execution_mode: suspendable + input: {groups: [{keys: [b]}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[i][field][j]] contains n if { field := "keys"; n := count(values.a) } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_deferred_inner_head_loop_matching_key + input: {groups: [{keys: [c]}, {keys: [b, a]}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[i][field][j]] contains n if { field := "keys"; n := 2 } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: keyed_set_deferred_inner_head_loop_matching_key_suspendable + execution_mode: suspendable + input: {groups: [{keys: [c]}, {keys: [b, a]}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[i][field][j]] contains n if { field := "keys"; n := 2 } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: keyed_set_deferred_head_loop_multiple_statements_after_binding + data: {} + input: {groups: [{keys: [b]}, {keys: [c]}, {keys: [a]}], other: [{keys: [b]}, {keys: [c]}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[i][field][j]] contains n if { field := "keys"; x := 1; y := x + 1; n := y + 1 } + values[input.other[i][field][j]] contains n if { field := "keys"; x := 1; y := x; n := count(values.a) + y } + query: data.test.values.a + want_result: + set!: [1, 3] + + - note: keyed_set_deferred_head_loop_multiple_statements_after_binding_suspendable + execution_mode: suspendable + data: {} + input: {groups: [{keys: [b]}, {keys: [c]}, {keys: [a]}], other: [{keys: [b]}, {keys: [c]}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[i][field][j]] contains n if { field := "keys"; x := 1; y := x + 1; n := y + 1 } + values[input.other[i][field][j]] contains n if { field := "keys"; x := 1; y := x; n := count(values.a) + y } + query: data.test.values.a + want_result: + set!: [1, 3] + + - note: keyed_set_deferred_inner_head_loop_wildcard_indices + data: {} + input: {groups: [{keys: [a]}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[_][field][_]] contains 2 if { field := "keys" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: keyed_set_deferred_inner_head_loop_wildcard_indices_suspendable + execution_mode: suspendable + data: {} + input: {groups: [{keys: [a]}]} + modules: + - | + package test + values.a contains 1 + values[input.groups[_][field][_]] contains 2 if { field := "keys" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: keyed_set_key_loop_hoisted_before_some_in + data: {} + input: {m: [a], xs: [2]} + modules: + - | + package test + values.a contains 0 + values[x[i]] contains v if { x := input.m; some v in input.xs } + query: data.test.values.a + want_result: + set!: [0, 2] + + - note: keyed_set_key_loop_hoisted_before_some_in_suspendable + execution_mode: suspendable + data: {} + input: {m: [a], xs: [2]} + modules: + - | + package test + values.a contains 0 + values[x[i]] contains v if { x := input.m; some v in input.xs } + query: data.test.values.a + want_result: + set!: [0, 2] + + - note: keyed_set_key_loop_hoisted_before_with_rejected + data: {} + input: {m: [a], ok: false} + modules: + - | + package test + p if input.ok + values.a contains 0 + values[x[i]] contains 1 if { x := input.m; not p with input.ok as true } + query: data.test.values.a + want_error: "the `with` keyword is not supported by the compiler yet" + allow_interpreter_success: true + + - note: keyed_set_key_loop_hoisted_before_with_rejected_suspendable + execution_mode: suspendable + data: {} + input: {m: [a], ok: false} + modules: + - | + package test + p if input.ok + values.a contains 0 + values[x[i]] contains 1 if { x := input.m; not p with input.ok as true } + query: data.test.values.a + want_error: "the `with` keyword is not supported by the compiler yet" + allow_interpreter_success: true + + - note: keyed_set_key_loop_hoisted_before_statement_loop + data: {} + input: {m: [a], xs: [2, 3]} + modules: + - | + package test + values.a contains 0 + values[x[i]] contains v if { x := input.m; v := input.xs[_] } + query: data.test.values.a + want_result: + set!: [0, 2, 3] + + - note: keyed_set_key_loop_hoisted_before_statement_loop_suspendable + execution_mode: suspendable + data: {} + input: {m: [a], xs: [2, 3]} + modules: + - | + package test + values.a contains 0 + values[x[i]] contains v if { x := input.m; v := input.xs[_] } + query: data.test.values.a + want_result: + set!: [0, 2, 3] + + - note: keyed_set_head_loop_recheck_after_body_backtrack + data: {} + input: {groups: [[b], [c]]} + modules: + - | + package test + values.a contains 1 + values[g[_]] contains n if { g := input.groups[_]; n := count(values.a) } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_head_loop_recheck_after_body_backtrack_suspendable + execution_mode: suspendable + data: {} + input: {groups: [[b], [c]]} + modules: + - | + package test + values.a contains 1 + values[g[_]] contains n if { g := input.groups[_]; n := count(values.a) } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_head_loop_recheck_after_some_in_backtrack + data: {} + input: {groups: [[a], [b], [c]]} + modules: + - | + package test + values.a contains 1 + values[g[_]] contains n if { some g in input.groups; g[0] != "a"; n := count(values.a) } + values[g[_]] contains 2 if { some g in input.groups; g[0] == "a" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: keyed_set_head_loop_recheck_after_some_in_backtrack_suspendable + execution_mode: suspendable + data: {} + input: {groups: [[a], [b], [c]]} + modules: + - | + package test + values.a contains 1 + values[g[_]] contains n if { some g in input.groups; g[0] != "a"; n := count(values.a) } + values[g[_]] contains 2 if { some g in input.groups; g[0] == "a" } + query: data.test.values.a + want_result: + set!: [1, 2] + + - note: keyed_set_input_alias_key_before_dependency + data: {} + input: {key: b} + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); z := input.key } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_input_alias_key_before_dependency_suspendable + execution_mode: suspendable + data: {} + input: {key: b} + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); z := input.key } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_input_alias_matching_key_still_recurses + data: {} + input: {key: a} + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); z := input.key } + query: data.test.values.a + want_error: "recursion detected" + + - note: keyed_set_input_alias_matching_key_still_recurses_suspendable + execution_mode: suspendable + data: {} + input: {key: a} + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); z := input.key } + query: data.test.values.a + want_error: "recursion detected" + + - note: keyed_set_nested_input_alias_unify_before_dependency + data: {} + input: {cfg: {keys: [b]}} + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); input.cfg["keys"][0] = z } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_nested_input_alias_unify_before_dependency_suspendable + execution_mode: suspendable + data: {} + input: {cfg: {keys: [b]}} + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); input.cfg["keys"][0] = z } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_undefined_input_alias_before_dependency + data: {} + input: {} + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); z := input.key } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_undefined_input_alias_before_dependency_suspendable + execution_mode: suspendable + data: {} + input: {} + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); z := input.key } + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_dynamic_input_alias_not_reordered + data: {} + input: {key: b, i: 0, keys: [b]} + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); z := input.keys[input.i] } + query: data.test.values.a + want_error: "recursion detected" + + - note: keyed_set_dynamic_input_alias_not_reordered_suspendable + execution_mode: suspendable + data: {} + input: {key: b, i: 0, keys: [b]} + modules: + - | + package test + values.a contains 1 + values[z] contains n if { n := count(values.a); z := input.keys[input.i] } + query: data.test.values.a + want_error: "recursion detected" + + - note: keyed_set_rule_key_checked_before_value_loops + data: {} + input: {} + modules: + - | + package test + target := "b" + values.a contains 1 + values[target] contains [count(values.a)][_] + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_rule_key_checked_before_value_loops_suspendable + execution_mode: suspendable + data: {} + input: {} + modules: + - | + package test + target := "b" + values.a contains 1 + values[target] contains [count(values.a)][_] + query: data.test.values.a + want_result: + set!: [1] + + - note: keyed_set_rule_key_matching_value_loop_self_read + data: {} + input: {} + modules: + - | + package test + target := "a" + values.a contains 1 + values[target] contains [count(values.a)][_] + query: data.test.values.a + want_error: "recursion detected" + + - note: keyed_set_rule_key_matching_value_loop_self_read_suspendable + execution_mode: suspendable + data: {} + input: {} + modules: + - | + package test + target := "a" + values.a contains 1 + values[target] contains [count(values.a)][_] + query: data.test.values.a + want_error: "recursion detected" + + - note: keyed_set_key_bound_by_value_loop + data: {} + input: {items: {a: 7, b: 8}} + modules: + - | + package test + values.a contains 0 + values[k] contains input.items[k] + query: data.test.values.a + want_result: + set!: [0, 7] + + - note: keyed_set_key_bound_by_value_loop_suspendable + execution_mode: suspendable + data: {} + input: {items: {a: 7, b: 8}} + modules: + - | + package test + values.a contains 0 + values[k] contains input.items[k] + query: data.test.values.a + want_result: + set!: [0, 7] + + - note: keyed_set_rule_key_reading_child_before_empty_value_loop + # Accepted conservative behavior: the key is checked before value loops, + # so a key rule that reads the projected child reports recursion even + # when the value collection is empty. + data: {} + input: {items: []} + modules: + - | + package test + values.a contains 1 + target := count(values.a) + values[target] contains input.items[_] + query: data.test.values.a + want_error: "recursion detected" + + - note: keyed_set_rule_key_reading_child_before_empty_value_loop_suspendable + execution_mode: suspendable + data: {} + input: {items: []} + modules: + - | + package test + values.a contains 1 + target := count(values.a) + values[target] contains input.items[_] + query: data.test.values.a + want_error: "recursion detected" diff --git a/tests/rvm/rego/keyed_projection_differential.rs b/tests/rvm/rego/keyed_projection_differential.rs new file mode 100644 index 000000000..57feb3fb3 --- /dev/null +++ b/tests/rvm/rego/keyed_projection_differential.rs @@ -0,0 +1,390 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Differential test for keyed partial-set projection. +//! +//! Querying `data.test.values.a` lets both engines skip rule bodies whose key +//! cannot be "a". That pruning must never change the answer, so for every +//! generated policy and input the projected result has to equal the "a" entry +//! of the unprojected `data.test.values`, on the interpreter and on the RVM in +//! both execution modes. + +use anyhow::{anyhow, bail, Result}; +use regorus::languages::rego::compiler::Compiler; +use regorus::rvm::vm::{ExecutionMode, RegoVM}; +use regorus::{Engine, Rc, Value}; + +const PROJECTED: &str = "data.test.values.a"; +const FULL: &str = "data.test.values"; + +// Head key expressions; each must be safe once the body binds `x`/`y`. +const KEYS: &[&str] = &["x", "input.key", "input.groups[_]", "y[_]", "x[0]"]; + +// Body statements; combined in order, so binding/use order varies. +const STMTS: &[&str] = &[ + "x := input.key", + "x := \"a\"", + "x := \"b\"", + "x := input.missing", + "some x in input.groups", + "some i; x := input.groups[i]", + "[x, _] := [input.key, 1]", + "x := [input.key]", + "y := input.groups", + "y := [g | some g in input.groups; g != \"b\"]", + "count(input.groups) > 0", + "not input.deny", + "every g in input.groups { g != \"z\" }", + "input.key == \"a\"", +]; + +const INPUTS: &[&str] = &[ + r#"{"key": "a", "groups": ["a", "b"]}"#, + r#"{"key": "b", "groups": ["b", "c"]}"#, + r#"{"key": "a", "groups": [], "deny": true}"#, + r#"{"groups": ["z", "a"]}"#, + r#"{}"#, +]; + +fn bodies() -> Vec> { + let mut bodies: Vec> = STMTS.iter().map(|s| vec![*s]).collect(); + for a in STMTS { + for b in STMTS { + if a != b { + bodies.push(vec![*a, *b]); + } + } + } + bodies +} + +// Rego v0 lets a body shadow `input`; projection must not filter on the +// global input before the shadow is bound. +const V0_SHADOWS: &[&str] = &[ + "input := {\"key\": \"a\", \"groups\": [\"a\"]}", + "some input in [{\"key\": \"a\", \"groups\": [\"c\", \"a\"]}]", +]; + +fn v0_bodies() -> Vec> { + let mut bodies = Vec::new(); + for shadow in V0_SHADOWS { + bodies.push(vec![*shadow]); + for stmt in STMTS { + bodies.push(vec![*shadow, *stmt]); + bodies.push(vec![*stmt, *shadow]); + } + } + bodies +} + +fn policy(key: &str, body: &[&str], v0: bool) -> String { + format!( + "package test\n{}\ + values.a contains 0\n\ + values[{key}] contains 1 if {{\n {}\n}}\n\ + values[{key}] contains 2 if {{\n {}\n input.key != \"b\"\n}}\n", + if v0 { "import future.keywords\n" } else { "" }, + body.join("\n "), + body.join("\n "), + ) +} + +fn engine(policy: &str) -> Result { + let mut engine = Engine::new(); + engine.set_rego_v0(policy.contains("import future.keywords")); + engine.add_policy("test.rego".to_string(), policy.to_string())?; + Ok(engine) +} + +fn rvm_eval(policy: &str, entry: &str, input: &Value, mode: ExecutionMode) -> Result { + let mut engine = engine(policy)?; + let compiled = engine.compile_with_entrypoint(&Rc::from(entry))?; + let program = Compiler::compile_from_policy(&compiled, &[entry]) + .map_err(|err| anyhow!("rvm compile {entry}\n{policy}{err}"))?; + let mut vm = RegoVM::new(); + vm.load_program(program); + vm.set_data(Value::new_object())?; + vm.set_execution_mode(mode); + vm.set_input(input.clone()); + vm.execute_entry_point_by_name(entry) + .map_err(|err| anyhow!("rvm {mode:?} {entry}\n{policy}input {input}\n{err}")) +} + +fn is_invalid_policy(err: &anyhow::Error) -> bool { + let err = err.to_string(); + ["unsafe", "used before definition", "Undefined variable"] + .iter() + .any(|msg| err.contains(msg)) +} + +fn check(key: &str, body: &[&str], v0: bool) -> Result { + check_policy(&policy(key, body, v0), INPUTS) +} + +fn check_policy(policy: &str, inputs: &[&str]) -> Result { + let policy = policy.to_string(); + let mut oracle = match engine(&policy) { + Ok(engine) => engine, + // Unsafe or otherwise invalid generated policy. + Err(_) => return Ok(0), + }; + let mut checked = 0; + let mut rvm_rejects_policy = false; + for input in inputs { + let input = Value::from_json_str(input)?; + oracle.set_input(input.clone()); + let expected = match oracle.eval_rule(FULL.to_string()) { + Ok(full) => full["a"].clone(), + // Generated policies that are statically invalid. + Err(err) if is_invalid_policy(&err) => return Ok(0), + Err(err) => return Err(err), + }; + let mut projected = engine(&policy)?; + projected.set_input(input.clone()); + let projected = projected.eval_rule(PROJECTED.to_string())?; + if projected != expected { + bail!( + "interpreter\n{policy}input {input}\nprojected {projected}\nunprojected {expected}" + ); + } + + // The RVM is compared with its own unprojected result so that + // pre-existing interpreter/RVM differences unrelated to projection + // do not mask projection bugs. + for mode in [ExecutionMode::RunToCompletion, ExecutionMode::Suspendable] { + if rvm_rejects_policy { + break; + } + // The interpreter can skip a statically invalid head when the + // body fails first; the RVM compiler rejects it up front. Only + // that compile-time rejection is tolerated, and only for the RVM. + let expected = match rvm_eval(&policy, FULL, &input, mode) { + Ok(full) => full["a"].clone(), + Err(err) + if is_invalid_policy(&err) && err.to_string().starts_with("rvm compile") => + { + rvm_rejects_policy = true; + break; + } + Err(err) => return Err(err), + }; + let projected = rvm_eval(&policy, PROJECTED, &input, mode)?; + if projected != expected { + bail!("rvm {mode:?}\n{policy}input {input}\nprojected {projected}\nunprojected {expected}"); + } + } + checked += 1; + } + Ok(checked) +} + +#[test] +fn keyed_projection_matches_unprojected_evaluation() -> Result<()> { + let mut checked = 0; + let mut failures = Vec::new(); + for key in KEYS { + let cases = bodies() + .into_iter() + .map(|body| (body, false)) + .chain(v0_bodies().into_iter().map(|body| (body, true))); + for (body, v0) in cases { + match check(key, &body, v0) { + Ok(count) => checked += count, + Err(err) => failures.push(format!("{err:#}")), + } + } + } + if !failures.is_empty() { + bail!( + "{} of the generated cases diverged:\n{}", + failures.len(), + failures.join("\n---\n") + ); + } + // Guard against the generator silently producing only invalid policies. + anyhow::ensure!(checked > 1000, "only {checked} cases checked"); + Ok(()) +} + +// Head keys bound only by head value loops: the key cannot be checked before +// the value loop that binds it. +const VALUE_LOOP_RULES: &[&str] = &[ + "values[k] contains input.items[k]", + "values[k] contains input.items[k][_]", + "values[k] contains input.items[k] if { input.items }", + "values[input.keys[i]] contains input.vals[i]", + "values[k] contains [k, v][_] if { some v in input.vals; k := input.keys[_] }", + "values[k] contains input.items[k] if { k := input.key }", +]; + +const VALUE_LOOP_INPUTS: &[&str] = &[ + r#"{"items": {"a": 7, "b": 8}, "keys": ["b", "a"], "vals": [1, 2], "key": "a"}"#, + r#"{"items": {"a": [7, 9], "b": []}, "keys": ["a"], "vals": [], "key": "b"}"#, + r#"{"items": {}, "keys": [], "vals": [3]}"#, + r#"{}"#, +]; + +#[test] +fn keyed_projection_value_loop_bound_keys() -> Result<()> { + let mut checked = 0; + let mut failures = Vec::new(); + for rule in VALUE_LOOP_RULES { + for shadow in ["", "k := \"b\"\n"] { + let policy = format!("package test\n{shadow}values.a contains 0\n{rule}\n"); + match check_policy(&policy, VALUE_LOOP_INPUTS) { + Ok(count) => checked += count, + Err(err) => failures.push(format!("{err:#}")), + } + } + } + if !failures.is_empty() { + bail!( + "{} cases diverged:\n{}", + failures.len(), + failures.join("\n---\n") + ); + } + anyhow::ensure!(checked >= 30, "only {checked} cases checked"); + Ok(()) +} + +// Self-referencing contributions: a rule whose head value or body reads +// `values.a` must be rejected by key before that read whenever its key is not +// "a", and must report recursion when it is. Unprojected evaluation always +// recurses here, so the oracle is a variant with the self-reference replaced +// by a constant and the contribution replaced by a marker. +const SELF_KEYS: &[&str] = &["target", "data.test.target", "targets[_]", "input.key", "x"]; +const SELF_AUX: &[(&str, &str)] = &[("\"b\"", "[\"b\", \"c\"]"), ("\"a\"", "[\"c\", \"a\"]")]; +// (projected value, oracle value, body). A body that reads `values.a` before +// a key derived from another rule is known may conservatively report +// recursion: early filtering never evaluates other rules before the body. +const SELF_SHAPES: &[(&str, &str, Option<&str>)] = &[ + ("[count(values.a)][_]", "[\"m\"][_]", None), + ("count(values.a)", "\"m\"", None), + ("[n, count(values.a)][_]", "[\"m\"][_]", Some("n := 1")), + ("n", "\"m\"", Some("n := count(values.a)")), + ("[count(values.a)][_]", "[\"m\"][_]", Some("x := input.key")), + ( + "[count(values.a)][_]", + "[\"m\"][_]", + Some("some x in input.groups"), + ), + ("count(values.a)", "\"m\"", Some("x := target")), + ( + "[g | some g in input.groups; count(values.a) > 0][_]", + "[\"m\" | some g in input.groups][_]", + Some("x := input.key"), + ), +]; + +fn self_policy(key: &str, value: &str, body: Option<&str>, aux: (&str, &str)) -> String { + let rule = match body { + Some(body) => format!("values[{key}] contains {value} if {{\n {body}\n}}"), + None => format!("values[{key}] contains {value}"), + }; + format!( + "package test\ntarget := {}\ntargets := {}\nvalues.a contains 0\n{rule}\n", + aux.0, aux.1 + ) +} + +fn check_self(key: &str, shape: &(&str, &str, Option<&str>), aux: (&str, &str)) -> Result { + let (value, oracle_value, body) = *shape; + let oracle_body = body.map(|body| body.replace("count(values.a)", "1")); + let policy = self_policy(key, value, body, aux); + let oracle_policy = self_policy(key, oracle_value, oracle_body.as_deref(), aux); + if engine(&oracle_policy).is_err() { + return Ok(0); + } + let body_reads_self = + key.contains("target") && body.is_some_and(|body| body.contains("values.a")); + let marker = Value::from("m"); + let verdict = |oracle: Value, projected: Result, label: &str, input: &Value| { + let recursive = oracle.as_set().is_ok_and(|set| set.contains(&marker)); + let expected = if recursive { + Value::Undefined + } else { + oracle.clone() + }; + match projected { + Err(err) + if (recursive || body_reads_self) + && err.to_string().contains("recursion detected") => + { + Ok(()) + } + Ok(projected) if !recursive && projected == expected => Ok(()), + other => Err(anyhow!( + "{label}\n{policy}input {input}\nprojected {}\nexpected {}", + match other { + Ok(value) => value.to_string(), + Err(err) => format!("error {err}"), + }, + if recursive { + "recursion error".to_string() + } else { + expected.to_string() + } + )), + } + }; + let mut checked = 0; + for input in INPUTS { + let input = Value::from_json_str(input)?; + let mut oracle = engine(&oracle_policy)?; + oracle.set_input(input.clone()); + let expected = match oracle.eval_rule(FULL.to_string()) { + Ok(full) => full["a"].clone(), + Err(err) if is_invalid_policy(&err) => return Ok(checked), + Err(err) => return Err(err), + }; + let mut projected = engine(&policy)?; + projected.set_input(input.clone()); + verdict( + expected, + projected.eval_rule(PROJECTED.to_string()), + "interpreter", + &input, + )?; + for mode in [ExecutionMode::RunToCompletion, ExecutionMode::Suspendable] { + let expected = match rvm_eval(&oracle_policy, FULL, &input, mode) { + Ok(full) => full["a"].clone(), + Err(err) if is_invalid_policy(&err) => return Ok(checked), + Err(err) => return Err(err), + }; + verdict( + expected, + rvm_eval(&policy, PROJECTED, &input, mode), + &format!("rvm {mode:?}"), + &input, + )?; + } + checked += 1; + } + Ok(checked) +} + +#[test] +fn keyed_projection_rejects_self_reads_by_key() -> Result<()> { + let mut checked = 0; + let mut failures = Vec::new(); + for key in SELF_KEYS { + for shape in SELF_SHAPES { + for aux in SELF_AUX { + match check_self(key, shape, *aux) { + Ok(count) => checked += count, + Err(err) => failures.push(format!("{err:#}")), + } + } + } + } + if !failures.is_empty() { + bail!( + "{} of the generated cases diverged:\n{}", + failures.len(), + failures.join("\n---\n") + ); + } + anyhow::ensure!(checked > 200, "only {checked} cases checked"); + Ok(()) +} diff --git a/tests/rvm/rego/mod.rs b/tests/rvm/rego/mod.rs index c47c6eef2..286d87337 100644 --- a/tests/rvm/rego/mod.rs +++ b/tests/rvm/rego/mod.rs @@ -17,6 +17,8 @@ use std::fs; use std::sync::Arc; use test_generator::test_resources; +mod keyed_projection_differential; + #[derive(Serialize, Deserialize, PartialEq, Debug)] struct TestCase { pub data: Option, diff --git a/tests/rvm/vm/suites/keyed_set_operations.yaml b/tests/rvm/vm/suites/keyed_set_operations.yaml new file mode 100644 index 000000000..7b4d79792 --- /dev/null +++ b/tests/rvm/vm/suites/keyed_set_operations.yaml @@ -0,0 +1,104 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +cases: + - note: keyed_set_accumulates_and_deduplicates + literals: + - {} + - "key" + instructions: + - "Load { dest: 0, literal_idx: 0 }" + - "Load { dest: 1, literal_idx: 1 }" + - "LoadFalse { dest: 2 }" + - "ObjectSetAdd { obj: 0, key: 1, value: 2 }" + - "ObjectSetAdd { obj: 0, key: 1, value: 2 }" + - "LoadNull { dest: 2 }" + - "ObjectSetAdd { obj: 0, key: 1, value: 2 }" + - "Return { value: 0 }" + want_result: + key: + set!: [false, null] + + - note: keyed_set_skips_undefined_key_and_value + literals: + - {} + - "key" + - "#undefined" + instructions: + - "Load { dest: 0, literal_idx: 0 }" + - "Load { dest: 1, literal_idx: 1 }" + - "Load { dest: 2, literal_idx: 2 }" + - "ObjectSetAdd { obj: 0, key: 1, value: 2 }" + - "ObjectSetAdd { obj: 0, key: 2, value: 1 }" + - "Return { value: 0 }" + want_result: {} + + - note: keyed_set_preserves_shared_object_and_set + literals: + - {} + - "key" + instructions: + - "Load { dest: 0, literal_idx: 0 }" + - "Load { dest: 1, literal_idx: 1 }" + - "LoadFalse { dest: 2 }" + - "ObjectSetAdd { obj: 0, key: 1, value: 2 }" + - "Move { dest: 3, src: 0 }" + - "LoadNull { dest: 2 }" + - "ObjectSetAdd { obj: 0, key: 1, value: 2 }" + - "Return { value: 3 }" + want_result: + key: + set!: [false] + + - note: keyed_set_rejects_non_object_even_for_undefined + literals: + - "#undefined" + instructions: + - "LoadNull { dest: 0 }" + - "Load { dest: 1, literal_idx: 0 }" + - "ObjectSetAdd { obj: 0, key: 1, value: 1 }" + - "Return { value: 0 }" + want_error: "Register 0 does not contain an object" + + - note: keyed_set_rejects_non_set_leaf + literals: + - {"key": 1} + - "key" + instructions: + - "Load { dest: 0, literal_idx: 0 }" + - "Load { dest: 1, literal_idx: 1 }" + - "LoadNull { dest: 2 }" + - "ObjectSetAdd { obj: 0, key: 1, value: 2 }" + - "Return { value: 0 }" + want_error: "Register 0 does not contain a set" + + - note: keyed_set_rejects_non_set_leaf_even_for_undefined_value + literals: + - {"key": 1} + - "key" + - "#undefined" + instructions: + - "Load { dest: 0, literal_idx: 0 }" + - "Load { dest: 1, literal_idx: 1 }" + - "Load { dest: 2, literal_idx: 2 }" + - "ObjectSetAdd { obj: 0, key: 1, value: 2 }" + - "Return { value: 0 }" + want_error: "Register 0 does not contain a set" + + - note: keyed_set_undefined_value_preserves_existing_set + literals: + - {} + - "key" + - "#undefined" + instructions: + - "Load { dest: 0, literal_idx: 0 }" + - "Load { dest: 1, literal_idx: 1 }" + - "LoadNull { dest: 2 }" + - "ObjectSetAdd { obj: 0, key: 1, value: 2 }" + - "Move { dest: 3, src: 0 }" + - "Load { dest: 2, literal_idx: 2 }" + - "ObjectSetAdd { obj: 0, key: 1, value: 2 }" + - "Return { value: 0 }" + want_result: + key: + set!: [null]