Skip to content

Latest commit

 

History

History
515 lines (328 loc) · 31.2 KB

File metadata and controls

515 lines (328 loc) · 31.2 KB

Optimizer

The optimizer is an optional AST transformation pass that runs between parsing and execution. It rewrites the parsed Block tree in place — producing a semantically equivalent but simpler tree — so that the interpreter does less work at runtime.

Implementation: Sources/LuaInterpreterParser/Optimizer.swift in the internal LuaInterpreterParser target (no separate library product). Both execution backends import Parser and call Optimizer.optimize(_:) when optimizationEnabled is true.


Table of Contents

  1. Where It Fits in the Pipeline
  2. Enabling and Disabling the Optimizer
  3. Pass 0 — Constant Propagation
  4. Pass 1 — Constant Folding
  5. Pass 2 — Dead Code Elimination
  6. Pass 3 — Unused Local Elimination
  7. Why Only These Optimizations
  8. Type-driven optimizations
  9. Correctness Constraints
  10. Design Decisions and Edge Cases

1. Where It Fits in the Pipeline

Source String
      │
      ▼
  ┌────────┐
  │  Lexer │
  └────────┘
      │  [Token]
      ▼
  ┌────────┐
  │ Parser │
  └────────┘
      │  Block (AST)
      ▼
  ┌───────────┐   (when optimizationEnabled == true)
  │ Optimizer │  LuaInterpreterParser/Optimizer.swift
  └───────────┘
      │  Block (transformed AST)
      ├──────────────────────────┐
      ▼                          ▼
  ┌─────────────┐      ┌──────────────────┐
  │ Interpreter │      │ Bytecode VM      │
  │ (tree-walk) │      │ (compile + run)  │
  └─────────────┘      └──────────────────┘
      │  [LuaValue]
      ▼
     Output

The optimizer is a pure function — Optimizer.optimize(_ block: Block) -> Block — with no side effects and no runtime state. It cannot fail: if a transformation cannot be applied safely, the original subtree is returned unchanged. The interpreter receives whatever the optimizer produces and executes it normally.

The three passes run in sequence:

propagateConstants → constantFold → eliminateUnusedLocals

Constant propagation runs first so that the folding pass sees as many literal operands as possible. Unused-local elimination runs last because propagation may cause previously-read locals to become unread (their reads replaced by literals), making them newly eligible for removal.


2. Enabling and Disabling the Optimizer

Swift API

// Optimizer enabled (default)
let interpreter = Lua.createInterpreter()
let interpreter = Lua.createInterpreter(optimizationEnabled: true)

// Optimizer disabled
let interpreter = Lua.createInterpreter(optimizationEnabled: false)

// Apply the optimizer manually to a parsed block
let optimized = Lua.optimize(block)

Lua.run(_:) always uses an optimizer-enabled interpreter. For fine-grained control, use Lua.createInterpreter(optimizationEnabled:).

REPL command line

LuaREPL [--optimize | --no-optimize] [script]

--optimize is the default. Passing --no-optimize runs the script without the optimization pass; this is useful for debugging or when comparing output between the two modes.


3. Pass 0 — Constant Propagation

Entry point: Optimizer.propagateConstants(block:)

Constant propagation replaces reads of local variables whose initial value is a known literal with that literal directly. This happens before constant folding so that expressions containing those locals can be folded in the next pass.

local PI = 3.14159
local r  = 5
local area = PI * r * r   -- becomes 3.14159 * 5 * 5 after propagation
                           -- then folds to 78.53975 in the next pass

Eligibility: the mutation scan

Before any substitution, a full traversal of the block tree (including nested function bodies) builds a Set<String> of every name that appears as an assignment target:

  • Plain identifier on the left-hand side of an assignment (x = ..., x, y = ...)
  • Numeric for loop variable (for x = ...)
  • Generic for loop variables (for x, y in ...)

A local is eligible for propagation only if its name is absent from this set. The scan is intentionally conservative: it recurses into nested function bodies, so if any same-named variable is assigned anywhere in the entire block tree — even inside a closure that shadows it with a local of its own — the outer local is not propagated. This avoids the need for full closure analysis.

Propagation walk

Statements in a block are processed left-to-right, accumulating a [String: Expression] constants map:

  1. When local x = <literal> is encountered and x is not in the mutated set, x → literal is added to the map.
  2. All subsequent reads of x in expression positions are replaced with the literal.
  3. If a later local x = <anything> is encountered (whether or not the new value is a literal), x is removed from the map for the remainder of that scope — the new declaration shadows the outer constant.

Scope handling

Nested blocks (do, if bodies, while, for, repeat) receive a copy of the current constants map. New constants added inside a nested block do not propagate back to the outer scope.

Function bodies are handled separately:

  • A fresh mutation set is computed for each function body via collectMutations(block: fb.body). This means inner locals are analysed independently of outer scope mutations, so a local declared and used only inside a function is propagatable even if a same-named variable is assigned at the outer level.
  • Parameter names are removed from the inherited outer constants map before entering the body, since parameters shadow outer locals inside the function.

What propagation enables

After propagation, reads of literal-valued locals become literals. The constant-folding pass that follows can then fold arithmetic on those literals:

-- Before propagation:  y = x * 2  (x is an identifier — can't fold)
-- After propagation:   y = 5 * 2  (x replaced by 5 — folds to 10)
-- After folding:       y = 10

Additionally, once all reads of a local have been replaced by its literal value, the local no longer appears in the read set used by the unused-local elimination pass, so it is automatically removed:

local x = 42
print(x)
-- After propagation: print(42)   — x no longer appears in any read
-- After elimination: local x = 42 is dropped (unused)
-- Final:             print(42)

4. Pass 1 — Constant Folding

Entry point: Optimizer.constantFold(block:)

Constant folding evaluates expressions at compile time (during the optimization pass) rather than at runtime. When both operands of an arithmetic, bitwise, relational, or string expression are literals, the result is computed immediately and the expression is replaced with the resulting literal.

Arithmetic

Operator Behavior
+, -, * Integer: wrapping (&+, &-, &*). Float: IEEE 754.
/ Always produces .float, even for 4 / 2 → 2.0.
// Integer: Lua floor-division (rounds toward −∞). Returns nil (no fold) if divisor is zero, to preserve the runtime error. Float: (a/b).rounded(.down).
% Integer: Lua modulo (a - floor(a/b)*b, same sign as divisor). Returns nil if divisor is zero. Float: truncatingRemainder.
^ Always produces .float via pow(Double, Double).
Bitwise &, |, ~, <<, >> Integer only; shifts by ≥64 produce 0, negative counts reverse direction. Bitwise operations on mixed int/float are not folded.

Wrapping arithmetic (&+ etc.) is intentional: Lua integers are 64-bit two's-complement with defined wraparound, so math.maxinteger + 1 == math.mininteger is required to fold correctly.

Concatenation

.. is handled before the numeric path. A literal on either side is coerced to its Lua string representation:

  • .integer(v) → "\(v)" (decimal, no decoration)
  • .float(v) → formatLuaFloat(v) (matches tostring output, e.g. 1.0, 1.5e+20)
  • .string(s) → s (identity)

1 .. 2 folds to "12", and 1.0 .. "x" folds to "1.0x".

Relational and equality

For two integers, two floats, or two strings, all six comparisons (<, <=, >, >=, ==, ~=) fold to .true or .false.

Cross-type comparisons are not folded because Lua raises an error for integer < string and some metamethods may override comparison for tables.

Unary operators

Operator Fold condition
- (negate) Operand is .integer or .float
not Operand is definitively falsy (nil, false) or truthy (any other literal)
# Operand is .string — folds to byte count
~ (bitwise not) Operand is .integer

Short-circuit and / or

and and or are not simple arithmetic — they return one of their operands, not a boolean — so they need special handling:

Expression Fold condition
false and E Fold to false (right side not evaluated)
nil and E Fold to nil
true and E Fold to E (left side discarded)
true or E Fold to true
false or E Fold to E

This is correct because Lua's and returns the left operand if it is falsy, otherwise the right; or returns the left if truthy, otherwise the right.

isDefinitelyFalsy / isDefinitelyTruthy

The optimizer uses conservative predicates:

  • isDefinitelyFalsy: only .nil and .false
  • isDefinitelyTruthy: .integer, .float, .string, .true

Identifiers, table constructors, and function calls are never classified because their runtime value is unknown at optimization time.


5. Pass 2 — Dead Code Elimination

Dead code elimination is part of the constant-folding pass (foldStatements, foldIfStatement).

Dead code after break

Within a statement list, any statement after a break is unreachable. Those statements are dropped — except for label statements, which are kept because they may be the target of a goto from elsewhere in the enclosing block.

for i = 1, 10 do
    break
    print("never")   -- dropped
    ::continue::      -- kept (reachable via goto from outside the break)
end

goto is intentionally not treated as a dead-code terminator. The label that goto targets typically appears after the goto in the same statement list, and removing it would break the jump. See §10 for the full discussion.

Constant-condition branches

while false do … end — the entire loop is replaced with .empty. A while true do … end condition is not eliminated (the loop may still break).

if / elseif / else — each branch condition is folded first:

  • If the condition is definitively truthy: the branch body is kept (wrapped in a do…end block) and remaining branches are discarded.
  • If the condition is definitively falsy: that branch is skipped and the next elseif clause is tried with the same logic.
  • The first branch that is not definitely false or definitely true becomes the new top-level condition.
  • If all branches are eliminated and there is an else body, the else is kept; otherwise the whole statement becomes .empty.

6. Pass 3 — Unused Local Elimination

Entry point: Optimizer.eliminateUnusedLocals(block:)

This pass removes local declarations whose declared names are never read in any reachable position in the entire block tree.

Algorithm

Step 1 — Collect reads. A single traversal over the entire block tree builds a Set<String> of all identifier names that appear in a read (non-write) position. Write-only positions are:

  • The left-hand side of an assignment where the target is a plain .identifier. Only plain identifiers are write-only; compound targets like t.x or t[k] read their base (t), so they are still counted as reads of t.
  • The name list of a local declaration itself (the names being introduced are not reads).

Step 2 — Remove bindings. For each local declaration, if all declared names are absent from the read set, the declaration is a candidate for removal. The names list is checked as a whole (all-or-nothing) because a multi-assignment like local x, y = f() is a single call — dropping the binding entirely is only safe if no name is ever used.

When a binding is dropped:

  • Each RHS expression is inspected with isPure.
  • Pure expressions are dropped entirely (they have no observable effect).
  • Impure expressions (function calls, method calls, field/index accesses) are kept as .expressionStatement nodes so their side effects still occur.

isPure

isPure takes the set of names currently in scope as locals. Reading a local is always pure; reading a global goes through _ENV and can trigger __index, making it impure.

Expression Pure?
Literals (.nil, .true, .false, .integer, .float, .string) Yes
.identifier(name) where name is in scope as a local Yes
.identifier(name) where name is a global No — global reads can trigger __index on _ENV
.functionDef Yes — capturing the environment has no side effects
.unary(op, operand) where isPure(operand) Yes — if operand is a literal, no metamethod fires
.binary(op, l, r) where both sides are pure Yes
.tableConstructor with all pure fields Yes
.functionCall, .methodCall No — may have side effects
.field, .index No — may trigger __index metamethod

Being conservative here prevents incorrect elimination of calls like local _ = os.time() where the call has observable effects.

Scope handling

eliminateUnusedLocals re-enters nested scopes (function bodies, loop bodies, block bodies) independently. Each nested scope is re-optimized with a fresh read collection from that scope's subtree.

This is intentionally coarse: a name read in an inner scope causes the binding in an outer scope to be considered "used." This is conservative and correct. It means a local whose only read is inside a dead branch may still be kept; that is acceptable.

The locals set passed to isPure is accumulated as each statement in a block is processed: a localDeclaration or localFunction statement adds its names to the set for all subsequent statements in the same block.


7. Why Only These Optimizations

This section explains why four specific optimizations were selected and why many other standard optimizations were deliberately excluded.

Why constant propagation was selected

Constant propagation unlocks folding opportunities that constant folding alone cannot reach. A common Lua pattern is to define named constants at the top of a file or function:

local MAX_RETRIES = 3
local BASE_URL    = "https://api.example.com"
local TIMEOUT_MS  = 30 * 1000   -- after folding: 30000

Without propagation, any expression that reads MAX_RETRIES, BASE_URL, or TIMEOUT_MS stays as an identifier reference at runtime. With propagation, those reads become literals immediately, and subsequent arithmetic on them folds away entirely.

The transformation is sound for locals that are never reassigned. The mutation scan ensures that any local with a write anywhere in scope is excluded from propagation.

Why constant folding was selected

Lua code frequently contains numeric constants — loop bounds, bit masks, conversion factors, table size hints — that are written as expressions for readability:

local TIMEOUT = 60 * 60 * 24    -- seconds in a day
local MASK    = 0xFF & 0x0F
local PREFIX  = "foo" .. "/"

Constant folding collapses these at parse time so the interpreter sees a single literal at runtime. The benefit is real and the transformation is unambiguously correct: a literal expression has no side effects and no dependence on runtime state, so replacing it with its computed value cannot change program behavior.

It also enables dead code elimination: classifying conditions as definitely-true/false requires folding if (1 > 2) to if false first.

Why dead code elimination was selected

Dead code elimination is a direct companion to constant folding. Once an if false branch is identified, there is no reason to execute it. Keeping unreachable code wastes interpreter cycles and, in a tight loop, can noticeably affect performance.

The break-based dead code elimination is similarly low-risk: any statement between break and the end of a block is literally unreachable by the Lua control-flow rules.

Why unused local elimination was selected

Unused locals appear commonly in real code:

  • Temporary debug variables left behind during development
  • Multi-return calls where only some values are needed: local _, err = pcall(f)
  • Explicit discard patterns: local _unused = heavy_computation()

Removing the binding means the interpreter never allocates the local in the environment, never stores the value, and never touches it during GC. For the common case where the RHS is a literal or a simple expression, it also avoids pushing and popping the value at runtime.

After constant propagation replaces all reads of a local with its literal value, the local automatically becomes unused and is eliminated by this pass without any additional effort.

What was deliberately excluded

Function inlining

Inlining replaces a call site with a copy of the called function's body. In a tree-walking interpreter, inlining is unattractive for several reasons:

  1. Lua functions are first-class values. A variable f might hold a different function at each call site, or might be replaced between calls. Inlining is only sound when the callee is statically known, which requires control-flow analysis that goes well beyond simple tree rewriting.

  2. Closures capture mutable state. An inlined copy of a closure body would need its own copy of all captured upvalues, which changes the sharing semantics. Two closures that share an upvalue (local x; local f = function() x = x+1 end) would diverge after inlining.

  3. Recursive functions cannot be inlined without a bound on depth. Detecting mutual or self-recursion and choosing a safe inline depth is non-trivial.

  4. The performance model is wrong for a tree-walker. Bytecode VMs benefit from inlining because the overhead of a function call includes bytecode dispatch and stack frame setup. In a tree-walking interpreter the overhead is Swift function calls, which the Swift compiler already optimizes. The marginal gain from inlining Lua functions is small.

Loop unrolling

Unrolling replicates a loop body N times to reduce branch overhead. For a tree-walking interpreter the overhead of the loop condition check is a single Swift comparison, making unrolling essentially a wash. It also dramatically increases the AST size and has no benefit when the trip count is not statically known (the common case).

Common subexpression elimination (CSE)

CSE identifies expressions computed multiple times and replaces repeated computations with a reference to the first result. It requires:

  • A value numbering pass to identify equivalent subexpressions.
  • An alias analysis to verify that no mutation between two uses invalidates the cached result.
  • Introduction of synthetic temporaries, which in Lua means new local bindings.

In Lua this is particularly hard because almost any expression can trigger a metamethod, and metamethods have arbitrary side effects. a + b computed twice might legitimately return different values if __add is stateful. Ruling this out statically requires knowing the types of a and b, which requires type inference the interpreter does not perform.

Tail call optimization (TCO)

Lua 5.5 specifies proper tail calls — a tail-position call must not grow the call stack. This is a semantic requirement, not merely a performance optimization. It is already handled (or to be handled) at the interpreter level, not by AST rewriting. An AST optimizer cannot synthesize tail calls because the tail-call property is a property of the interpreter's call stack discipline, not of the AST.

Escape analysis / stack allocation of tables

Tables that do not escape the function that creates them could in principle be allocated on the stack. This requires a full escape analysis over the object graph, which is substantially more complex than what a single-pass AST rewriter can do. It would also require changes to the runtime value representation (LuaValue and LuaTable), not just the AST.

Full JIT-style type specialization

Generating different code paths for every operator combination based on a whole-program type analysis is the core technique used by LuaJIT. The interpreter now performs limited type-driven specialization (see Type-driven optimizations below) but does not JIT or restructure the entire program.

Strength reduction

Replacing x * 2 with x + x or x * 4 with x << 2 yields a small constant-factor improvement. For a tree-walker the arithmetic is Swift integer arithmetic regardless, and the hardware will perform the reduction anyway. The benefit is negligible compared to the complexity of recognizing patterns correctly (especially for the case where x could be a float or a table with __mul).

Summary of the selection criteria

The four implemented optimizations share a set of properties that distinguish them from the rejected ones:

Property Const propagation Const folding Dead code Unused locals
Can be done by pure AST rewrite Yes Yes Yes Yes
Requires no type inference Yes Yes* Yes Yes
Requires no alias analysis Yes Yes Yes Yes
Cannot change semantics under any metamethod Yes Yes Yes Yes
Benefit visible even in a tree-walking interpreter Yes Yes Yes Yes
Complexity proportionate to benefit Yes Yes Yes Yes

An optimization that fails any of these criteria was excluded. The goal was a pass that is simple enough to be provably correct, not a pass that squeezes the last drop of performance from the tree-walker.

* Constant folding now uses a partial type map (TypeInference in TypeInference.swift) for branch DCE on propagated booleans and for eliding redundant runtime checks; it does not perform whole-program inference.


8. Type-driven optimizations

When sources use optional type annotations (local x <int>, function f() <int>, etc.) or strict typing mode (inferred types without annotations), three layers cooperate:

Layer File What it does
Shared inference TypeInference.swift TypeEnvironment, staticType, collectMutations — used by TypeChecker, Optimizer, and Compiler
AST optimizer Optimizer.swift Registers types from annotations or strict inference (TypeResolution.*Unchecked); seeds TypeEnvironment from the type-check pass; folds typed integer/float operands when values are literals (after propagation)
Bytecode Compiler.swift, BytecodeInterpreter.swift Skips checkType when needsRuntimeTypeCheck is false; emits addInt / subInt / mulInt when both operands are statically integer
Tree-walker Interpreter.swift, Environment.swift Skips coerce on assign/init when static types are already compatible

Optimizer.optimize(_:typingMode:seedTypeEnv:) and Interpreter.execute pass the session typingMode and post-check typeEnv so REPL chunks and strict-inferred locals (local i = 5) get the same folding benefits as explicit <int> annotations.

Safety rule: optimizations apply only to scalar types (integer, float, boolean, string, number) with known static sources. Operands typed any, or reads of table / function / field access, keep the generic metamethod-capable paths. In strict mode, parameters and for-loop variables receive concrete types from annotations, body-usage inference (parameters: typed locals, assignments, annotated returns), or static inference (numeric-for integer bounds, pairs/ipairs shapes); the optimizer registers those types the same way as annotated locals.

Call Lua.parse (or TypeChecker.check) before Lua.optimize on typed code so types are validated first.


9. Correctness Constraints

The optimizer is required to preserve all observable behaviors of a Lua program. The constraints enforced are:

  1. Only never-reassigned locals are propagated. The mutation scan collects every name that appears as an assignment target anywhere in the block tree. A local is propagated only if its name is absent from this set.

  2. Inner shadowing removes the outer constant from scope. When a local x = ... declaration is encountered inside a nested block, x is removed from the constants map for the remainder of that scope, regardless of whether the new value is a literal. This ensures that reads of the inner x see the inner value, not the outer literal.

  3. Function parameters shadow outer constants. Before propagating into a function body, parameter names are removed from the inherited constants map. A parameter named x always refers to the argument, never to an outer local x = 5.

  4. Side effects of RHS expressions in unused locals are preserved. When a local declaration is dropped, each impure RHS is re-emitted as an expression statement.

  5. Labels reachable by goto are never removed. Dead-code elimination after break scans the dropped statements and re-inserts any .label nodes. goto is never treated as a terminator.

  6. Integer arithmetic uses wrapping semantics. Swift's +, -, * trap on overflow; the optimizer uses &+, &-, &* to match Lua's two's-complement wraparound.

  7. Division by zero is not folded. // and % with a literal zero denominator are left unfolded so the runtime error fires at the correct point.

  8. / always produces float. Even 4 / 2 folds to .float(2.0), not .integer(2).

  9. ^ always produces float. Even 2 ^ 3 folds to .float(8.0).

  10. Float concatenation uses Lua formatting. formatLuaFloat is used, not Swift's default Double description, to match the tostring output that the runtime would produce.

  11. while true is not eliminated. Only while false (definitively falsy condition) is removed; while true might be the intended infinite loop with an internal break.


10. Design Decisions and Edge Cases

Constant propagation mutation scan is flat and conservative

The mutation scan collects assignment targets from the entire block tree in a single flat set, including inside nested function bodies. This is conservative: if a closure assigns to a variable named x, the outer local x = 5 is not propagated even if the closure's x is a local of its own (shadowing the outer one).

The alternative — tracking scope during the mutation scan to distinguish upvalue mutations from local mutations — would require a full binding analysis pass. For the current implementation, the conservative approach is correct at the cost of missing some propagation opportunities when names are reused across nested functions.

Each function body receives a fresh mutation set computed only from that body's own statements. This means inner locals propagate independently even if a same-named variable is assigned at the outer level.

Propagation does not cross function-body boundaries into outer scope

The constants map passed into a function body is a copy. Any new constants added while propagating through the function body do not flow back to the outer scope. This mirrors lexical scoping: a local declared inside a function is not visible outside it.

goto is not a dead-code terminator

It is tempting to treat goto symmetrically with break: both unconditionally transfer control, so statements after either are unreachable. However, goto differs in a critical way:

goto skip
print("dead")   -- unreachable
::skip::         -- BUT: this label must survive
print("alive")

If statements after goto are dropped, the ::skip:: label is also dropped, and the goto now targets a non-existent label — a runtime error. The optimizer would have broken a valid program.

break, by contrast, exits the current loop. Any label that appears after a break inside a loop body is only reachable from outside the break, i.e., from a goto earlier in the same loop iteration. Even there, the label must be kept. Hence the special-case: after break, scan the remaining statements and re-insert any .label nodes; drop everything else.

goto labels cannot be determined safe-to-drop without a full control-flow graph analysis, so the optimizer conservatively leaves the entire statement list intact when goto is encountered (only trimming after break).

All-or-nothing local binding removal

When a local declaration has multiple names (local x, y = f()), the optimizer removes the declaration only when all names are unused. If even one name is used, the entire binding is kept as-is.

An alternative would be to replace an unused name with _ or to restructure the declaration. This was rejected because:

  • The bound values come from a single multi-return expression. Splitting the binding would require introducing a temporary to hold all returned values, then assigning only the needed ones — this adds complexity and is not a win for a tree-walker.
  • The common case (one or two return values where one is discarded) is well served by the all-or-nothing rule: local val, _ = pcall(f) keeps the binding because val is read.

isPure is conservative on binary expressions

binary(+, identifier("x"), integer(1)) is classified as pure if x is a known local (in scope as a local, not a global) and the other operand is a literal. This might seem wrong — what if x holds a table with __add? However, note that this expression is evaluated only if the local it initializes is later dropped. If the local is dropped, the expression is also dropped. A pure expression that is dropped has no observable effect, so the purity decision does not matter for correctness.

The conservatism matters for the inverse case: a call like f() is correctly classified as impure, so local _ = f() is rewritten to f() rather than deleted.

Function bodies are re-optimized independently

When eliminateUnusedLocals encounters a functionDecl or functionDef, it recursively applies the pass to the function body with a fresh read collection from that body. This is correct because a function body is a new lexical scope: locals declared outside the function are accessed as upvalues, not as locals of the function's block, so they don't appear in the inner read set.

This means a local that is only captured by a closure is still considered "used" by the outer block (the closure body's reads do reference it), but a local inside the closure that is itself unused is also eliminated.