Mathematical Framework for Language (Physics-Inspired Semantics)

This repo contains my Semester 5 R&D work (Aug-Nov 2025, report finalized Dec 2025) on a physics-inspired mathematical framework for language semantics. The core idea is to model language as state evolution in a Hilbert-space-like geometry, with operators controlling how semantic information accumulates over a sentence. Think of it as a lightweight, interpretable alternative to huge LMs: instead of billions of parameters, we study structured operators + constrained evolution that can still capture semantic alignment and redundancy suppression.

The work is a stepping stone, not a final system. The current framework is intentionally minimal so that future (better) approaches can build on a clean, interpretable base.

Motivation (Why This Exists)

• Physics explains complex phenomena with small sets of principles (states, operators, evolution). Language shows structured behavior too.
• Modern LMs are powerful but computationally heavy and often hard to interpret.
• We ask: Can language meaning be modeled as constrained semantic evolution, using physics-style operators?
• If yes, we may get lighter frameworks that still show semantic structure (alignment, redundancy suppression, energy separation).

Core Ideas (Short Summary)

• Word embeddings are mapped to a frequency domain (DFT) to expose structure.
• Each word becomes a complex-valued function phi in a C*-like space.
• Sentence meaning evolves as a recursive semantic state psi.
• Projection-like operators suppress redundant semantic content.
• A non-unitary global operator W' learns how to amplify meaningful sentences and suppress incoherent ones.

Approaches Explored

1. Word-Based Projection (Local Redundancy)

Project the state away from the current word's direction:

• Captures immediate repetition (e.g., "the the").

2. Context-Based Projection (Global Redundancy)

Project using the current accumulated context psi:

• Suppresses redundancy already present in the full sentence history.
• More global, context-aware filtering.

3. Uniform Language Model (Baseline)

Unitary evolution with Hermitian H:

• All sequences treated equally.
• Redundancy not penalized -> fails to model semantic validity.

4. Non-Unitary Evolution (Learnable)

Introduce non-Hermitian term W so evolution becomes non-unitary:

• Enables amplification of meaningful sentences.
• Suppresses redundant/incoherent sequences.
• W' is learned with a physics-inspired loss.

Pseudocode (Learning W')

Below is a clean pseudocode version of what the notebook implements.

Input: sentence pairs (A, B) with similar meaning, distractor C
Params: alpha (speed limit), lambda_align (alignment weight)
Init: W' (learnable complex matrix)

for each training set (A, B, C):
    # 1) Prepare words
    embed words with GloVe (100d)
    phi = DFT(embeddings)        # frequency-domain word functions

    # 2) Static semantic evolution (projector-only)
    psi_0 = phi_1
    for n = 2..N:
        H0 = I - |phi_n><phi_n| / ||phi_n||^2
        psi_n = exp(-i * H0) * psi_{n-1}

    # 3) Non-unitary steering (learned global operator)
    psi_final = psi_N
    captured = W' * psi_final
    energy = ||captured||^2

    # 4) Loss = energy + alignment + speed + constraint
    L = -(energy_A + energy_B)
        + lambda_align * ||psi_A - psi_B||^2
        + speed_penalty(psi trajectory, alpha)
        + annular_constraint(||W'||)

    optimize W' with Adam

Return: trained W' and semantic energy/similarity metrics

Experiments (What We Checked)

Experiments are in sighN_W_learning.ipynb:

• Semantic energy: meaningful sentences show higher energy than random or scrambled ones.
• Cosine similarity: semantically equivalent sentences converge in semantic space.
• Redundancy suppression: repeated words reduce energy.

The presentation and report include plots showing these trends.

Files

• RnD_quantumLang.pdf - full report (Dec 2025)
• RnD_presentation.pdf - slide deck (Mar 23, 2026)
• sighN_W_learning.ipynb - experiments + training loops

Status / Future Work

This is not the final approach. It's a foundation for better versions. Planned improvements:

• richer Hamiltonian formulations
• grammar-aware operators
• end-to-end training on large corpora
• path-integral / group-theoretic formulations
• stronger embedding fine-tuning to match the framework

How to Run (Notebook)

The notebook downloads GloVe embeddings and runs experiments with PyTorch.

# open the notebook locally
jupyter notebook sighN_W_learning.ipynb

Dependencies: torch, numpy, matplotlib. The notebook auto-downloads GloVe 6B 100d embeddings.

---

If you're reading this: feel free to build on the framework, criticize it, or extend it. This work is meant to spark better, more efficient and interpretable semantic modeling.