Emacs Backbone

An external orchestration program (written in Gleam) that manages Emacs configuration through bidirectional JSON-RPC over stdio, providing deterministic, dependency-aware package installation and configuration management inspired by NixOS principles.

Motivation

After over a decade of using Emacs, I grew frustrated with a persistent problem: my configuration would break unexpectedly, and I'd have to debug it before I could actually get work done.

The root cause: no true dependency resolution

Traditional Emacs configuration tools don't actually resolve dependencies. with-eval-after-load is just a convenience macro that defers code execution until a library loads - it doesn't guarantee execution order across multiple deferred blocks. Similarly, use-package's :after keyword delays loading until dependencies are available, but doesn't provide deterministic ordering. There's no built-in way to declare "configure B only after A is fully ready" and have the system guarantee it.

This leads to non-deterministic behavior where the same configuration could behave differently across restarts or machines. Race conditions and timing issues create bugs that are nearly impossible to track down.

The lazy loading dilemma

I've been a Doom Emacs user for years with heavily customized config. Doom's lazy loading is excellent for fast startup, but I kept hitting the same frustrating pattern:

1. Make a mistake in my config or custom elisp
2. Error only appears when that feature loads - could be hours into my session
3. Hard to debug because the context is lost

The problem isn't lazy loading itself - it's that errors hide until runtime. Traditional approaches optimize for startup speed but defer error discovery.

The fail-fast philosophy

That's why Emacs Backbone takes the opposite approach: load everything at startup in deterministic order and encounter all errors immediately. Fix them right away while the context is fresh. Once Emacs starts successfully, you can be confident nothing will break hours into your session. If you comment out a package, its config-unit automatically skips - no manual cleanup needed.

This trades a few extra seconds at startup for reliability. Since Emacs is a long-lived process that runs for days or weeks, I'd rather fail fast at startup than risk hours of lost work from a mid-session crash.

What Emacs Backbone provides

Inspired by NixOS principles, Emacs Backbone treats configuration as a dependency graph that's resolved deterministically using topological sorting:

• True dependency resolution - Declare dependencies explicitly, the system figures out the order
• Deterministic execution - Same declarations always produce the same order
• Immediate feedback - All configuration errors surface at startup
• Reproducible behavior - Your config works the same way every time, on every machine

Every package and configuration unit declares its dependencies explicitly, and the system ensures they execute in the correct order - every single time.

Why Gleam?

I happened to be learning Gleam when this idea crystallized, and it turned out to be the perfect choice. Gleam's strong static type system and compiler give me confidence that once it compiles, it will work. The compiler catches potential issues before runtime, which is exactly the kind of reliability I was seeking for my Emacs configuration. This compile-time safety complements the deterministic runtime behavior - together they ensure that configuration problems are caught early and behavior remains consistent.

Why not just `use-package :after`?

A common question is: If use-package has the :after keyword, why do we need config-unit! and an external orchestrator?

The short answer is that use-package is an excellent package loader, but a very limited state manager. When your configuration scales, relying solely on :after for dependency management breaks down in a few critical ways.

1. The Cross-Package Integration Problem (N-to-N)

The most glaring limitation of use-package :after is handling configurations that depend on multiple packages being loaded and configured first.

Take the evil-org package as an example. To configure it properly, you need both evil and org to be fully initialized. Where does the integration code go? With use-package, you are forced into awkward, tightly coupled nesting:

;; The use-package way: Awkward and conceptually messy
(use-package org
  :config
  ;; Do org stuff...
  (use-package evil-org
    :after evil
    :config
    ;; Wait, does this belong in the 'org' block or the 'evil' block?
    (evil-org-set-key-theme)))

This forces your integration logic to live inside the namespace of a single, arbitrary package.

With emacs-backbone and config-unit!, you decouple the integration entirely. You create a distinct, independent unit that explicitly declares its upstream dependencies. It sits perfectly at the intersection of the two, keeping your base package configs clean:

;; The emacs-backbone way: Clean, decoupled dependency declaration
(config-unit! evil-org-integration
  :requires (evil-org)
  :after (org-base evil-base)
  :config
  (evil-org-set-key-theme))

2. Reactivity vs. Determinism

Under the hood, use-package :after is just syntactic sugar for Emacs's built-in with-eval-after-load. This is a reactive system. It tells Emacs: "Keep an eye out, and if package X ever happens to load, run this code."

Because it is reactive, your configuration's execution order is highly vulnerable to the physical order of your init.el file, autoload triggers, or user actions. This is a common source of silent failures, race conditions, and infinite loading loops.

emacs-backbone is deterministic. The Gleam backend reads your config-unit! declarations, parses them into a strict Directed Acyclic Graph (DAG), and uses topological sorting to guarantee mathematically perfect execution order over JSON-RPC. The physical location of the code in your files no longer matters; the dependencies dictate the flow.

3. Granularity of State

use-package operates strictly at the package level—it waits for a package to call (provide 'some-package).

However, in advanced Emacs setups, you often need to depend on a logical state, not just a package. For example, you might want your customized modeline to load only after your entire "UI Framework" (fonts, themes, and fringes) is fully configured. config-unit! allows you to depend on arbitrary blocks of configuration state to resolve, giving you orchestration granularity that use-package simply cannot provide.

Overview

Emacs Backbone separates the configuration framework from user configuration, similar to Doom Emacs:

• Core System (~/.config/emacs): Gleam backend, Emacs Lisp framework, macros, and orchestration logic
• User Configuration (~/.config/backbone): Your personal config.org, package declarations, and config units

This separation allows you to:

• Version control your user config independently from the core
• Receive core updates without affecting your personal configuration
• Share your configuration with others easily

Core Concepts

The heart of Emacs Backbone lies in two declarative macros that replace traditional Emacs configuration patterns:

package! - Declarative Package Management

The package! macro declares packages with explicit dependencies, eliminating load-order guesswork:

;; Simple package declaration
(package! magit)

;; Package from a specific repository
(package! evil
  :repo "emacs-evil/evil"
  :host "github")

;; Package with dependencies - Emacs Backbone ensures correct installation order
(package! evil-collection
  :repo "emacs-evil/evil-collection"
  :deps (evil))

;; Local package with subdirectories
(package! lsp-bridge
  :local "~/projects/lsp-bridge"
  :files ("*.el" "*.py" "acm" "core" "langserver")
  :no-compilation t
  :deps (markdown-mode yasnippet))

Key features:

• Dependency resolution: Use :deps to declare package dependencies - the system guarantees they install first
• Repository flexibility: Install from GitHub, GitLab, or any git host with :repo and :host
• Local packages: Use :local for packages from local filesystem, with :files to include subdirectories
• Version pinning: Use :branch, :tag, or :ref for reproducible builds
• No manual ordering needed: Declare packages anywhere in your config - dependencies are resolved automatically

config-unit! - Dependency-Ordered Configuration

The config-unit! macro replaces use-package, with-eval-after-load, and scattered configuration blocks with explicit, ordered units:

;; Basic configuration unit
(config-unit! evil-config
  :requires (evil)  ; Require evil before running this unit
  :config
  (evil-mode 1)
  (setq evil-want-C-u-scroll t))

;; Configuration with ordering dependencies
(config-unit! evil-collection-config
  :requires (evil-collection)
  :after (evil-config)  ; Guaranteed to run after evil-config
  :config
  (evil-collection-init))

;; Complex dependency graph - automatically resolved
(config-unit! my-keybindings
  :after (evil-config evil-collection-config which-key-config)
  :config
  (global-set-key (kbd "C-c g") 'magit-status))

;; Runtime prerequisites: require env vars and executables before running
(config-unit! llm-tools
  :requires (gptel)
  :env (OPENAI_API_KEY ANTHROPIC_API_KEY)
  :executable (rg git)
  :config
  (message "LLM tooling is ready"))

Key features:

• :requires - Require listed features before running the unit body
• :after - Execution ordering between config-unit! names (not package! names); unlike eval-after-load, this is guaranteed
• :env - Require environment variables to be set before the unit runs
• :executable - Require external binaries to exist on PATH before the unit runs
• :config - Configuration body that runs in deterministic order
• Topological sorting - Complex dependency graphs are automatically resolved
• Circular dependency detection - Fails fast if your configuration has cycles

How They Work Together

1. Declaration phase: You declare package! and config-unit! in any order in your config.org
2. Dependency resolution: Gleam backend extracts all declarations and builds a dependency graph
3. Package installation: Packages install in dependency order (packages with no deps first)
4. Configuration execution: After all packages install, config units execute in topologically sorted order
5. Guaranteed determinism: Same declarations always produce the same execution order

This replaces the traditional Emacs configuration chaos with a predictable, reproducible system.

Installation

Prerequisites

• Emacs 29+ (with native compilation support recommended)
• Bun (runtime for generated/emacs-backbone.mjs)

Core Installation

1. Clone the core repository:

   git clone git@github.com:nohzafk/emacs-backbone.git ~/.config/emacs
   cd ~/.config/emacs

User Configuration Setup

1. Create your user configuration directory:

   mkdir -p ~/.config/backbone

2. Create your configuration:

   Create ~/.config/backbone/config.org with your Emacs configuration using package! and config-unit! macros. For example:

   * Package Declarations
   #+begin_src emacs-lisp
   (package! evil
     :repo "emacs-evil/evil")
   
   (package! which-key
     :repo "justbur/emacs-which-key")
   #+end_src
   
   * Configuration Units
   #+begin_src emacs-lisp
   (config-unit! evil-setup
     :requires (evil)
     :config
     (evil-mode 1))
   
   (config-unit! which-key-setup
     :requires (which-key)
     :after (evil-setup)
     :config
     (which-key-mode 1))
   
   (config-unit! ai-setup
     :requires (gptel)
     :after (which-key-setup)
     :env OPENAI_API_KEY
     :executable (git rg)
     :config
     (message "AI tooling enabled"))
   #+end_src
   

3. Start Emacs:

   emacs

4. Tangle and reload from inside Emacs:

   • C-h r e to tangle ~/.config/backbone/config.org into config.el
   • C-h r r to reload the freshly tangled config

   On first startup, Emacs Backbone may warn that config.el does not exist yet. That is expected until you tangle it for the first time.

Example Configuration

For a complete, real-world example of an Emacs Backbone configuration, see:

nohzafk's literate-darwin config

Configuration

Customization Variables

emacs-backbone-user-directory

Default: ~/.config/backbone

Directory containing your user configuration. Customize if you want to use a different location:

(setq emacs-backbone-user-directory "~/my-custom-config")

emacs-backbone-bun-executable

Default: Auto-detected (/opt/homebrew/bin/bun or from PATH)

Path to the Bun executable used for generated/emacs-backbone.mjs:

(setq emacs-backbone-bun-executable "/opt/homebrew/bin/bun")

emacs-backbone-enable-debug

Default: nil

Enable verbose logging and debugging output:

(setq emacs-backbone-enable-debug t)

Usage

Interactive Commands

• M-x emacs-backbone-start - Initialize Emacs Backbone (auto-invoked in init.el)
• M-x emacs-backbone-tangle-config - Tangle config.org to config.el
• M-x emacs-backbone-reload-config - Full reload: tangle + reinitialize

The Gleam backend is managed over stdio, so it exits with Emacs. There is no separate manual shutdown command.

Keybindings

• C-h r e - Tangle configuration
• C-h r r - Reload configuration

Development

Building the Project

bun run build            # Regenerate generated/emacs-backbone.mjs
gleam test               # Run tests

Generated Bun Backend

The repo ships a committed Bun bundle so end users do not need Gleam installed at runtime. Emacs always starts Backbone with:

(list emacs-backbone-bun-executable
      (expand-file-name "generated/emacs-backbone.mjs" user-emacs-directory))

Regenerate that bundle during development with:

bun run build

This script:

1. Runs gleam build
2. Bundles the generated Gleam JavaScript into generated/emacs-backbone.mjs
3. Produces the stable backend entrypoint Emacs runs with Bun

Run the committed bundle directly with:

bun generated/emacs-backbone.mjs

If Emacs cannot find Bun in its environment, set emacs-backbone-bun-executable to an absolute path.

Developer Workflow

Developers need both Gleam and Bun.

When you change backend code under src/:

1. Edit the Gleam source
2. Run bun run build
3. Run gleam test
4. Commit both the source changes and the updated generated/emacs-backbone.mjs

When you change only the Emacs Lisp side under lisp/, you usually do not need to rebuild the generated backend bundle.

Normal User Workflow

Normal users need only Bun.

1. Install Bun
2. Clone this repo to ~/.config/emacs
3. Keep generated/emacs-backbone.mjs from the repo checkout
4. Start Emacs

No Gleam installation is required unless the user intends to modify the Gleam backend and regenerate the committed bundle.

Architecture

Two-Directory Structure

~/.config/emacs/                           # Core system (this repository)
├── src/                                   # Gleam backend
│   ├── emacs_backbone.gleam              # Entry point
│   ├── jsonrpc.gleam                      # JSON-RPC communication over stdio
│   ├── resolver.gleam                     # Dependency resolution
│   └── ...
├── lisp/                                  # Core Emacs Lisp framework
│   ├── backbone.el                       # JSON-RPC client & lifecycle
│   ├── macro-package.el                   # package! macro
│   ├── macro-config-unit.el               # config-unit! macro
│   └── ...
├── init.el                                # Core initialization
├── early-init.el                          # Early initialization
├── gleam.toml                             # Gleam project config
└── package.json                           # JavaScript module metadata

~/.config/backbone/                        # User configuration (you create this)
├── early-init.el                          # Your early initialization (optional)
├── config.org                             # Your literate configuration
├── config.el                              # Generated from config.org
├── config/                                # Your helper utilities
└── clis/                                  # Your CLI configurations

Elpaca Compatibility

Emacs Backbone currently bootstraps against Elpaca installer version 0.12 in lisp/package-manager.el.

This integration currently depends on some Elpaca implementation details, not just its public user-facing configuration surface. In particular, Backbone has workarounds for split packages in a shared monorepo checkout, such as magit-section living inside the shared magit repository checkout, and packages like embark-consult living inside the shared embark checkout.

That means Elpaca upgrades should be treated as compatibility work: if Elpaca's shared-source, clone-skip, or dependency-detection internals change, Backbone's integration may need to be updated as well.

Project Structure

• src/ - Gleam backend source code
• lisp/ - Core Emacs Lisp framework
• test/ - Gleam tests
• bin/ - Utility scripts

Troubleshooting

Config not found

If you see "Emacs Backbone user config not found", ensure:

1. ~/.config/backbone/config.el exists
2. You've tangled your config.org file
3. The emacs-backbone-user-directory variable points to the correct location

Backbone won't start

Check:

1. Bun is installed: which bun
2. The emacs-backbone-bun-executable variable is correct
3. generated/emacs-backbone.mjs exists in the repo
4. If you changed Gleam source, you rebuilt the bundle with bun run build

Package installation fails

1. Check the *emacs-backbone* buffer for errors
2. Enable debug mode: (setq emacs-backbone-enable-debug t)
3. Verify your package declarations syntax in config.el