cubical-transport-hott-lean4/docs/ALGEBRA_PLAN.md

ALGEBRA_PLAN.md — Dev_Algebra: the universal-macro layer

Drafted 2026-05-01 on Dev_REL2. Captures the design and implementation plan for the long-running Dev_Algebra branch, which lifts the project's universal question form (docs/QUESTIONS.md) to a full meta-level proof-organisation algebra — one universal macro that reflects comp at the source-code level, plus a small attribute-and-tactic layer for autodiscovery of proof methodology.

---

0. The headline

One macro. Built from comp. Aliases accrue by usage; tactics are search over a library that grows under structural-Path declarations alone.

A first sketch enumerated 32 macros (one per cubical primitive + boundary / face / substitution / soundness families). All 32 are frozen partial applications of a single universal macro, restructure, which is comp lifted from the cubical-CTerm world to the meta-Lean-source world. The codebase ships with one macro and zero aliases; aliases accrue when patterns earn names.

---

1. Goals

1.1 In scope (Dev_Algebra REL2.5)

• One universal macro restructure covering all proof-organisation operations: relocate, rename, factor, merge, splice, classify, refactor-with-witness, etc.
• An attribute @[macroAlias] letting users (or the system itself) name recurring restructure invocations as ordinary Lean defs.
• An attribute @[methodology] registering tactic-fragments tagged by classifier, plus the cubical_search tactic that walks the registry, applies fragments via restructure, and transports fragments along declared structural Paths to derive new candidates from old ones.
• A widget rendering the question-graph and dispatching code actions via MakeEditLinkProps.ofReplaceRange.
• Incremental reorganisation: existing theorems gain question / classifier annotations file-by-file. Existing names are preserved as derived corollaries — no breaking change downstream.

1.2 Out of scope (deferred to future REL3+)

• Proof-body synthesis. Bodies remain hand-written (or written by AI agents in conventional tactic mode). The macro layer manages structure, never bodies.
• Higher-question algebra (paths-between-classifier-equivalences; 2-cells in the question category). Out of scope until cells-spec §8.
• Cross-language tooling (e.g., a CLI that batch-restructures outside the LSP session). Listed in §10 OQ.

1.3 Non-goals

• Replacing Lean 4's existing tactic framework. cubical_search is a tactic built on top of the standard infrastructure, not a replacement.
• Eliminating hand-written tactic scripts. The boundary is deliberate: structure is mechanical, bodies are creative.

---

2. The universal macro: restructure

2.1 Signature

restructure
  (i        : MetaPosition)        -- where in source: file slot,
                                   -- namespace position, decl ID
  (Context  : MetaCType)           -- meta-type of the artifact:
                                   -- theorem, def, instance, file,
                                   -- classifier-set, …
  (φ        : MetaClassifier)      -- when this restructuring applies
  (witness  : MetaArtifact)        -- new content valid on φ
  (fallback : MetaArtifact)        -- existing content off-φ
  : Edit Unit                      -- effect: source mutation

Same five fields as comp i A φ u t, promoted to the meta level. The macro emits zero or more MakeEditLinkProps.ofReplaceRange calls in the Edit monad.

2.2 Meta-mirror types

namespace Algebra

inductive MetaCType where
  | theorem  : MetaCType   -- a `theorem foo : T := proof`
  | definition : MetaCType -- a `def foo := body`
  | instance : MetaCType
  | structure : MetaCType
  | inductive_ : MetaCType -- a Lean `inductive` declaration
  | file : MetaCType
  | namespace_ : MetaCType
  | classifierSet : MetaCType
  | dependencyEdge : MetaCType

inductive MetaClassifier where
  | always       : MetaClassifier        -- "everywhere"
  | never        : MetaClassifier        -- "nowhere"
  | atDecl       : Name → MetaClassifier
  | inFile       : System.FilePath → MetaClassifier
  | underAttribute : Name → MetaClassifier
  | dependencyOf : Name → MetaClassifier
  | meet : MetaClassifier → MetaClassifier → MetaClassifier
  | join : MetaClassifier → MetaClassifier → MetaClassifier

inductive MetaArtifact where
  | source : String → MetaArtifact          -- raw Lean text
  | declAt : Lean.Syntax → MetaArtifact     -- a syntax tree
  | refTo  : Name → MetaArtifact            -- a reference to existing decl
  | empty  : MetaArtifact                   -- "remove this"

end Algebra

Every restructuring operation in the codebase reduces to a restructure call with these data.

2.3 Frozen aliases — the 32 macros revisited

Alias	Frozen arguments
transport_artifact i ctx w	φ := .always, witness := w, fallback := w
relocate_invariant i src dst	Context := .file, classifier inFile src, witness inFile dst
compose_proof_fragments	pure restructure (no freezing)
multi_compose ...	φ := join_of(branches), weave witnesses
rename_throughout x y	φ := atDecl x, witness := y, fallback := x
dispatch_on_shape S brs	Context := .inductive_, fold over branches
present_alternative T e	Context := MetaGlue T e _ (Glue lifted to meta)
submit_face_proof t a	classifier-conditioned glueIn-shape
extract_underlying g	inverse of present_alternative
define_question_shape S	Context := .inductive_, witness = the schema decl
instantiate_question S c args	restructure at a fresh position
MetaPath a b	Context := .definition, witness emits an alias
treat_as_equivalence	MetaPath plus a propositional witness
materialize	leaf: emit Lean text via ofReplaceRange
parse_back	dual leaf: read Lean source into a Question value
preserve_typing	guard composed over any restructure call
preserve_equivalences	guard checking declared MetaPaths survive
… (all others)	curry / pin / specialise the same five parameters

Implementation: the codebase ships with restructure itself (~150 lines). Each @[macroAlias] def … is a 1–3-line shorthand. The widget surfaces "name this pattern?" when an instantiation recurs, automatically inserting a new alias.

---

3. The Edit monad and Context comonad

3.1 The pair

namespace Algebra

/-- The `Edit` monad: a thread of source mutations.  Leaf operation
    is `ofReplaceRange`; everything else composes from there. -/
structure Edit (α : Type) where
  run : Lean.Server.CodeActionContext → IO (α × List MakeEditLinkProps)

instance : Monad Edit := …

/-- The `Context` comonad: at each point in the source, exposes the
    surrounding state — theorems in scope, classifiers applicable
    to the current goal, the question-graph neighbourhood. -/
structure Context (α : Type) where
  here  : α
  scope : Lean.Environment
  graph : QuestionGraph
  pos   : Lean.Syntax

instance : Comonad Context := …

end Algebra

3.2 The distributive law

The comonad provides context; the monad consumes context and produces edits:

/-- Lift a context-aware decision into an edit. -/
def Algebra.contextualEdit
    (decide : Algebra.Context α → Algebra.Edit β) :
    Algebra.Context α → Algebra.Edit β :=
  fun ctx => decide ctx

This is the standard "comonad-to-monad" distributive setup; it lets you write context-aware code actions ergonomically:

def renameQuestion (newName : String) : Context CompQ → Edit Unit :=
  contextualEdit fun ctx => do
    let oldName := ctx.here.name
    -- find every reference to oldName in ctx.scope
    let refs ← ctx.scope.findReferences oldName
    -- emit one ofReplaceRange per reference
    for r in refs do
      ofReplaceRange r.range newName

3.3 Soundness invariant

Every Edit operation passes through a preserve_typing guard:

def Edit.guarded (e : Edit α) : Edit α := do
  let (a, edits) ← e.run
  -- Apply edits to a fresh source buffer; type-check
  let buf ← applyEdits edits
  let result ← typeCheck buf
  if result.hasErrors then
    throw "restructure would break typing — aborting"
  return (a, edits)

This is the global invariant: no Edit ever surfaces in the editor that would break type-checking. The user can click any code action confidently.

---

4. The autodiscovery tactic: cubical_search

4.1 The methodology library

@[methodology]
def constLineSolver : Methodology :=
  { classifier := IsConstLine
    body       := fun q => CompQ.const_line_is_identity q (by classifier_check) }

@[methodology]
def fullFaceSolver : Methodology :=
  { classifier := IsFullFace
    body       := fun q => CompQ.full_face_is_identity q (by classifier_check) }

-- … one per cubical-core axiom; ~12-15 base methodologies.

The attribute registers the methodology in a global discrtree, indexed by classifier shape.

4.2 The tactic

syntax "cubical_search" : tactic

elab_rules : tactic
  | `(tactic| cubical_search) => do
    let goal ← Lean.Elab.Tactic.getMainGoal
    let goalType ← goal.getType
    -- 1. Reify goal as a CompQ (via parse_back from §3 of ALGEBRA_PLAN)
    let q ← reifyAsCompQ goalType
    -- 2. Find applicable methodologies via classifier matching
    let candidates ← MethodologyLibrary.findMatching q
    -- 3. Try each in priority order
    for M in candidates do
      try
        let proof ← M.body q
        Lean.Elab.Tactic.assignGoal goal proof
        return
      catch _ => continue
    -- 4. Try methodology-transport: for each existing M and each
    --    declared MetaPath M.classifier ↦ Q, attempt the transport
    let transported ← deriveByTransport q
    for M' in transported do
      try ... (same as step 3) ...
    -- 5. Structured failure
    throwError "no methodology applies; consider registering one
                 for {q.classifierShape}"

4.3 The methodology-transport mechanism

The crucial autodiscovery payoff:

def deriveByTransport (q : CompQ) : MetaM (List Methodology) := do
  let knownPaths ← getStructuralPaths
  let library ← getMethodologyLibrary
  let mut out := #[]
  for path in knownPaths do
    -- path : MetaPath classifierA classifierB
    if path.target.classifier.matches q then
      for M in library.matching path.source.classifier do
        let M' := transp path.line .top M.body
        out := out.push { classifier := q.classifierShape, body := M' }
  return out.toList

Once a small library of base methodologies exists, every new structural Path declared in the codebase automatically generates new methodology candidates by transporting existing methodologies across the path. Twenty starting methodologies + a hundred declared paths → potentially thousands of derived methodologies, each formally certified-by-construction.

4.4 Failure as a feature

When cubical_search fails it emits a structured report:

no methodology applies for question shape:
  CompQ
    body     := .glue ψ T f fInv s r c A
    φ        := .top
    isPath   := false
    isConst  := false
    isPi     := false
    isGlue   := true (matched)

candidates considered:
  ✗ glueAtTopSolver — guarded by `IsConstLine`, didn't fire
  ✗ glueAtTopSolver_specialised — registered for ψ = eq0, current ψ = eq1

derive-by-transport:
  no MetaPath connects current classifier to a known one

would you like to register a new methodology?  [click here]

The "click here" is itself a code action that opens a skeleton @[methodology] def … declaration via ofReplaceRange, which the human (or the next agent) fills in.

---

5. The widget

5.1 Surface

A Lean.Widget.UserWidgetDefinition rendering, for the active declaration:

• The current CompQ value (or its absence) at the cursor.
• The classifier shape of the goal.
• The list of applicable methodologies.
• Buttons: "factor question," "rename classifier," "compose with …," "transport along …," "name this pattern."
• The question-graph neighbourhood (5 hops in each direction), rendered as an interactive node-link diagram.

5.2 Code-action plumbing

Every button corresponds to one or more Edit actions. When clicked, the widget calls back to Lean (via Lean.Widget.RpcCall), the Edit runs, the source mutates via ofReplaceRange, the LSP re-elaborates, and the widget re-renders the new state.

5.3 No-LSP fallback

For users without the widget (CLI, headless CI), the same operations are available as lake exe algebra-restructure subcommands. The widget is the convenience surface; the underlying algebra works either way.

---

6. Phases

Phase	Deliverable	Days
A	MetaCType / MetaClassifier / MetaArtifact data types — meta-mirror of CType / FaceFormula / CTerm	3
B	restructure macro + Edit monad + Context comonad + soundness guard	5
C	@[macroAlias] attribute + alias-suggestion widget	3
D	UserWidgetDefinition rendering question-graph; ofReplaceRange integration	4
D′	@[methodology] attribute + cubical_search tactic + methodology-transport clause	4
E	Reorganisation — incremental annotation of existing theorems with @[question] / @[classifier]; aliases accrue as patterns earn names	open-ended

Committed: ~19 days for Phases A–D′. Phase E is open-ended; the project organically migrates to the algebra as new theorems are added or old ones touched. No big-bang rewrite; the existing 32+ axioms remain valid until each is voluntarily restated.

---

7. Risks & mitigations

Risk	Likelihood	Mitigation
restructure design hard to get right with no escape hatches	Medium	Phase B explicitly tests against ~10 representative restructuring scenarios from existing engine refactors before committing the design.
Macro debuggability — failed elaboration surfaces inside macro internals, not user source	Medium	Every restructure call wraps in a context-rich error report naming the classifier that didn't fire and the artifact that wasn't found.
Editor lock-in (widget assumes Lean LSP + WebView client)	Low	§5.3 fallback: same operations as lake exe algebra-restructure subcommands. Formal artifact (the Lean source) is still the source of truth.
Search performance — cubical_search walking a large library on every goal	Medium	MethodologyLibrary indexed by classifier shape (discrtree). Failed matches are O(1) on classifier disjointness; only matching methodologies are tried.
Compile-time cost — every macro expansion triggers Lean elaboration	Low	Macro outputs are small (ofReplaceRange calls); elaboration cost is dominated by re-checking the user's actual proof, not the macro itself.
Two different generated tactic scripts represent the same morphism	Low	Canonical-form pass on emitted source; structural equality on restructure invocations (REL2.5+ refinement).

---

8. Sequencing relative to REL2

            cubical-engine main (REL1 landed; REL2 Phase 1+2 on Dev_REL2)
                            │
            ┌───────────────┴───────────────┐
            ▼                                ▼
       Dev_REL2 (continuing)           Dev_Algebra (new, parallel)
       Phase 3: paideia K7             Phase A: meta-types
       (5–10d, paideia repo)           Phase B: restructure
                                       Phase C: macroAlias
                                       Phase D: widget
                                       Phase D': cubical_search
                                       Phase E: incremental reorg
            │                                │
            └────────────┬───────────────────┘
                         ▼
            Coordinated merge train when both arcs ready
            (engine `Dev_REL2` + `Dev_Algebra` → main; topolei,
            paideia → main; engine issue #1 closes with K7 +
            algebra-driven proof restructure)

The two arcs are independent at the engine level (neither blocks the other); they coordinate at merge time.

---

9. Definition of "done"

• Every existing eval_* / vTransp_* / vCompValue_* / Glue / Soundness theorem has at least one corresponding @[methodology] registration that closes its representative question via cubical_search.
• The widget renders the question-graph for any open Lean file.
• A code action exists for: factor, compose, rename, relocate, attach-classifier, declare-MetaPath, transport-methodology.
• A regression suite verifies that every code action preserves type-checking on the engine's existing test corpus.
• KERNEL_BOUNDARY.md §3.7 (cubical-aware tactics) updated to record cubical_search as a mid-horizon delivery (still pending full cubical_simp for §3.7's strongest form).

---

10. Open questions (logged here)

1. Domain of restructure — strictly cubical-core artifacts (theorems / definitions in CubicalTransport.*), or everything in scope (any Lean declaration)? Cubical-core is simpler and more justifiable; everything-in-scope is more general but harder to keep sound. Default: cubical-core, with a per-call opt-in to broader scope.
2. Persistence — graph computed on the fly each LSP session (always-fresh, slower), or persisted as Lean attributes (cached, possibly stale). Default: on the fly, with an optional cache file generated by lake exe algebra-cache.
3. CLI tool — do we ship lake exe algebra-restructure from day one, or wait for editor adoption? Default: from day one, so headless CI can verify code actions.
4. AI prior surface — does cubical_search consult a learned prior (from past successes) for ordering candidates? Out-of-scope for REL2.5; tracked for REL3+.

---

11. Why this matters (summary)

The Eulerian framing throughout the project has emphasised river bed → ferry → carrying load for REL2. Dev_Algebra adds the map: a navigable register of currents, a tooling infrastructure that lets you trace any flow, splice rivers, divert without losing volume. The map is built from the same primitive the rivers are built from. Every layer of the system, from the cubical-CTerm engine through the Lean-source-organisation algebra, is the same comp-shape applied at a different stratum. The codebase is closed under its own operations — and the autodiscovery tactic is the visible face of that closure.

---

End of ALGEBRA_PLAN.md. Companion to QUESTIONS.md (philosophy) and EULERIAN.md (poetic record).