cubical-transport-hott-lean4/NAGA_IR_PLAN.md

NAGA_IR_PLAN.md — direct naga IR construction from EMLPath

A pick-up-in-one-session plan for eliminating the last text-format intermediary in the render pipeline: the GLSL string that currently sits between Rust EMLExpr and naga::Module. Future author: read §1–§4 before writing any code; §5 is the staging; §6–§8 are the traps.

---

0. TL;DR

Today (after P6): Lean EMLPath → FFI → Rust EMLPath → Rust GLSL string → naga-glsl parser → naga::Module → SPIR-V → GPU.

Target: Lean EMLPath → FFI → Rust EMLPath → naga::Module (direct) → SPIR-V → GPU. No GLSL text format. No parser.

Why: the text stage is the last unverified intermediate in the pipeline. naga-glsl is a third-party GLSL parser; its semantic interpretation of our shader strings is trust-based. Eliminating it shortens the trust boundary from "naga's GLSL frontend plus Lean+Rust emitters must all agree" to "Lean and Rust emitters must produce equivalent naga modules".

It is not an axiom-killer on its own — compileEMLPath_correct remains an axiom — but it removes a 10⁴-line dependency from the trust surface and brings a Lean-side naga-IR model (future work) within reach.

---

1. Prerequisite reading

Read these before writing any code. Skip any item at your peril: each pitfall in §6 is a trap you'll step into without the background.

1.1 Upstream (naga + wgpu)

Match the version pinned in native/canvas-rs/Cargo.toml (currently wgpu = "22.1" → naga 22.1.0). Paths given are on this machine's cargo registry; repoint as needed.

File	What to read it for
$CARGO_HOME/registry/src/*/naga-22.1.0/src/lib.rs	Module, Function, EntryPoint, Expression, Statement, Literal, MathFunction, Type, TypeInner. Top-level IR vocabulary.
$CARGO_HOME/registry/src/*/naga-22.1.0/src/arena.rs	Arena<T>, UniqueArena<T>, Handle<T>. Why every expression lives behind a handle, why types must dedupe.
$CARGO_HOME/registry/src/*/naga-22.1.0/src/valid/mod.rs	Validator, ValidationFlags, Capabilities. Validation is a hard precondition for SPIR-V emission — unvalidated modules panic the writer.
$CARGO_HOME/registry/src/*/naga-22.1.0/src/back/spv/mod.rs	write_vec(&Module, &ModuleInfo, &Options, Option<&PipelineOptions>) -> Result<Vec<u32>, Error>. One call; needs ModuleInfo from the validator.
$CARGO_HOME/registry/src/*/naga-22.1.0/src/front/glsl/*	Reference implementation of a frontend building a Module. Good place to copy patterns from.
$CARGO_HOME/registry/src/*/wgpu-22.1.0/src/lib.rs (search ShaderSource)	Three variants: SpirV(Cow<[u32]>), Glsl{...}, Naga(Cow<'static, Module>). The Naga variant takes a module directly — we can pass our constructed Module without round-tripping through SPIR-V, though SPIR-V is the default and more widely exercised path.

1.2 In-repo context

Doc / source	Read for
RENDER_BRIDGE_GAP.md §4, §6	The original framing of the rendering-stack axioms. §6 describes the multi-layer Projection/Observation pipeline this plan is a prerequisite for.
Topolei/GPU/Spec.lean	ShaderSemantic, shaderVar, shaderVarWithDim, EMLPath.toColor, the axioms (compileEMLPath, compileEMLPath_correct, render_faithful). The semantic the naga module must realise.
Topolei/EML.lean + Topolei/EML/Path.lean	EMLExpr, PlotConfig, EMLPath, EMLPath.toFragShaderProbe. The Lean-side shader emitter — the naga builder must produce a module whose SPIR-V behaves identically on the seven P7 probes.
native/canvas-rs/src/eml.rs	Rust-side EMLExpr / EMLPath + emlexpr_from_lean walker + to_frag_shader_probe string emitter. The input side of the naga builder (walker is unchanged); the output side is what's being replaced.
native/canvas-rs/src/lib.rs::offscreen_render_pixel	Where the shader-module handoff happens. The swap-in point.
Topolei/Render/Probe.lean	probeEmitterDiff (string diff Lean vs Rust GLSL) + pixel probes. The test surface that must stay at 10/10 after the swap.

1.3 Background concepts

• Arena-based IR. Every Expression and Type is stored by handle in an arena owned by the enclosing Function / Module. Construct nodes strictly bottom-up: a handle is only valid once all its subexpressions exist in the same arena.
• Emit statements. Computing an expression isn't the same as making its value available to later statements. Non-const expressions must be wrapped in Statement::Emit(range) entries in the function body, naming the ranges of expression handles that are now "live" at this point in control flow. Forgetting this is a silent-mis-emit trap (emits a module that validates but produces wrong code).
• UniqueArena vs Arena. Types go in UniqueArena so f32 is allocated once and reused. Expressions go in Arena (each Literal(1.0) is a distinct node). Mixing these up is a validation-time error.
• Bindings. Entry-point args/results need Bindings (Location for varyings, BuiltIn for special inputs like Position). Uniform globals need ResourceBinding { group, binding } matching the wgpu bind-group layout.

---

2. Current state — what's wired today

native/canvas-rs/src/lib.rs::offscreen_render_pixel builds a wgpu::ShaderModule via ShaderSource::Glsl { shader: fragment_glsl, stage: Fragment, defines: ... }. Behind the scenes wgpu calls naga-glsl, parses the string, produces a naga::Module, then writes SPIR-V via naga::back::spv::write_vec.

The fragment_glsl string comes from eml::EMLPath::to_frag_shader_probe (or PlotConfig::toFragShader for the decorated live-render path).

Emission structure (reference)

The probe shader emitted today is a small fixed scaffold plus one EML expression. Reproduced here in the order a naga builder will need to materialise the pieces:

#version 450
layout(location=0) in  vec2 uv;
layout(location=0) out vec4 fragColor;
layout(set=0, binding=0) uniform Uniforms {
    float u_time;        // offset  0
    float u_pathParam;   // offset  4
    vec2  u_resolution;  // offset  8
};
void main() {
    float px = uv.x;
    float py = uv.y;
    float {dim} = u_pathParam;
    float v = {body};            // the EML expression
    float r = 0.5 + 0.5 * cos(6.2832 * v);
    float g = 0.5 + 0.5 * cos(6.2832 * v + 2.094);
    float b = 0.5 + 0.5 * cos(6.2832 * v + 4.189);
    fragColor = vec4(r, g, b, 1.0);
}

Every line above has a direct naga IR counterpart; §5 is the translation.

---

3. Target state — what the naga IR builder looks like

A new module native/canvas-rs/src/emit_naga.rs exposes:

pub fn build_probe_module(path: &crate::eml::EMLPath) -> naga::Module;

Internal organisation:

// Top-level orchestrator.  Produces a fully-validated Module.
pub fn build_probe_module(path: &EMLPath) -> Module { ... }

// The fragment entry-point's body + its expression/statement arenas.
fn build_main_fn(module: &mut Module, types: &ProbeTypes,
                 globals: &ProbeGlobals, path: &EMLPath) -> Function { ... }

// The EML-specific piece: EMLExpr → Handle<Expression> under an
// extending function arena.  This is the structural heart.
fn emit_emlexpr(
    expr: &EMLExpr,
    module: &Module,
    function: &mut Function,
    env: &EmitEnv,
) -> Handle<Expression> { ... }

// The scaffolding types referenced by multiple stages.
struct ProbeTypes {
    f32:       Handle<Type>,
    vec2_f32:  Handle<Type>,
    vec4_f32:  Handle<Type>,
    uniforms:  Handle<Type>,  // struct { f32, f32, vec2<f32> }
}

struct ProbeGlobals {
    uniforms_buf: Handle<GlobalVariable>,  // @group(0) @binding(0)
    uv_in:        Handle<GlobalVariable>,  // @location(0) input
    frag_out:     Handle<GlobalVariable>,  // @location(0) output
}

// Compile-time env mapping EMLExpr variable names to the naga
// expression handle that evaluates to that variable's value at the
// current point.  `path.dimName` maps to the `u_pathParam` field of
// the uniform; "px" / "py" map to uv.x / uv.y; everything else is
// a "bound to 0.0" fallback (matching shaderVar's fallback).
struct EmitEnv {
    px: Handle<Expression>,
    py: Handle<Expression>,
    dim_name: String,
    dim_value: Handle<Expression>,
}

Outside the builder:

// In offscreen_render_pixel (and the live renderer later):
let module = emit_naga::build_probe_module(&path);
let shader_module = device.create_shader_module(
    wgpu::ShaderModuleDescriptor {
        label: Some("probe-fragment"),
        source: wgpu::ShaderSource::Naga(Cow::Owned(module)),
    }
);

---

4. The EMLExpr translation table

Every variant maps to one or two naga expression nodes. The table is small because EMLExpr is small — this is why the "maximally correct" pathway is tractable.

EMLExpr	naga Expression
One	Expression::Literal(Literal::F32(1.0))
Var "px"	env.px (reuse handle)
Var "py"	env.py (reuse handle)
Var name where name == path.dimName	env.dim_value (reuse handle)
Var name (other)	Expression::Literal(Literal::F32(0.0)) — matches shaderVar's fallback
Eml(l, r)	exp(l) - log(r): two Math calls + one Binary::Subtract

And the main-body scaffolding above v = eml(...):

GLSL line	naga expressions
float px = uv.x	AccessIndex { base: uv_in, index: 0 }
float py = uv.y	AccessIndex { base: uv_in, index: 1 }
float dim = u_pathParam	AccessIndex { base: Expression::Load(uniforms_buf), index: 1 }
0.5 * cos(6.2832 * v + phase)	Literal + Math(Cos) + Binary(Mul/Add) chains
fragColor = vec4(r, g, b, 1.0)	Expression::Compose { ty: vec4_f32, components: [r, g, b, Literal(1.0)] }; then Statement::Store { pointer: frag_out, value: composed }

Every intermediate expression must be reachable via Statement::Emit before the Store that uses them.

---

5. Staging — seven commits, each independently verifiable

Each stage adds strictly more and ships green regression. Do not skip stages — they're ordered so validation catches each class of mistake in isolation.

Status: Stages 1–6 done as of 2026-04-25. All 17 probes pass on the Vulkan-Intel-Iris-Xe target. Stage 7 (cutover / drop the GLSL feature for the probe path) is the only remaining follow-up.

Stage 1 — hardcoded red pixel — ✅ done

Goal: prove the SPIR-V pipeline works end-to-end with zero EML content.

• build_probe_module ignores path; returns a module whose fragment main writes vec4(1.0, 0.0, 0.0, 1.0).
• Wire it into offscreen_render_pixel behind a #[cfg(feature = "naga_ir")] or a boolean.
• Run probe-test. Expected: the seven pixel probes fail (pixels are red, not the EML color), but the adapter line prints and no panic occurs. The failure mode confirms wiring; a panic would mean module validation failed.
• Commit.

Stage 2 — uniforms bound — ✅ done

Goal: confirm the bind-group layout matches.

• Build the Uniforms struct type in the module.
• Add uniforms_buf global with @group(0) @binding(0).
• Output vec4(u_pathParam, u_pathParam, u_pathParam, 1.0).
• Confirm probe pixels equal (u_pathParam, u_pathParam, u_pathParam) — for pathParam = 0.5 every channel is ≈ 0.5.
• Commit.

Stage 3 — varyings wired — ✅ done

Goal: reach px = uv.x, py = uv.y.

• Add uv_in global with @location(0) input binding.
• Output vec4(uv.x, uv.y, 0.0, 1.0).
• Confirm pixel (x, y) in framebuffer has r = (x+0.5)/w, g = 1 - (y+0.5)/h (NDC y-flip).
• Commit.

Stage 4 — minimal EML emission — ✅ done

Goal: walk the EMLExpr tree.

• Implement emit_emlexpr covering One, Var, Eml.
• Output v = eml_body(path); write vec4(v, v, v, 1.0).
• For plotExp (body exp(px) - log(1.0)), pixel (64, 64) should give v ≈ exp(0.504) - log(1.0) ≈ 1.655. Stored as 1.655 clamped by the format. Since Rgba32Float doesn't clamp, the value is literal.
• Commit.

Stage 5 — full probe-shader equivalent — ✅ done

Goal: match EMLPath.toColor pixel-for-pixel.

• Add the cos-cycle: r = 0.5 + 0.5 * cos(6.2832 * v), g with phase 2.094, b with phase 4.189.
• All seven pixel probes pass.
• Commit.

Stage 6 — emitter-drift test extension — ✅ done

Goal: the naga-IR and GLSL-string emitters must agree on pixel output.

• Keep both build_probe_module (naga) and to_frag_shader_probe (GLSL) usable.
• Add a third probe row to ProbeTest: render the same path via both paths, assert the pixel outputs agree.
• Runs 3 emitter-drift + 7 naga-pixel + 7 glsl-pixel = 17 probes.
• Commit.

Stage 7 — cutover — pending

Goal: make naga-IR the default, drop naga-glsl from the probe.

• offscreen_render_pixel now uses ShaderSource::Naga unconditionally.
• Keep to_frag_shader_probe only for the rustEmitProbeShader FFI (used by the emitter-drift test on the Lean side — Lean's string emitter still has to agree with something human-readable).
• Remove features = ["glsl"] from wgpu in Cargo.toml if no remaining code path needs it. (Check: PlotConfig::toFragShader is still a GLSL string used by the interactive render loop — keeping glsl for that path is reasonable until a separate pass converts the decorated plot shader to naga IR too.)
• Commit.

---

6. Known pitfalls

These are the traps the naga-glsl frontend handles silently and a hand-built module must get right.

1. Expression-before-use. If you add Expression::Binary { left, right } to the function arena before left is in the arena, validation fails with a cryptic InvalidExpression. Always append subexpressions first.
2. Statement::Emit. Non-constant expressions must be wrapped in an Emit statement or they're not computed at runtime. Track the last-emitted handle as you build; emit a fresh range on each "new live handle". See naga/src/front/glsl/builtins.rs for the pattern.
3. UniqueArena for types. module.types.insert(ty, Span::UNDEFINED) deduplicates — calling insert(f32, _) twice returns the same handle. Do not call Arena::append on the types arena; that API isn't exposed on UniqueArena.
4. Struct layout must match Rust Uniforms. naga's struct-layout validator rejects mismatched offsets/strides. Enforce Layout::Std140 (Vulkan UBO convention) and specify each member's offset / span explicitly. Cross-check against native/canvas-rs/src/lib.rs::Uniforms — time: f32 @ 0, path_param: f32 @ 4, resolution: [f32; 2] @ 8.
5. ResourceBinding { group: 0, binding: 0 } on the uniform global. Mismatch with wgpu's BindGroupLayout is a runtime error in create_render_pipeline.
6. Entry-point arguments and result. Fragment entry-point takes uv: vec2<f32> at Location(0) and returns vec4<f32> at Location(0). Naga's EntryPoint encodes these on the Function::arguments / result directly, not as globals — differs from the GLSL surface where they're top- level in/out. Two idioms coexist in naga modules; the argument-binding idiom is cleaner. Study naga/src/front/glsl/functions.rs for the translation.
7. NDC y-flip. The GLSL emitter produces a shader that reads uv.y matching the vertex-shader's pos * 0.5 + 0.5 convention. wgpu+Vulkan flip Y in the rasterizer. A naga-IR fragment shader inherits the same conventions — don't add a compensating flip.
8. Validation capabilities. ValidationFlags::all() is usually what you want. Some naga features (e.g., subgroup ops) require enabling specific Capabilities; our probe shader uses none of these. Keep Capabilities::empty() initially.
9. Debug names. module.types[h].name = Some(...) and Function::named_expressions are optional but make SPIR-V debuggers (RenderDoc) vastly more usable. Worth the 10 lines.
10. naga::Module::Default. Module doesn't implement Default; construct explicitly with every arena as ::default() and entry_points: Vec::new(). Typo'd field initialisers are silent.

---

7. Verification strategy

Each stage is verifiable independently, but the composite check is:

7.1 Static

• naga::valid::Validator::new(ValidationFlags::all(), Capabilities::empty()).validate(&module)? must succeed. Run on every module before SPIR-V emission. Validation failures are informative (point at the handle + issue).

7.2 Dynamic

• All 10 probes pass (3 emitter-drift + 7 pixel). This is the authoritative end-to-end check. If pixels diverge from the GLSL path by more than tolerance, the naga emitter has a bug.
• Byte-level SPIR-V diff (optional). Dump the SPIR-V bytes from the naga-IR path and the naga-glsl path; compare. Byte-equality is unlikely but semantic equivalence is checked by the pixel probes.

7.3 Stage-by-stage check

Run probe-test after each stage. At Stage 5 onward all 10 pass; earlier stages will have pixel-probe failures by design but still exercise adapter selection + module validation + render pass.

---

8. Open questions (decide before Stage 1)

1. Keep the GLSL path for PlotConfig::toFragShader (decorated plot shader)? That shader has grid/axes/curve overlay code not representable in EMLExpr. Options:
   • (a) Keep GLSL for decorated plots; naga-IR only for probe shader. Simpler. Keeps features = ["glsl"].
   • (b) Extend EMLExpr with a "decoration" AST for grid/axes/curve. Principled, larger scope.
   • (c) Let the decorated plot be a fixed naga-IR template with an EMLExpr-hole for the body. Middle ground. Recommendation: (a) for the first pass; revisit (c) once the probe is direct.
2. Pipeline caching. Each probe call recreates the wgpu device (seconds per call). Separate concern from naga IR, but if we're touching the same pipeline setup code, worth a paragraph.
3. wgpu ShaderSource::Naga vs round-trip via ShaderSource::SpirV. Naga hands the module directly to wgpu's internal SPIR-V writer; SpirV accepts pre-written bytes. Functionally equivalent for our use; Naga is one less step but SpirV gives byte-level inspectability for debugging. Recommendation: use Naga for the production path and expose a --dump-spirv debug switch for the bytes.
4. Error-path UX. naga validation errors are structured (ValidationError); surface them in offscreen_render_pixel's return as Err(format!(...)) rather than via panic!. A mis-built probe module should produce a sentinel, not a crash.

---

9. Formal status after this pass

This push does not eliminate compileEMLPath_correct as an axiom. The axiom's content is still "the shader handle produced by Lean's compileEMLPath has semantic equal to EMLPath.toColor". What it eliminates is the unverified text stage:

Pipeline layer	Before	After
Lean → Rust AST	structured (P6)	structured (unchanged)
Rust AST → shader input	Rust emits GLSL text	Rust builds naga IR
shader input → naga Module	naga-glsl parses ~10⁴ LOC	same arena we built
naga Module → SPIR-V	naga:🔙:spv (unchanged)	unchanged
SPIR-V → GPU	wgpu + driver (unchanged)	unchanged

The "same arena we built" line is the correctness gain: the module the SPIR-V writer sees is the exact structure we constructed from EMLExpr, with no parse step in between. A future Lean model of naga IR could then close the remaining gap to a theorem — but that is a separate project.

---

10. Session budget

Realistic one-session target: Stages 1–5 (hardcoded red → full probe parity). That's ~500–800 Rust lines, most of it in the new emit_naga.rs, with careful arena bookkeeping. Expect two hours of naga-docs reading before writing one stage.

Stages 6–7 (drift test + cutover) are ~100 lines each and can go in a follow-up.

---

End of NAGA_IR_PLAN.md. Update §8 if the decisions change. Update §5 as stages land — tick them here rather than spreading status across session summaries.