Tapeout: Cross-Artifact Integration Review for SoC IP Upgrades

Ayomide Adekoya · Tapeout Labs · August 2026

How we built a system that reviews, patches, and verifies an IP upgrade across hardware, firmware, and verification -- the .tapeout contract language underneath it, the Rust graph engine that makes whole-chip queries instant, and the agent that runs EDA tools mid-reasoning.

The problem

Take a new revision of a DMA controller. A register moves from 0x08 to 0x10. A #define three repos away is now wrong. An interrupt renames from done_int to completion_irq_o. The SoC wrapper still drives the old pin. A reset flips polarity and every assertion against the old edge passes vacuously. None of that fails at the point of change. It fails in integration, or in silicon.

That review is still mostly a senior engineer reading SystemVerilog, SystemRDL, IP-XACT, C headers, IRQ maps, and test plans by hand. Those formats were never meant to be joined. Industry studies put verification above half of project effort, with first-silicon success rates going the wrong way. Leading-edge programs sit in the hundreds of millions to over a billion dollars. The expensive part is keeping the program coherent while pieces move.

SoC-scale agent work keeps failing in the same place. HWE-Bench (417 hardware bug-fix tasks) put the best agent at 70.7% overall and under 65% on complex SoCs, with failures concentrated in fault localization, hardware semantics, and coordination across RTL, config, and verification. Phoenix-bench measured a 37--58% drop moving the same agents from software to hardware. CLOSER-Bench treats chip work as budgeted closure, not localized codegen. Cadence bought ChipStack and demos prompt → mental model → UVM → Xcelium / Jasper on a block -- block-level DV automation inside a commercial tool chain. IP-upgrade collateral (register maps, firmware headers, IRQ wiring, DV sequences that still assume the old revision) is a different failure surface.

We start where the upgrade breaks firmware and DV. Parsers establish identity across artifacts. Models propose what parsers cannot decide. Tools execute. Humans sign off.

What you get when you run it

Six stages from read-only review through draft, verify, contract, CI, and chip-level multi-IP review.
Figure 1. Stages on the path from IP review toward whole-program engineering. Review is what the mutation corpus and dma-upgrade fixture measure. Draft, verify, contract, CI, and chip share the same harness.

Mutation corpus and dma-upgrade score the review path. Draft, verify, contract, CI, and chip share the same harness; those stages are not in the corpus totals.

Pipeline

Pipeline: extract, diff, impact deterministic on top; link, deduplicate, report below. Models enter at LINK and optionally in --full-review.
Figure 2. Top row is deterministic (no model). Models enter at LINK. With --full-review, a multi-turn session runs after graph corroboration. Everything the model emits stays PROPOSED until a human decides.

Extract is deterministic. SystemVerilog goes through tree-sitter (CST, byte offsets kept). Reset polarity comes from sensitivity lists (negedge rst_n), then condition negation, then name convention last. Registers go through systemrdl-compiler, OpenTitan hjson reggen, or IP-XACT via peakrdl. Unresolvable constants ($clog2, casts) stay unresolved. Unresolved widths downgrade downstream findings. They do not get filled in.

Diff is structural: offset moved, access RW→RO, port direction flipped, interrupt gone. Impact joins against the SoC snapshot by value, not by name. A firmware #define binds to a register when evaluated integers match old offsets / reset words. Name containment is not the binder.

Interrupt identity is also deterministic when SoC RTL is present. A port is an interrupt because of where its net goes, not because the name contains irq. Connectivity tracing catches dma_complete with no irq token. Each port gets evidenced / proven-unreached / unanalyzable. Only a proof of non-routing softens a removal finding. Missing evidence does not count as proof.

The .tapeout format

Five source languages, none of them express cross-artifact relationships cleanly. So we defined TOF (.tapeout): flat text, formal EBNF, recursive-descent parser that fails on malformed lines. Open source.

tapeout 1
ip dma revision v2 top dma_ctrl

module dma_ctrl ports 20 registers 5 interrupts 1 @ rtl/dma_ctrl.sv:12
port completion_irq_o output @ rtl/dma_ctrl.sv:31
register CONTROL offset 0x00 width 32 reset 0x0 @ regs/dma.rdl:8
  field EN [0:0] reset 0x0 sw rw @ regs/dma.rdl:10
  field IE [1:1] reset 0x0 sw rw @ regs/dma.rdl:14
register STATUS offset 0x10 width 32 reset 0x1 @ regs/dma.rdl:22

define DMA_STATUS_OFFSET value 0x08 @ fw/dma_regs.h:24
bind register:dma.STATUS -> define:DMA_STATUS_OFFSET value 0x10 confidence 1.0 confirmed
finding FW-001 confirmed blocking "Stale firmware offset for STATUS"
  before 0x08 after 0x10 register dma.STATUS @ fw/dma_regs.h:24

One fact per line. Deterministic order (offset, then name). Emit → parse → emit is a fixed point, so git diff on two .tapeout files is a semantic hardware diff. Compaction vs IP-XACT comes from dropping nesting, not from abbreviating keywords. Hex carries explicit 0x. Provenance is @ path:line.

Trust is grammar. A bind carries confirmed (parser evidence) or proposed (model or heuristic) plus confidence in [0, 1]. A downstream reader cannot treat a model guess as a parser fact without ignoring the tier field.

Trust tiers

CONFIRMED from parsers can block merges; PROPOSED from agent or heuristic needs human approval; UNKNOWN is surfaced, never dropped.
Figure 3. Only CONFIRMED findings block a merge. Model output cannot set exit code 1 by itself, and cannot clear a parser finding.

CONFIRMED -- a parser saw it. May be severity: blocking. PROPOSED -- model or heuristic, advisory at any confidence. UNKNOWN -- something changed and nothing explained it; surfaced, not dropped. Exit rule: rc == 1 iff a CONFIRMED blocking finding exists. Full-review system prompt: deterministic findings are ground truth for structural facts. The model adds context. It does not override them.

Integration graph

Every review builds a typed graph: registers, fields, ports, interrupts, resets, defines, assertions, address regions as nodes; containment, connectivity, value exposure, IRQ routing, address mapping as edges. Impact is a traversal under hardware causality rules, not an undirected BFS. Undirected BFS leaks through hub nodes (field → register → sibling field). The traversal keeps per-relation direction and a descend-after-ascend guard.

conf(path) = ∏e ∈ path conf(e)     (prune when conf < 0.8; min-composition also supported)

Production review uses rustworkx. tapeout-graph is a separate Rust/PyO3 engine (CSR via counting sort, columnar properties, epoch-stamped visited buffers so queries cost O(reached)).

CSR build and impact BFS from 1K to 1M nodes. At 100K nodes: 10.8ms build, 25.1us query.
Figure 4. Criterion, release+LTO, Apple Silicon. At ~100K nodes: 10.8 ms build, 25.1 µs impact query. Cap is 1M nodes. Numbers from tapeout-graph/BENCHMARKS.md.

validate_bind checks proposed relationships by offset equality, mask containment, region ownership, and connectivity. The useful case is rejecting a stale define whose value no longer matches the register. Equality of small constants alone never proves identity.

Production review still walks rustworkx. tapeout-graph is what we bench and harden against as the contract grows. On wide-fanout graphs it is several times faster than the pure-Python path (engine_bench.json: ~5--8× on the port-fanout microbench).

Where the model is allowed in

Model layers: linker, draft, EDA execution, closed-loop revision, full review session.
Figure 5. Each layer shipped only after the check harness for it existed. Model output stays PROPOSED.

Semantic linking

done_int gone, completion_irq_o new. No parser can decide rename vs remove+add. That call is the model's first job.

Evidence packet to Claude Fable 5, tool calls, schema-enforced propose_equivalence, structural veto, PROPOSED finding, engineer decision, training row.
Figure 6. Linking loop. Read-only tools only. Schema enforces confidence and evidence. Human decision becomes a training example.

Agent stack: pydantic-ai, Claude Fable 5 at temperature 0.1. Tools: SQL over hdl-kgraph, source line reads, release-note search. Output schema:

class EquivalenceProposal(BaseModel):
    removed_signal: str
    added_signal: str
    confidence: float = Field(ge=0.5, le=1.0)
    evidence: list[str] = Field(min_length=1)
    reasoning: str

Below 0.5 is unrepresentable -- the signal goes to unmatched_removed and surfaces as UNKNOWN. A proposal naming a signal the parsers never removed or added is discarded. Prompt bias: prefer a missed rename over a wrong one. False negatives come back as unconnected-pin blockers. False positives bless a broken merge.

Fallback when the agent is off (tapeout/linker/heuristics.py):

score(r, a) = 0.45 · J(Tr, Ta) + 0.25 · lex(r, a) + 0.30 · rel(r, a)
Heuristic weights: 0.45 semantic, 0.30 release-note, 0.25 rapidfuzz lexical.
Figure 7. Heuristic weights. Width must match. Scores ≥ 0.45 enter greedy 1:1 assignment.

J is Jaccard on semantic tokens. lex is rapidfuzz. rel credits a release-note line only when both names appear with renam. The heuristic cannot open the wrapper. That is why the agent exists.

irq-rename--dma: done_int to completion_irq_o with eleven expected findings across CONFIRMED, PROPOSED, UNKNOWN.
Figure 8. Corpus case irq-rename--dma. Eleven must-fire expectations in evals/corpus/irq-rename--dma/expected.json. Solid = CONFIRMED (parser). Dashed = PROPOSED / UNKNOWN.

Draft, verify, full review

Draft: deterministic evidence packet (file, line range, old value, new value) → model in batches of 8 (tapeout/drafts/agent.py) → unified diffs + manifest. Approval required.

Verify: tools run the checks. Closed loop is draft → verify → observe failure → revise, default max 5 iterations (LoopConfig.max_iterations). The model does not declare success. The tool does.

--full-review (requires TAPEOUT_AGENT=1 and working model credentials; fails loud otherwise) hands every finding to a multi-turn session: read source, search specs (BM25 + plan/execute/reflect), query the graph, run an EDA tool when it has a hypothesis. Prompt line: use EDA tools to verify, not to fish. Enrichments stay PROPOSED.

Commercial EDA on PATH

Default verify in our fixtures runs open tools: Verilator --lint-only / --assert, GCC -fsyntax-only. That is what the closed loop grades patches against today.

Customer environments have Cadence / Synopsys / Siemens binaries. tapeout/eda_console/ is the adapter for those. It opens a persistent TCL session to whatever is installed and lets the review agent drive it the same way it drives Verilator -- after it already has a hypothesis. No UVM codegen. No Jasper test-plan codegen from a natural-language prompt. Session + command + structured parse.

Discovery (discovery.py) looks on PATH for vivado, jg (JasperGold), vsim (Questa), xrun (Xcelium), dc_shell, vcs, sg_shell (SpyGlass), plus yosys / verilator / tclsh. Launch flags are tool-specific (vivado -mode tcl -nojournal -nolog, jg -tcl, xrun -tcl, …). Verilator and VCS are one-shot only; they do not keep an interactive TCL session.

TclSession owns the process. Each command is followed by a unique puts sentinel so stdout boundaries stay clean across multi-line tool spam. Session state accumulates (loaded design, variables). Agent tools registered in agent_tools.py:

parse_report is deterministic post-processing. Timing: Vivado WNS/TNS/WHS/THS table, DC slack lines, generic fallbacks (TimingReport). Elaboration: errors / warnings / modules / unresolved refs (ElaborationReport). Lint: SpyGlass and Vivado message formats, plus a generic path (LintMessage). The model does not get to reinterpret those numbers; it gets the structured object.

Safety runs before every run_tcl (safety.py). Blocked prefixes: rm, file delete, exec rm, …. Blocked patterns: write_bitstream, program_device, delete_project. Session-kill commands (exit, quit, close_project -force) are refused so the agent cannot tear down the session by accident. Optional capture writes command/stdout pairs to ~/.tapeout/eda_captures/ for later training.

With no commercial binary on PATH, discover_tools() returns an empty map and the adapter stays dark. With licenses present, the agent drives those tools through the safety gate to check findings the deterministic pipeline already raised.

No Harbor final_report.json on the 417-task HWE-Bench set is checked in yet. Next run: pre-bake SQLite graphs per commit, HTTP tools inside the container, A/B same model with and without graph.

Measured runs

Mutation corpus: precision and recall 1.00 on the deterministic track.
Figure 9. From eval_report.json (loaded by the figure script). Heuristic track on the mutation corpus.
26-case mutation corpus by breakage family.
Figure 10. Corpus composition. 26 cases, corpus hash 967507f36ebbf007.

Mutation corpus (tapeout 0.3.0, 2026-08-05, eval_report.json): 26 synthetic upgrades, labeled breakage classes. Heuristic track totals tp=87, fp=0, fn=0 -- precision / recall / F1 = 1.00. Noise counters all zero. Wall ~32.7 s for the suite. Scope: the deterministic rules on breakages this corpus models.

dma-upgrade fixture (head_to_head_report.json, grounding remeasured 2026-08-11 with the fixed filename regex): seven labeled cross-artifact breakages. Deterministic Tapeout track recalls 7/7 in 1.27 s, 45 findings, 92 cited sources, 0 ungrounded. Diff-scoped Opus 4.6 review (IP diff only) cites downstream files with ungrounded rate 0.4167. Full-context review on the same model: ungrounded rate 0.0, ~13.6 s. Same recall, different grounding and cost.

HWE-Bench N=96 grounding study (head_to_head_hwe.json): diff-scoped mean ungrounded rate 0.199 [0.152, 0.250]; full-tree 0.038 [0.012, 0.072]; recall ~0.35--0.40 either way; median full-tree input tokens ~68K. Tapeout recall on that consumer set is 0.0 -- those consumers are DV sequences and C drivers, outside the register-offset / reset-word bindings the deterministic track covers.

Public datasets

HIR (tapeout-labs/hir-relationships): register/field ↔ firmware #define relationship prediction. Convention-tier labels are mechanical (OpenTitan regtool, Caliptra PeakRDL). Splits by IP / by design. Train is 35,667 pairs (3,963 positive) in local train.jsonl.

HIR split sizes across train, hard, val, Caliptra, hard test.
Figure 11. HIR splits. Easy-tier AUC near 1.0 is expected under convention following.
Edge classifier v7 AUC by tier vs token-containment baseline; v6 to v7 same-IP AP and recall@P90.
Figure 12. Production classifier from models/eval_matrix_v7.json. Proposals only (GRAPH-002). Never silent CONFIRMED edges. Opt-in via TAPEOUT_EDGE_MODEL; bad config fails loud.

Token-containment baseline hits AUC ~0.995 on easy tiers. Hard tiers (lexicon mutations, HWE co-modification pairs, picorv32 hand labels, LiteX / CMSIS-SVD conventions) move the score. On HWE-Bench co-modification pairs the token baseline is still ahead of v7 (0.895 vs 0.856 AUC in the matrix). Those edges stay PROPOSED; agent + human review stay in the loop.

aplace-transitions: 784,774 placement transitions across 28 circuits on HF. Separate line from integration review.

Direction

Near term: take the graph we already build in review and expose it as tools inside Harbor on pku-liang/hwe-bench (417 tasks). Same model, with and without graph tools. OpenTitan (245 tasks) needs Synopsys VCS images we do not ship. Chisel repos (XiangShan, Rocket Chip, 86 tasks) need a different extraction path than tree-sitter SV.

Further out: hold a live model of the program, drive commercial EDA for the repetitive work, keep humans on signoff. ChipStack showed the block-verification version of that story inside Cadence. We are building the cross-artifact version, starting at IP upgrades.

Semiconductors were a $791.7B industry in 2025. Chip design talent is scarce. The useful outcome looks like what happened one layer up when PCB tooling got good enough that small teams could ship boards that used to need a department (Trace was our version of that).

Closing

Parsers establish structural facts. Models handle semantic work that rules do not encode cleanly. EDA tools evaluate the implementation. .tapeout is the representation that keeps trust tiers visible.

IP integration is the current scope. It is where we measure cross-artifact state, coordinated edits, tool feedback, and evidence retention. If those hold under wider tasks, scope expands. If they do not, the harness numbers move.

About the author

I'm Ayomide. Most of my work sits between AI systems and hardware.

I built ML/infra at Apple and Meta, and hardware through NASA RockSat-C while studying EE + CS at Howard. Before Tapeout I co-founded Trace (AI-native PCB design): .trace_sch / .trace_pcb, agentic KiCad, $150K pre-seed, 65+ boards through manufacturing.

Trace is where the representation problem showed up for me. Models got useful once a circuit stopped looking like CAD syntax and started looking like a structured engineering object. Tapeout is that question one layer down.

Building with Justin Bell (CMU ECE, CPU DV at AMD) and Darin ten Bruggencate (EDA GTM across Cadence, Synopsys, Altium, Quilter).

Selected work

see more →

ayo@tapeoutlabs.com -- if you are upgrading an IP this quarter and want to replay a historical integration against the system, write.

Sources

Wilson / Siemens 2024 Functional Verification Study · HWE-Bench · Phoenix-bench · CLOSER-Bench · Cadence AVE / ChipStack · Synopsys L4 · Siemens Fuse · SIA 2025 · aplace-transitions / HIR. Figure script: public/blog/figures/make_figures.py. Eval chart loads eval_report.json; edge classifier loads eval_matrix_v7.json.