PCBBench

Method

Each run gives one agent one task prompt. Scripts check the board it delivers, an AI judge reviews its layout and routing, and PCBWay prices it. The score covers the board only.

A run

  1. The agent gets the task prompt with its model and medium reasoning effort filled in, a clean machine with KiCad, and up to 12 hours. AI spend is capped at $100 ($250 for Fable 5.2).
  2. The run's work folder, logs, tokens and AI cost are collected.
  3. Scripts run DRC and ERC with the run's own KiCad version, match the fab files to the board, and check the BOM, placement file, parts, gates and layout.
  4. Opus 5.5 reviews an anonymous copy of the board.
  5. PCBWay quotes 1 and 5 assembled boards, after any fixes needed to order them.

Score

Each item earns its points times a fraction from 0 to 1. A board that fails a gate loses 20 points. Differences under about 5 points aren't meaningful. "Change the mix" on the leaderboard reweights the groups.

ItemPointsFraction
Design review 20
Layout101 minus 0.1 per minor and 0.2 per major layout finding, down to 0
Routing10the same, for routing findings
Circuit checks 20
Decoupling5RP2350 power pins with a capacitor within 3.2 mm
USB protection50.5 for an ESD part on D+ and D−, plus 0.25 for each line with a series resistor
Antenna keepout51 when there are keepouts and no copper is in them
Battery drain51 at 5 µA or less drawn by unswitched battery dividers, falling linearly to 0 at 25 µA
Layout checks 30
Thermal pads10mean over the exposed pads of their vias over the number needed, 0 for a pad with no plane on another layer
Grounding10USB, QSPI and microSD track length over a ground plane on the next layer
Silkscreen5texts at least 0.8 mm high
Board size51 at the Feather RP2350's 50.8 × 22.8 mm area or less, falling linearly to 0 at 5 times that area
Orderable 20
Fab notes51 when every fab option in the notes can be ordered from PCBWay
Board51 when the board needs no rework
BOM lines101 minus 0.2 for each BOM line that needs a fix, down to 0
Parts data 10
3D models5parts with a 3D model
Supplier part numbers5BOM lines with a valid number from at least one supplier

Gates

  • Prompt features. The BOM has a part for every feature the prompt asks for.
  • Crystal. Its load capacitors are within 25% of what its datasheet needs, with 3 pF of board capacitance.
  • Buttons. BOOT pulls QSPI_SS to ground through a switch, and RUN has its own switch to ground.
  • Power path. The charger takes USB VBUS, charges the battery connector and feeds the 3.3 V regulator.
  • No fatal finding. The judge found nothing that stops the assembled board working, damages a part or stops PCBWay building it.

Checks

Design review

Opus 5.5 on high reasoning, in the Sprocket CLI with its KiCad connector, sees the KiCad project, the BOM, renders and layer images, without run names. It lists layout and routing problems with their parts, nets and evidence, rated minor, major or fatal. It gives no scores.

Circuit checks

Scripts read the board file for the nearest capacitor to each RP2350 power pin, ESD parts and series resistors on USB D+ and D−, copper in the Wi-Fi antenna keepout, and the current battery dividers draw when the board is off.

Layout checks

Vias in the exposed pads of the RP2350, charger and regulator. USB, QSPI and microSD track length over a ground plane on the next layer. Silkscreen text height. Board area.

Orderable

Fab options PCBWay doesn't offer, DRC errors or fab files that differ from the board, and BOM lines with invalid part numbers, too little stock for 5 boards or a footprint that doesn't match the part.

Parts data

Parts with a 3D model that loads, and BOM lines with a valid LCSC, Digi-Key or Mouser part number.

Not in the score

  • AI cost. The run's API cost at market prices, including subagents.
  • Time. How long the agent was working.
  • Board price. PCB, assembly and parts at PCBWay's instant-quote prices, without shipping. PCBWay makes at least 5 PCBs, so 1 board means 5 PCBs with 1 assembled. Parts are priced at distributor list prices.

The README has every check's exact rules, and the bench code runs them.