Method
Each run gives one agent one task prompt. Scripts check the board it delivers, an AI judge reviews its layout and routing, and PCBWay prices it. The score covers the board only.
A run
- The agent gets the task prompt with its model and medium reasoning effort filled in, a clean machine with KiCad, and up to 12 hours. AI spend is capped at $100 ($250 for Fable 5.2).
- The run's work folder, logs, tokens and AI cost are collected.
- Scripts run DRC and ERC with the run's own KiCad version, match the fab files to the board, and check the BOM, placement file, parts, gates and layout.
- Opus 5.5 reviews an anonymous copy of the board.
- PCBWay quotes 1 and 5 assembled boards, after any fixes needed to order them.
Score
Each item earns its points times a fraction from 0 to 1. A board that fails a gate loses 20 points. Differences under about 5 points aren't meaningful. "Change the mix" on the leaderboard reweights the groups.
| Item | Points | Fraction |
|---|---|---|
| Design review 20 | ||
| Layout | 10 | 1 minus 0.1 per minor and 0.2 per major layout finding, down to 0 |
| Routing | 10 | the same, for routing findings |
| Circuit checks 20 | ||
| Decoupling | 5 | RP2350 power pins with a capacitor within 3.2 mm |
| USB protection | 5 | 0.5 for an ESD part on D+ and D−, plus 0.25 for each line with a series resistor |
| Antenna keepout | 5 | 1 when there are keepouts and no copper is in them |
| Battery drain | 5 | 1 at 5 µA or less drawn by unswitched battery dividers, falling linearly to 0 at 25 µA |
| Layout checks 30 | ||
| Thermal pads | 10 | mean over the exposed pads of their vias over the number needed, 0 for a pad with no plane on another layer |
| Grounding | 10 | USB, QSPI and microSD track length over a ground plane on the next layer |
| Silkscreen | 5 | texts at least 0.8 mm high |
| Board size | 5 | 1 at the Feather RP2350's 50.8 × 22.8 mm area or less, falling linearly to 0 at 5 times that area |
| Orderable 20 | ||
| Fab notes | 5 | 1 when every fab option in the notes can be ordered from PCBWay |
| Board | 5 | 1 when the board needs no rework |
| BOM lines | 10 | 1 minus 0.2 for each BOM line that needs a fix, down to 0 |
| Parts data 10 | ||
| 3D models | 5 | parts with a 3D model |
| Supplier part numbers | 5 | BOM lines with a valid number from at least one supplier |
Gates
- Prompt features. The BOM has a part for every feature the prompt asks for.
- Crystal. Its load capacitors are within 25% of what its datasheet needs, with 3 pF of board capacitance.
- Buttons. BOOT pulls QSPI_SS to ground through a switch, and RUN has its own switch to ground.
- Power path. The charger takes USB VBUS, charges the battery connector and feeds the 3.3 V regulator.
- No fatal finding. The judge found nothing that stops the assembled board working, damages a part or stops PCBWay building it.
Checks
Design review
Opus 5.5 on high reasoning, in the Sprocket CLI with its KiCad connector, sees the KiCad project, the BOM, renders and layer images, without run names. It lists layout and routing problems with their parts, nets and evidence, rated minor, major or fatal. It gives no scores.
Circuit checks
Scripts read the board file for the nearest capacitor to each RP2350 power pin, ESD parts and series resistors on USB D+ and D−, copper in the Wi-Fi antenna keepout, and the current battery dividers draw when the board is off.
Layout checks
Vias in the exposed pads of the RP2350, charger and regulator. USB, QSPI and microSD track length over a ground plane on the next layer. Silkscreen text height. Board area.
Orderable
Fab options PCBWay doesn't offer, DRC errors or fab files that differ from the board, and BOM lines with invalid part numbers, too little stock for 5 boards or a footprint that doesn't match the part.
Parts data
Parts with a 3D model that loads, and BOM lines with a valid LCSC, Digi-Key or Mouser part number.
Not in the score
- AI cost. The run's API cost at market prices, including subagents.
- Time. How long the agent was working.
- Board price. PCB, assembly and parts at PCBWay's instant-quote prices, without shipping. PCBWay makes at least 5 PCBs, so 1 board means 5 PCBs with 1 assembled. Parts are priced at distributor list prices.
The README has every check's exact rules, and the bench code runs them.