Efficiency frontier
Score versus cost
← Lower cost
Higher score ↑
Independent model evaluation U.S. customs brokerage
Customs intelligence,
measured.
BrokerBench tests whether language models can navigate the statutes, regulations, tariff schedule, and applied reasoning expected in licensed U.S. customs brokerage work.
—
evaluation items
—
models compared
—
scored / attempted results
—
knowledge domains
Interpretation note
A current design-review release
These results predate the current benchmark configuration.
Completion cap
—
01 / Leaderboard
Model performance
Compare substantive accuracy with the operational cost and response time observed during each run.
Loading verified results…
What stands out
Accuracy tells only part of the story.
Loading cross-model observations…
02 / Breakdown
Where models succeed—and struggle
Domain and response-format views reveal performance differences hidden by a single aggregate score.
Knowledge map
Score by customs domain
0%
100%
Loading domain results…
Task format
Score by response type
Output discipline
Format validity
A strict format contract makes responses machine-verifiable. Truncated outputs are counted as invalid.
03 / Fresh knowledge
A post-cutoff tariff test
Fifteen classification items are locked to HTSUS 2026 Revision 11, effective after the models’ stated knowledge cutoffs, and require applying chapter notes and tariff descriptions.
Temporal holdout
2026 HTS Revision 11
Loading fixed-evidence holdout results…
Loading the cross-model finding…
Model result
15 direct classifications
Chapter detail
Fixed-evidence accuracy
Five questions per chapter
04 / Sample evaluations
See what BrokerBench asks
One independently authored example from every knowledge domain shows the reasoning, precision, and output discipline the benchmark is designed to test.
9
knowledge domains represented
These illustrative examples are separate from the scored corpus. Answers are checked against the linked official authority.
0
scored questions disclosed
Loading source-verified samples…
05 / Methodology
Built for auditable comparison
BrokerBench uses deterministic scoring, pinned evaluation data, and portable result records so every published aggregate can be traced to a specific run.
Substantive score
Unweighted mean of deterministic per-item scores.
Observed cost
Provider-reported cost for the complete benchmark run.
Observed latency
End-to-end request timing under a consistent concurrency limit.
Evaluation corpus
A broad test of broker knowledge
Questions span regulatory procedure, tariff classification, valuation, entry, origin and marking, broker compliance, and integrated practice. Exact answers are held out from this publication.
- Release
- —
- Scoring
- Deterministic
- Repetitions
- —
- Seed policy
- —
Response formats
— items
Technical provenance
Dataset SHA-256
—
Source run IDs
—
Generation setting
—
Publication status
design_review / current v0.2
06 / Contact
Contact us
Have a question about the benchmark, a source correction, a model evaluation request, or an interest in licensed-broker review? We’d like to hear from you.