Build a fair scorecard for comparing repeated swap outcomes

A fair swap scorecard reports how the observations were selected, whether the trades completed and how costs were measured. A single average can hide failed attempts, large outliers or a sample dominated by one easy token pair.

Record the sample before the result

Specify networks, pairs, size ranges, observation times, wallet setup state and the benchmark used. Separate unexecuted quote comparisons from actual fills. Report missing quotes and unsupported requests rather than quietly deleting them from the denominator.

Transaction inclusion is distinct from submission. A quoted route or submitted request should not be counted as a completed swap without the corresponding outcome evidence.

Choose complementary metrics

• Completion ratio for the intended tasks.
• Total economic cost and total traded value.
• Cost as a percentage of value, weighted by trade size.
• Median cost per completed task and selected tail observations.
• Separate setup and failed-attempt spending.

These answer different questions. A value-weighted ratio measures cost across the capital traded; a median task cost describes a typical observed task. Neither can substitute for the other.

Illustrate the weighting problem

In a hypothetical sample, one $100 trade costs 2% and one $10,000 trade costs 0.2%. The unweighted mean rate is 1.1%. Total cost is $22 across $10,100 traded, approximately 0.2178%. Both calculations are valid descriptions only when their weighting is named.

Limit the conclusion

A scorecard supports claims about the sampled conditions. It does not prove that one service is universally cheapest. If two methods were observed at different times or on different requests, disclose the confound. Publish enough per-observation data to make the reported aggregates checkable, while avoiding unnecessary wallet-history disclosure. A smaller matched sample is often more interpretable than a larger collection with inconsistent cost boundaries.