Duplex World Leaderboard
Five speech-to-speech systems, eleven metrics, six worlds.
Pick a world, a pillar and the systems you care about; every figure below answers to it. Intervals are the 95% intervals the paper publishes, and they are drawn rather than only printed, because most of the gaps on this page are smaller than they look.
One metric, five systems
The whole table
Table 3 of the paper, for the world selected above. Click any column to sort by it and to put it in the chart. The tinted cell in each column is the best value in that column; every metric here is higher-is-better.
Eleven metric columns. The table scrolls sideways; the system column stays put.
Two metrics at once
No combined scoreThe reason this page has no single index: a system that does the job and a system that sounds good are not the same system, and averaging the two hides exactly that. Put any metric on either axis and see.
Pass@1 by conversation type
Table 14 · realistic channelEleven interaction shapes. The nine enterprise types are shared by all five enterprise worlds; rerouting and all-day assistance arise only in Pathfinding, and their reward is a different definition, so they sit below the rule rather than being averaged in with the nine. The outlined cell in each row is that type's best system.
What was run
Every enterprise world instantiates the same nine interaction shapes, three scenarios each. Pathfinding shares one of them and adds two that only navigation elicits. Hover a cell for what it is worth in conversations.
Plus twelve authored Pathfinding variants that are released but not run: 156 scenarios authored, 144 campaigned.
How long a conversation runs
3,825 conversationsEvery conversation in the corpus, by world. The bar is the interquartile range, the notch the median, and the whisker the 5th to 95th percentile.
Every figure on this page is read from data.js, the file the whole site reads, so nothing here can disagree with anything else. Table 3 for the per-world blocks and the pooled block, Table 14 for the per-type figures, which the paper publishes as point estimates with no interval, and the duration densities are computed by gen_violins.py from the 3,825 conversation records rather than transcribed. What the metrics mean is on the metrics page. There is deliberately no combined index: the paper does not compute one, and a weighted average of a pass rate and a mean opinion score is a number with no unit. Filter state is kept in the address bar, so any view on this page is a link.