This post is largely inspired from this 538 article talking about James Harden’s outstanding performance in multiple aspects of a 2018 NBA game. It’s also a result of my musings on the cool bubble charts that Alicia Bay made last season. When looking at multi-dimensional data such as these, it can be difficult to quickly determine the “best” performances. If you choose just a single metric to sort performances with, you may miss teams that sacrificed some scoring in that metric to score more points elsewhere. One solution to this problem is to look at the Pareto-optimal performances.
The concept of Pareto-optimality was originally developed for economics. The idea is to find the frontier where any additional output from one metric will come at the expense of another. Here’s a nice visual representation I stole from wikipedia charting economic production of 2 items:
All of the points A-H are Pareto-optimal. If you are at any of these points, there is no way to produce more of both item 1 and item 2. Choosing to increase the output of item 1 will come at the expense of the output of item 2 and vice versa. In contrast, points N and K are not Pareto-optimal. If you are at these points, you can produce more of Item 1 and of Item 2 simply by moving to point D or point E. Obviously there are ways to push past these boundaries with more raw materials or better processes, but if those are fixed, then the best you can do is to be on this frontier.
Pareto-optimality is a useful concept because it helps to identify trade-offs. These trade-offs are rarely 1 to 1, and choosing whether or not to make a trade-off will depend on your specific wants and needs. Sometimes taking a slight step back in production of one item will allow for drastic increase in the production of another, making the trade-off beneficial. Other times, it will require a massive sacrifice in one area just to get a minor increase in the other, meaning you should not make that trade-off.
Another use of Pareto-optimality is to identify processes that are strictly superior to your own and change your process to be more in line with another. If you were at point K or point N on the above chart, instead of pondering trade-offs, it might be insightful to research processes D and E. Adapting those processes may allow overall performance increase without trade-offs.
Back to FRC
Okay, so moving back to FRC, the first question we should ask is which metrics should we look at for a team’s performance? Ideally we’d like to split the game into small pieces to see where performance tradeoffs lie. Elo and OPR are holistic indicators of team performance, so they won’t work so well. I’ve decided to go with hatch panel points, cargo points, and endgame points. Notable missing ways to score points are in sandstorm and penalty points, which I’ve chosen to neglect due to minimal point value and inconsistency respectively. Note that the chart above only shows Pareto-optimality with 2 inputs, but the same principles can be applied to an arbitrary number of inputs.
I’ve pulled the calculated contribution to each of these 3 categories for all 2019 events. There were 18 Pareto-optimal performances using these categories. Here they are sorted by endgame points:
|team Number||Nickname||Event||hatch Panel Points||cargo Points||endgame Points|
|525||Swartdogs||Northern Lights Regional||3.44||9.87||14.00|
|846||The Funky Monkeys||Silicon Valley Regional||5.03||12.27||13.37|
|359||Hawaiian Kids||Hawaii Regional||5.36||15.57||12.82|
|1305||Ice Cubed||ONT District Georgian College Event||6.96||11.42||12.50|
|5199||Robot Dolphins From Outer Space||Los Angeles Regional||5.65||11.56||12.41|
|604||Quixilver||Silicon Valley Regional||7.16||12.69||12.12|
|33||Killer Bees||FIM District Center Line Event||8.02||10.88||12.11|
|2767||Stryke Force||Curie Division||7.12||15.34||11.97|
|1986||Team Titanium||Seven Rivers Regional||6.07||15.59||11.95|
|1323||MadTown Robotics||Sacramento Regional||11.72||22.60||11.87|
|2056||OP Robotics||ONT District McMaster University Event||13.57||18.54||11.56|
|3061||Huskie Robotics||Seven Rivers Regional||13.90||3.88||7.96|
|2910||Jack in the Bot||Newton Division||13.57||16.12||7.93|
|2056||OP Robotics||FIRST Ontario Provincial Championship – Technology Division||12.56||18.92||7.64|
|1323||MadTown Robotics||Newton Division||13.22||22.70||7.58|
|2910||Jack in the Bot||Pacific Northwest FIRST District Championship||13.78||16.88||6.25|
And here’s the same data represented in a bubble chart:
Analysis and Conclusions
There’s a couple of interesting points in these data. First, there is a very clear drop-off for endgame points at the last 6 entries here. All teams above that were at or around 12 points per match in endgame points, which would be a full level 3 climb. The teams below were all either at a championship-level event or were 3061 (more on them in a second). The last team in the top-tier of endgame points, 2056 at McMaster, drives much of the weird shape in the above chart. In addition to being on the upper end of endgame points, this performance was also 4th highest in hatch panel points and 3rd highest in cargo points. Only being outclassed in cargo by 1323 and only being outclassed in hatch panels by 2910 and 3061.
3061 is a really fascinating entry on this list, and exemplifies trade-offs well. Every other team here made it all the way to champs, but 3061 got knocked out in the semi-finals at Seven Rivers. I don’t mean this as a knock on 3061, in fact, quite the opposite. 3061 didn’t even make playoffs at any of their events in 2017 or 2018, so 2019 was a huge step up for them. They clearly made the choice to specialize heavily in hatch panels to the complete exclusion of cargo and Level 3 climbs, and it paid off big time for them. Plus, they get to be on this list right alongside household names like 2056 and 1323. 3061 would seem to make an excellent case study for Karthik’s Golden Rules.
I’m tentatively planning to automatically calculate Pareto-optimal performances next year in my scouting database. I’ll likely use 3 or 4 key game aspects depending on the game.
That’s all for now,