Ben Brode Explains Why Even "Simple" Solutions Take Time - Ladder Revamp
There have been a few posts out there asking why it took so long to arrive to these "simple" ladder changes that we'll see come March. Ben Brode answers!
- They go through multiple rounds of brainstorming and testing.
- The hardest part isn't always the implementation itself, it can be the testing itself.
- Simulations need to be run to see how things may look after certain amounts of time. "What's the experience like for players at each rank?"
Quote from Ben Brode
The first phase is coming up with the right solution. Often when you land on the right solution, it feels "obvious". Derek's GDC presentation has some great examples of hard problems that look like they had obvious solutions in hindsight.
We actually went through multiple rounds of brainstorming and testing pitches – everything from completely changing the system, to subtly tweaking it to reach our goals.
Once we have a pitch, we model it. We run a simulation and test what the results will be after 1, 2, 3, 6 months, and a year. Does it spread players out more, to improve matchmaking? How much rank inflation does it cause over time? What's the experience like for players at each rank? We have a team of Data Scientists who try and model player behavior to give us as accurate a picture as we can, to help make the right decision.
Once we have a design we are confident in, then it's ready for implementation. Often during the implementation process we identify edge cases or other snags. For example: We can't line a patch up to launch at midnight in all regions and all platforms on March 1st, so we need to have both ranked play systems live in the client, and build a way to transition between them at the exact right moment. The current design requires a significant change to how players earn their monthly ranked rewards, and we added several warnings and UI call-outs to help reduce the chance players miss out on their card back for March. We had to change the victory screen and the Quest Log screen to accommodate the new flow.
But implementation isn't the biggest factor in a system like this. In this case, it was testing. When we create a new card, it's pretty straightforward to make sure that it works. One tester plays the card in a large variety of weird situations and logs bugs related to it. Our automation engineers force it to be played millions of times and check to make sure the servers don't explode. But a system like this presents a more challenging testing requirement. It involves large numbers of players, normal season resets, feb->march season resets, and coordination between developers, testers, and automation engineers to make sure everyone has the tools they need.
I'm glad people are enjoying the changes – please make sure to give us feedback after you play with it!