Activision Secretly Experimented on Half the Players of Call of Duty: Modern Warfare 3. The Goal Was to Confirm What Had Long Seemed Obvious
Activision published an extensive document describing the importance of skills during matchmaking. It shows that without the so-called SBMM, players are more likely to become discouraged.
The basis of balanced gameplay in competitive online games is the appropriate selection of opponents. It usually occurs based on the players' skills, and this system is known as SBMM (skill-based matchmaking). Of course, some games don't use it, such as the recently released XDefiant, but in general, it is present in most titles of this type, including the Call of Duty series, which not everyone likes.
Critics of SBMM argue that this system creates a level playing field in every match, requiring a strong effort to win. Some people play only for fun and occasionally, which means that they don't want to break a sweat and believe that the experience would be much more enjoyable by randomly selecting players.
Activision Experiment
In January, Activision revealed on what basis matchmaking works and it turned out that the main factors are as many as 8, with the most important being the quality of the connection and the timing of the match. Now the studio has published a large document (25 pages!) titled "The Role of Skill in Matchmaking," from which we learn about an interesting experiment conducted at the beginning of the year in Modern Warfare III.
It turns out that during this period Activision secretly reduced the importance of skill balance in matchmaking for 50% of the North American community.
The experiment's outcome shouldn't be surprising - more than 90% of players who unknowingly tested the mentioned matchmaking played Call of Duty less frequently, and the top 10% of players experienced no change. Therefore, this is evidence that SBMM positively affects the enjoyment of gameplay. Random selection is only beneficial for the best players, and among average players, it causes frustration and leads to quitting the game.
We have found that balancing skill against other matchmaking factors quantifiably increases the extent to which most players play and enjoy Call of Duty. When skill is utilized in matchmaking, 80-90% of players experience better end-of-match placement, stick with the game longer and quit matches less frequently.
The entire article significantly expands on this topic, so if you are interested in more detailed data (you can, for instance, learn how a player's "skills" are calculated), I recommend taking a look at it. The conclusion, however, is that without SBMM, online shooters wouldn't engage us for so long and wouldn't make us want to play another match after each one.
