Online games are complex, emergent systems that are difficult to design and test. This difficulty invariably leads to failures being present in the game, negatively impacting the play experience of some. We present a taxonomy of possible failures, divided into temporal and non-temporal failures.
The taxonomy can guide the thinking of designers and testers alike, helping them expose bugs in the game.
Most gamers think game bugs are bad. Bugs can cause a game to crash, disturb game balance, or ruin an engaging game session. However, there is a sub-community of gamers who actually like bugs. Not in the resigned sense of, “I like this game, bugs and all.” Rather, in the more active sense of liking to find bugs, being motivated by the potential that a game bug might exist, and hence so motivated as to spend hours trying various quirky button and movement combinations in hopes of revealing a game bug. For these select few, game bugs are a passion, a hobby, with one imperative: videos showing bugs must be posted on YouTube.
YouTube’s bug videos are a rich resource that mix creativity, subversiveness and pure chance. The videos available provide a startling amount of coverage; far more than any single research group could ever hope to expose personally.
We became motivated to understand the variety of online game bugs, which could form the groundwork for unravelling the complexities of testing and verifying modern, emergent gameplay. YouTube presented a perfect starting point.
After watching many videos demonstrating curious bugs across a wide range of games and genres, we began to notice patterns emerging. Several of the videos showed bugs pertaining to defects in collision detection, while others had unexpected emergent behavior of physics engines, and so forth. These similarities indicated that a categorization of these buggy gameplay experiences was possible, leading to new methods of analyzing and solving online game bugs.
In some areas, particularly that of computer science, taxonomies have been used to create an organized, systematic approach to understanding a wide variety of data. The introduction of a bug taxonomy aided testers in focusing their thoughts, allowing them to generate new test cases and creating greater coverage of the system. Such taxonomies can also be used to validate test plan coverages and help newcomers understand the problem areas.