Games User Research: Informing Design

Games live and die commercially on the player experience. Games User Research is collectively the way we optimise the quality of that user experience in games, working with all aspects of a game from the mechanics and interface, visuals and art, interaction and progression, making sure every element works in concert and supports the game user experience (UX). This means that Games User Research (GUR) is essential and integral to the production of games and to shape the experience of players. Today, Games User Research stands as the primary pathway to understanding players and how to design, build, and launch games that provide the right game UX.

Games User Research (GUR) is an interdisciplinary field of practice and research concerned with ensuring the optimal quality of usability and user experience (UX) in video games. This means that GUR inevitably involves any aspect of a video game that players interface with, directly or indirectly: from controls, menus, audio, and artwork to the underlying game systems, infrastructure, as well as branding, customer support, and beyond. Essentially, any aspect of a video game that influences the user’s experience and perception of that game is of concern for an investigative GUR practice. This makes GUR a field that interfaces with more or less every other area of game development.

GUR is the field that helps us figure out if the experiences we hope to give our players are what we are indeed delivering, because GUR focuses on the players and their experience playing games, and this is at the heart of all games. In practice, GUR production uses methods from many research fields, including human-computer interaction, human factors, psychology, design, graphics, marketing, media studies, computer science, analytics, and other disciplines to deliver robust tests to assess all aspects of UX in a game. Contrary to the domains of QA and technical game testing, GUR is focused completely on evaluating players (based on observation of them playing or otherwise interacting with the game and its components, and analysis of the data they generate). GUR practitioners rely on experience analysis and on understanding player interaction. Their objective is not simply testing the player, but improving all aspects of a game’s design through building empirical evidence via experimentation and testing. How to do this in practice is an interesting challenge—games are intricate, interactive computational systems, where engagement is an important factor.

GUR methods are evolving constantly, and user testing is now commonplace in the games industry. The steady increase in the size of the target audience for games, as well as its increasing diversification, has led to a stronger need for GUR. This has brought an opportunity for the industry to innovate on different forms of play, allowing different types of interactions and contexts, and the accommodation of different types of users of all ages,  abilities, and motivations.

GUR methods are evolving constantly and user testing is now commonplace in the games industry, which globally has an annual revenue of over US$100 billion (outselling the motion picture and music industries combined), with billions of players across any culture and demographic. With such a massive and diverse audience, to make this industry a success, users have become more and more integrated into game development. The steady increase in the size of the target audience for games, as well as its increasing diversification, has led to a stronger need for GUR. This has brought an opportunity for the industry to innovate on different forms of play, allowing different types of interactions and contexts, and the accommodation of different types of users of all ages, intellectual abilities, and motivations. Now, more than ever, it is necessary for designers to develop an understanding of the users and their experiences of interacting with games.

Case Study: Evaluating the Onboarding Phase of Free-to Play Mobile Games 

In recent years, alongside technological advancements, the Freemium business model with the Free-to-Play (F2P, FtP), revenue model has come to dominate the mobile game market.  F2P functions by initially offering the game to the player for free, generating revenue via in-game advertisements and purchases

Although the model is attractive, due to the low financial barrier of entry for new players, developers face many challenges, such as discoverability issues caused by saturation in the F2P market across Android and iOS platforms. Moreover, the first few minutes of play, also referred to as the onboarding phase, is highly critical for F2P games. As they are also characterized by low player retention rates (i.e., most players that leave within a few minutes of play never return). While specific rates vary from game to game, and there are few verified data sources available on the topic, one general estimate suggests that less than a third of players return to a game after the first day, and retention rates drop exponentially as a function of the time since the player was last in contact with the game

The first few minutes of play, commonly referred to as the onboarding phase, of Free-to-Play mobile games typically display a substantial churn rate among new players. It is therefore vital for designers to effectively evaluate this phase to investigate its satisfaction of player expectations.

In the case study presented here, a lab-based mixed-methods approach was developed to specifically investigate the user experience of the onboarding phase in mobile games. Multiple methods were employed, including Heart-Rate Variability and Galvanic Skin Conductance) as well as a range of self-reported proxy measures including: a) stimulated recall, engagement graphs, b) flow state survey and c) post-game experience questionnaire.

These techniques were applied across 28 participants using three mobile Free-to-Play titles from different genres. The experimental setup was based on a within-subject design where each participant was exposed to three different mobile game onboarding phases.

The work provides two important contributions to Games User Research:

1) Evaluation of different research techniques (e.g. physiological measures and experience graphs) in the context of mobile games. The results indicate that physiological measures can be applied to evaluate mobile games, similar to PC and console game contexts, as well as for mobile productivity applications, though evidence for this use case is highly limited. While the event-dependent activation of GSR and HRV signals appears similar to those for other game formats, it is worth noting that the arousal-related signals were generally of lower amplitude than reported for other game formats

2) A framework for building data-driven recommendations on design elements that result in high and low arousal. During the initial research for this study, poorly designed onboarding phases, amongst others, were identified as one of the main reasons for the typical high churn rate of F2P mobile games. One of the goals of this study was to create a test setup that could be used to collect First-Time User Experience (FTUE) data and provide game designers with a tool to improve this phase. During the analysis of the physiological data, average player arousal graphs were created and compared to the designers’ intended user experience graphs, visualizing player experience. This comparison has the potential to help game designers improve the onboarding phase of their games highlighting differences between actual player arousal and the intended design.

The focus of the project was to evaluate the adaption of physiological, qualitative, and self-report techniques to evaluate UX in the onboarding phase of three F2P mobile games. A number of experiences were drawn from the methodological implementation that bears importance on future work regarding the adoption of physiological measures and user experience evaluation in mobile game contexts.

The three games used in the project: Pogo Chick, Candy Crush Jelly Saga and WinterForts. The games represent three different genres of mobile games. All three are designed with a Freemium business model in mind.  

See below for a range of case studies in Games User & -Experience Research 

Exploring the relationship between video game expertise and Fluid Intelligence

Hundreds of millions of people play intellectually-demanding video games every day. What does individual performance on these games tell us about cognition? Here, we describe two studies that examine the potential link between intelligence and performance in one of the most popular video games genres in the world (Multiplayer Online Battle Arenas: MOBAs). In the first study, we show that performance in the popular MOBA League of Legends’ correlates with fluid intelligence as measured under controlled laboratory conditions. In the second study, we also show that the age profile of performance in the two most widely-played MOBAs (League of Legends and DOTA II) matches that of raw fluid intelligence. We discuss and extend previous videogame literature on intelligence and videogames and suggest that commercial video games can be useful as ‘proxy’ tests of cognitive performance at a global population level.

Correlation between Heart Rate, Electrodermal Activity and Player Experience in First-Person Shooter Games

Psychophysiological methods are becoming more popular in game research as covert and reliable measures of affective player experience, emotions, and cognition. Since player experience is not well understood, correlations between self-reports from players and psychophysiological data may provide a quantitative understanding of this experience. Measurements of electrodermal activity (EDA) and heart rate (HR) allow making inferences about player arousal (i.e., excitement) and are easy to deploy. This paper reports a case study on HR and EDA correlations with subjective gameplay experience, testing the feasibility of these measures in commercial game development contexts. Results indicate a significant correlation (p < 0.01) between psychophysiological arousal (i.e., HR, EDA) and self-reported gameplay experience. However, the covariance between psychophysiological measures and self-reports varies between the two measures. The results are consistent across three different contemporary major commercial first-person shooter (FPS) games (Prey, Doom 3, and Bioshock).

Online-only friends, real-life friends or strangers? Differential associations with passion and social capital in video game play

The present study tests a recently proposed model in which social video game play supports wellbeing by contributing to a harmonious type of engagement with the game. Players (N = 2030) of the online-only multiplayer first-person shooter game, Destiny, reported the frequency they played with real-life friends, online-only friends and strangers, their type of engagement with the game – measured as harmonious and obsessive passion, and completed a wellbeing measure of social capital. Telemetry data also recorded their total time playing over the duration of the study. A structural equation model supported the prediction that harmonious – but not obsessive – passion would mediate the positive association between playing with others and social capital. The findings also supported a supplementary hypothesis that the three types of social relationships would be differentially associated with two forms of social capital – bridging versus bonding – as a function of the closeness of social ties. Real-life friends was positively associated with bonding, strangers with bridging, and online-only friends with both. Overall, these results emphasise that social interactions in (and around) online multiplayer video games are effective for building social capital, and do so by ensuring game play is in harmony with other goals and values.

Play With Me? Understanding and Measuring the Social Aspect of Casual Gaming

Social Gaming is a pervasive phenomenon, driven by the advent of social networks and the digitization of game distribution. This paper positions and defines Casual Social Games (CSGs) as a genre and platform agnostic subset of Social Games that incorporates browser, mobile, console and wearable digital games. Here we argue that – as CSGs impact the games industry, shape play patterns and audience characteristics, and proliferate to new platforms – understanding and measuring their social aspect becomes highly relevant. A randomized experiment on added social gameplay in a CSG on both mobile and Facebook serves to support this argument. Experimental results highlight that social gameplay is extremely important for engagement and monetization in casual games, even more so on mobile platforms. This does not only suggest that CSG developers will benefit from focusing on increased social interaction in their games, but that Game Analytics should strive to unify definitions and build a common body of knowledge around the social aspect of casual gaming.

Methods for Evaluating Gameplay Experience in a Serious Gaming Context

Gameplay experience (GX) is created during the process of player-game interaction, where this interaction has the goal to provide a motivating, fun experience for the player. Since GX is an important factor for the success of failure of a game, a formal classification of how to design for and evaluate GX is necessary. Using appropriate mechanisms for evaluation and measurement of GX allows the validation of good gameplay experiences. This paper presents an approach to formalize such evaluative methods and a roadmap for applying these mechanisms in the context of serious games. We first discuss related work of user experience (UX) and player experience models, based on which we propose a three-layer framework of GX. For each layer, a number of measurement methodologies are listed and our focus is put on physiological and technical metrics for game evaluation. Finally, we point out the potential use of this framework within the field of game-based learning and serious gaming for sports and health.

Game Time: Modeling and Analyzing Time in Multiplayer and Massively Multiplayer Games

Game time is a core feature of game design and study, and forms part of the gaming experience on a variety of levels. It can be viewed from multiple perspectives, for example, the time of the playing of the game or the flow of time in a game world. In this paper, a comprehensive game time model based on empirical research as well as recent theory is presented. It proposes various perspectives on game time and integrates them to allow coherent representation of the same events in the different perspectives. The model has been tested across tabletop and digital formats, and its applicability across game formats is demonstrated. Emphasis is placed on multiplayer and massively multiplayer role-playing games because these feature complex game time behavior not previously evaluated. The model considers game time as an interactively created and nonlinear feature of games and game play.