Player Profiling – with reading list

Reading list: see below.

Report on behavioral profiling: accessible via emailing me (see contact pane)

Slides from the IE Gaming Analytics Summit 2016: here.

The game industry is facing a surge of data, which results from increasingly available highly detailed information about the behavior of software and software users. The data can come from a variety of channels, e.g. behavioral telemetry, user testing, surveys, forums …, be high-dimensional, time-dependent and potentially very large. The old adage of big data having volume, velocity, variety and volatility holds very true for behavioral telemetry from games.

Profiling users has emerged across multiple data science application areas as a way of managing complex user data, and to discover underlying patterns in the behavior of the player base. Profiling users allow for a condensation and modeling of a complex behavioral space. Profiles allows us to consider players in a non-abstract, quantifiable way. Building an understanding about who the players are and how they will play, or are playing, the game.

In this post the background of user profiling will be outlined, and we will present a brief overview of the many different types of profiles that can be generated in game development contexts. Naturally, different types of profiles are used for different kinds of problems, have various strengths and weaknesses, etc. The post also forms an unofficial introduction to my talk at the Gaming Analytics Summit in London and you can also find the post on the summit website. In the post I am building on a lot of work by fellow analysts in industry and academia, and will later publish a reading list for those interested in the details of the different profiling techniques – this is just a brief overview. In the meantime, check out e.g. this paper for an introduction to clustering for profiling and this one for an example in action.

The idea of using customer data to inform marketing and product design has an extensive history in Information Science, where user profiling was developed to deal specifically with the problem of data overload. This is a prevalent issue in any user/customer-focused industry today, and certainly so in games. We easily extract dozens to hundreds of features from direct user-game interaction, and supplement these with data from marketing, attribution, playtesting, social networks and more. To makes things even more challenging, data are usually collected from large numbers of players, from potentially long-term interaction periods, and are typically temporally volatile.

Profiling permits a condensation of the behavioral space so any patterns can be located or hypotheses tested. These are then refined into a format where action can be taken on them, e.g. profiles that describe behavior, and potentially draw inferences as to the root causes of behavior. In games, there are typically two overall goals with player profiling:

1) Correlational: To correlate profiles with specific behaviors such as game completion potential, user experience, monetization, churn, retention, cross-game transportation, cross-promotion, social influence, etc.

2) Inferential: To investigate how and why specific behaviors occur as a function of user traits and/or behaviors.

We can also consider how profiles are developed, usually either bottom-up or top-down. The former is explorative, focused on locating patterns we did not know existed. This approach is useful as soon as we have data (beta, soft-launch etc.), and is usually feature intensive. Top-down profiling focuses on testing hypotheses, e.g. how valid already established profiles are given a new player cohort. This approach is useful post-launch, notably for consistency-testing profiles.

Profiles can be generated either to target individual or groups of players. Individual profiles seek to discover characteristics of specific people, and is based on data from only that person. Group profiling, which is vastly more common, tries to categorize individuals as a kind of individual – i.e. a type or group. Group profiling is less precise than individual profiling but what we need in practice to manage high-dimensionality datasets. Every group profile will have a fit which is the quality of the profile in terms of what it is applied to. Fit is an important component to integrate when considering how to distribute players into profiles, or taking action on players who fall into specific profiles. If a profile is 100% distributive it means that all properties applies fully to everyone in a group, e.g. “all bachelors are unmarried”. In practice, analytically generated group profiles are non-distributive to a greater or lesser degree – and often the latter. This is key concern when considering how to act on profiles, and means that techniques such as soft clustering – which groups people according to their distance to multiple cluster centers – have value in the daily practice of player profiling. In general, the more detailed we try to make profiles, the less players they apply to, and there is definite element of cost-benefit balancing in play here.

And finally, just to make sure we have the bases covered, player profiles can also be considered based on the information they are built from. Two core types are protean and data-driven profiles. The latter is based on actual behavioral, attitudinal or other data, while the former is based on theoretical models and design. Data-driven profiles are based on quantitative data, and can be developed from the earliest user testing – they are ideally updated throughout production and during post-launch. Protean profiles are based on theoretical models and commonly used in design. They can be defined from day 1 but importantly must be kept updated to remain useful, which means feeding in design changes and user testing data on a continual basis. They must also be integrated across the team to ensure coherence in their use.

The process of building profiles rests on well-established guidelines for knowledge discovery in IT, irrespective of the specific algorithms or models used for pattern recognition (ranging from simple but effective tools such as cohort analysis to machine learning). In Information Systems, these four basic steps are commonly used:

1) Discovery: A knowledge discovery process is performed to provide sets of correlated data for profiling, i.e. information about which patterns and correlations we see in the data. For example, that kill/death ratio appears to be important to progression in a FPS.

2) Selection: We decide which patterns to use and which behaviors to employ in the further work with developing profiles. For example, if we are interested in churn, we use patterns showing correlations between behavior and players leaving/staying in the game. Via experimental work we can also investigate causal relationships. Various types of machine learning algorithms can be employed to search the variance space, with clustering being a popular example.

3) Interpretation: In this step we define the profiles. This can be done in a variety of ways, but a sharp eye on the application is important. This is an often over-looked or under-prioritized phase leading to problems in the fourth step.

4) Application: This vital step involves taking action on the information contained in the profiles. This step is possibly the most difficult to execute in practice at it often involves communication between stakeholder groups that speak completely different languages.

The process is of course cyclic. Players change behavior, the composition of the population changes over time, as does game design in persistent or semi-persistent game. So profiles should be continually updated. We are never finished with profiling our players. It is also worth noting that profiling at all levels is not an objective process. We always make choices, the algorithm or model, how data are pre-processed, outcomes interpreted etc. Because of these choices, there is the potential for bias and bad decisions at all of these steps.

Focusing on data-driven profiles, there are a number of different types common in game analytics these days, including:

Snapshot profiles

Snapshot profiling is focused on developing an understanding of the patterns of behavior as they occur at the operational level. The data used for snapshot profiling are typically aggregate metrics about the players and/or their behavior. Generally, historical data are not used but rather information about the state of the players at the present. Typical examples include dimensional reduction of high-variety datasets about player characters in Massively Multi-Player Online Games (MMOGs) or other online multi-player games, in order to obtain an understanding about the composition of the current player base.

Dynamic profiles

Player behavior change as a function of time. Furthermore, persistent games which sees the same players interacting with a game over potentially long temporal periods, experience a constant change in the population of players. This is notably the case for games which have persistence as a key design factor in order to support F2P business models. Games themselves can also change over time, for example via patches, updates or expansions. These three factors jointly mean that profiles generated based on snapshot data have a limited period during which they are valid as representations of the player base. It is therefore increasingly common to see player profiles being iteratively generated as a function of set time intervals, e.g. 24 hours. While the underlying unsupervised machine learning methods are similar for snapshot and dynamic profiling, the latter are constantly regenerated and additionally permit historical viewpoints on changes in the behavior of the players (or systems), and also acts as a starting point for predictive analytics.

Predictive/Historical Profiling

Building predictive models is a key area in business intelligence in general, and predicting player behavior important in persistent games of any kind. As we all know, F2P games generally see only a small fraction of the players stay engaged with the game for a long time, and a similarly small fraction monetize via IAPs. Therefore, predicting which players that will stay engaged and/or monetize (building implicit profiles), or otherwise provide value, is of key concern in this sector.

Psychological Profiling

Telemetry data are only one source of information about players, and a relatively recently introduced one at that. While the focus here is on telemetry-driven approaches, it should be mentioned that behavioral profiling has an extended history based on information derived from user-testing, surveys, marketing data, etc. The idea of tying in observations from gameplay to profiling, or use gameplay behavior as the sole basis for profiling, was introduced much later. Models focusing on player motivations, personality, etc. have been around for over a decade, but only recently have we started correlating these with telemetry data. Similarly, the idea of using telemetry data to draw inferences about player psychology is also relatively recent, and there is limited publicly and/or systematic knowledge available at the time of writing.

On a final note, it is useful to consider spatio-temporal perspectives in player profiling. All games contain spatial and temporal elements, whether digital or non-digital. Especially in 3D digital games, the spatial dimension is often important to the perceived experience of the game. Furthermore, spatial navigation and positioning are key gameplay elements in many games. A number of approaches for e.g. trajectory analysis and –classification have been adapted for use in Game AI and in Game Analytics, used e.g. to detect bot programs, study player tactics or to train AI bots. Behavioral analysis can be carried out without considering the temporal and spatial dimensions of play, however, it is often necessary to include one or more of these in order to build the required insights. Snapshot profiling can be done without historical or spatial data, whereas dynamic profiling invariably ends up providing temporal patterns. Similarly, predictive modeling requires temporal information. Neither profiling approach requires spatial information, however. Spatial behavioral data are usually only included when needed given the purpose of the analysis. Additionally, spatio-temporal game analytics can be cumbersome and require that interpretation is performed in relation to the actual virtual environment. Ignoring this step in the analysis cycle lead to the risk of misinterpretation of the root causes of the observed behaviors.

Summarizing, player profiling is an incredibly useful tool as it allows us to bridge the gap between the users and analytics. Profiles can provide a deep understanding of the players, and serve as the basis for a range of analytical techniques including prediction. There are a treasure trove of techniques available which spans the range from descriptive methods to advanced machine learning, and this means that profiling as an exercise to glean value from player data is open to everyone. In practice, we can get very far with simple methods, even in complex situations. Profiling requires control of every step of the process however, and well-considered application of the profiles in practice.

Reading list

Behavioral profiling forms one of the core challenges of game analytics. This because behavioral profiling condenses what can be very varied (high-dimensional), volatile and potentially high volume data about the behavior of players within the confines of a game into descriptions that highlight the patterns of player behavior. This post is a reading list of work from academia and industry on various types of profiling techniques, methods, studies, issues etc. Some articles are peripheral but interesting to the core topic. The list is not comprehensive – there is a lot of work relevant to behavioral profiling in and around games that are not included, but I had to stop somewhere. If you think an important reference is missing, please send me an email, comment, tweet or similar, and I will add it in. 

For those outside of academia, please note that a lot of these articles are behind publisher paywalls, but can be accessed through any university library. A lot are also accessible via Google Scholar.

References on player modeling

  1. O. Missura and T. Gärtner. Player modeling for intelligent difficulty adjustment. In Proc. of the ECML-09 Workshop From Local Patterns to Global Models, 2009.
  2. R. Thawonmas and K. Iizuka. Visualization of online  game  players based on their action behaviors. Int. Journal of Computer Games Technology, 2008.
  3. C. Thurau, K. Kersting, and C. Bauckhage. Convex  Non-Negative  Matrix Factorization in the Wild. In Proc. IEEE Int. Conf. on Data Mining, 2009.
  4. B. Weber and M. Mateas. A Data Mining Approach to Strategy Prediction. In IEEE Symposium on Computational  Intelligence  in Games, 2009.
  5. B. G. Weber, M. John, M. Mateas and A. Jhala. Modeling Player Retention in Madden NFL 11. In Proceedings of IAAI, 2011.
  6. J.-K. L. R. Thawonmas, K. Yoshida and K.-T. Chen. Analysis of revisitations in online games. Journal of Entertainment Comp., 2011.
  7. Ong, H. Y., Deolalikar, S. and Penge, M. V. (2015) Player Behavior and Optimal Team Composition in Online Multiplayer Games. Stanford University, arXiv:1503.02230 [cs.SI]. Retrieved from: http://arxiv.org/abs/1503.02230
  8. Gao, L., Judd, J., Wong, D., Lowder, J. (2013). Classifying dota 2 hero characters based on play style and performance. Report for Univ. of Utah Course on ML. Retrieved from: http://spotidoc.com/doc/163929/classifying-dota-2-heroes-based-on-play-style-and-perform…
  9. Hadiji, F., Sifa, S., Drachen, A. and Thurau, C. (2014). Predicting Player Churn in the Wild. Proceedings of the IEEE Computational Intelligence in Games, Bonn, Germany.
  10. Runge, J., Gao, P., Garcin, F., and Faltings, B. (2014). Churn Prediction for High-value Players in Casual Social Games. In Proceedings of the IEEE Computational Intelligence in Games, Bonn, Germany.
  11. C. Thompson. Halo 3: How microsoft labs invented a new science of play. Wired Magazine, 15(9).
  12. T. Fields and B. Cotton. Social Game  Design:  Monetization  Methods and Mechanics. Morgan Kauffman Publishers, 2011.
  13. J. H. Kim, D. V. Gunn, E. Schuh, B. C. Phillips, R. J. Pagulayan, and D. Wixon. Tracking real-time user experience (true): A comprehensive instrumentation solution for complex systems. In Proc. of CHI, 2008.
  14. L. Mellon. Applying metrics driven development to MMO costs  and  risks. Versant Corporation, 2009.
  15. G. N. Yannakakis. Game AI Revisited. In Proceedings of ACM Computing Frontiers Conference, 2012.
  16. G. Zoeller. Game Development Telemetry. In Proceedings of the Game Developers Conference, 2011.
  17. G. N. Yannakakis and J. Hallam. Real-time Game Adaptation for Optimizing Player Satisfaction. IEEE Transactions on Computational Intelligence and AI in Games, 1(2):121-133, 2009.
  18. Eggert, C., Herrlich, M., Smeddinck, J. and Malaka, R. (2015). Classification of Player Roles in the Team-Based Multi-player Game Dota 2. Proceedings of Entertainment Computing, LNCS 9353, 112-125, Trondheim, Norway.
  19. Gao, L., Judd, J., Wong, D., Lowder, J. (2013). Classifying dota 2 hero characters based on play style and performance. Report for Univ. of Utah Course on ML. Retrieved from: http://spotidoc.com/doc/163929/classifying-dota-2-heroes-based-on-play-style-and-perform…
  20. Ong, H. Y., Deolalikar, S. and Penge, M. V. (2015) Player Behavior and Optimal Team Composition in Online Multiplayer Games. Stanford University, arXiv:1503.02230 [cs.SI]. Retrieved from: http://arxiv.org/abs/1503.02230
  21. F. Southey, G. Xiao, R. C. Holte, M. Trommelen and J. Buchanan. Semi-Automated Gameplay Analysis by Machine Learning. In proceedings of AIIDE, 2005.
  22. C. Thurau, K. Kersting, and C. Bauckhage. Convex  Non-Negative  Matrix Factorization in the Wild. In Proc. IEEE Int. Conf. on Data Mining, 2009.
  23. A. Drachen, G. N. Yannakakis, A. Canossa and J. Togelius. Player Modeling using Self-Organization in Tomb Raider: Underworld.  In  Proc. of IEEE Computational Intelligence in Games, 2009.
  24. J. Bohannon. Game-Miners Grapple With Massive Data. Science, 330(6000):30-31, 2010.
  25. D. Nozhnin, Predicting Churn: When Do Veterans Quit?, Gamasutra, August 30th, 2012. URL: http://www.gamasutra.com/view/feature/176747/predicting_churn_when_do_veterans_.php
  26. Sifa, R.; Bauckhage, C. and Drachen, A. The Playtime Principle: Large-Scale Cross-Games Interest Modeling. In Proceedings of the IEEE Computational Intelligence in Games, 2014.
  27. W. Feng, D. Brandt, and D. Saha, “A Long-term Study of a Popular MMORPG,” in Proc. of the 6th ACM SIGCOMM Workshop on Network and System Support for Games, 2007.
  28. Drachen, A.; Thurau, C.: Sifa, R. & Bauckhage, C. (2013): A Comparison of Methods for Player Clustering via Behavioral Telemetry. In Proceedings of Foundations of Digital Games 2013 (Chenia, Greece). Society for the Advancement of the Science of Digital Games (SASDG) Publishing.
  29. Drachen, A.; Sifa, R. & Thurau, C. The Name In the Game: Patterns in Character Names and Gamer Tags. Invited Paper. In Entertainment Computing (Elsevier Publishers), vol 5, issue 1, pp. 21-32. DOI: 10.1016/j.entcom.2014.02.001
  30. Bauckhage and SIfa 2015, k-maxoids clustering using vehicle usage data from Battlefield 3, identifying seven clusters of behavior showcasing preferences for vehicles
  31. Bauckhage, C., Drachen, A. and Sifa, R.: Clustering Game Behavior Data. IEEE Transactions of Computational Intelligence and AI in Games, Special Issue on Game Analytics, vol 7, issue 3, pp. 266-278.
  32. Drachen, A.; Sifa, R.; Bauckhage, C. and Thurau, C. (2012): Guns, Swords and Data: Clustering of Player Behavior in Computer Games in the Wild. In proceedings of IEEE Computational Intelligence in Games, 2012 (Granada, Spain), IEEE Publishers, pp. 163-170. DOI: 10.1109/CIG.2012.6374152
  33. Yannakakis, G. N.; Togelius, J. (2015). A Panorama of Artificial and Computational Intelligence in Games. in IEEE Transactions on Computational Intelligence and AI in Games. 7(4), 2015
  34. Lim, C.-U.; Harrel. D. F. (2015). Revealing Social Identity Phenomena in Videogames with Archetypal Analysis, Proceedings of 2nd Player Modeling Workshop, AAAI Artificial Intelligence in Interactive Digital Entertainment (AIIDE) 2015.
  35. Suznjevic, M.; Stupar, I. and Matijasevic, M. (2011). MMORPG Player Behavior Model based on Player Action Categories. In Proceedings of the 10th Annual Workshop on Network and Systems Support for Games (NetGames), pp. 1-6
  36. Holmgård, C.; Liapis, A.; Togelius, J. and Yannakakis, G. N. (2015). Monte-Carlo Tree Search for Persona Based Player Modeling. Proceedings of 2nd Player Modeling Workshop, AAAI Artificial Intelligence in Interactive Digital Entertainment (AIIDE) 2015
  37. Normoyle, A; Jensen, S. T. (2015). Bayesian Clustering of PLayer Styles for Multiplayer Games. In Proceedings of the AAAI Artificial Intelligence in Interactive Digital Entertainment (AIIDE) 2015
  38. Müller, S.; Kapadia, M.; Frey, S.; Klingler, S.; Mann, R. P.; Solenthaler, B.; Sumner, R. W. and Gross, M. (2015) Statistical Analysis of Player Behavior in Minecraft. Foundations of Digital Games, 2015.
  39. EI-Nasr, M.S. and Drachen, A. and Canossa, A., Game Analytics: Maximizing the Value of Player Data. Springer, 2013.
  40. Drachen, A.; Yancey, M.; Maquire, J.; Chu, D.; Wang, Y. I., Mahlman, T.; Schubert, M. and Klabjan, D. (2014): Skill-Based Differences in Spatio-Temporal Team Behaviour in Defence of The Ancients 2 (DotA 2). In Proceedings of the IEEE Consumer Electronics Society Games, Entertainment, Media Conference.
  41. Drachen, A.; Baskin, S.; Riley J. and Klabjan, D. Going Out of Business: Auction House Behavior in the Massively Multi-Player Online Game Glitch. In Journal of Entertainment Computing (Elsevier Publishers) 5, pp. 20-31. DOI: 10.1016/j.entcom.2014.02.001.
  42. Sifa, R.; Drachen, A. & Bauckhage, C. (2013): Behaviour Evolution In Tomb Raider Underworld. In Proceedings of IEEE Computational Intelligence in Games 2013.
  43. W. Feng, D. Brandt, and D. Saha (2007). A Long-term Study of a Popular MMORPG. In Proceedings of  of the 6th ACM SIGCOMM Workshop on Network and System Support for Games, 2007.
  44. Sifa, R.; Bauckhage, C. and Drachen, A. The Playtime Principle: Large-Scale Cross-Games Interest Modeling. In Proceedings of the IEEE Computational Intelligence in Games, 2014.
  45. Davenport, T. H.; Harris, J. G. (2007). Competing on Analytics: THe new science of winning. Boston, Harvard Business School Press.
  46. Mahlman, T.; Drachen, A.; Canossa, A.; Togelius, J.; Yannakakis, G. (2010). Predicting player behavior in Tomb Raider: Underworld. In Proceedings of the International Conference on Computational Intelligence and Games (Copenhagen, Denmark).
  47. Sifa, R.; Hadiji, F.; Drachen, A. and Runge, J. (2015). Predicting Purchase Decisions in Mobile Free-To-Play Games. In Proceedings of AIIDE 2015, AAAI Publishers.
  48. Yang, P. Harrison, B. and Roberts, D. L. (2014). Identifying patterns in combat that are predictive of success in moba games. Proceedings of the Foundations of Digital Games, Miami, Florida, USA.
  49. Harrison, B.; Roberts, D. (2011). Using Sequential Observations to Model and Predict Player Behavior. In Proceedings of Foundations of Digital Games, pp. 91-98.
  50. Drachen, A. & Schubert, M. (2013a). Spatial Game Analytics and Visualization. In Proceedings of IEEE Computational Intelligence in Games, 1-8. DOI: 10.1109/CIG.2013.6633629.
  51. Drachen, A. & Schubert, M. (2013b). Spatial Game Analytics. In: Game Analytics – Maximizing the Value of Player Data, Seif El-Nasr, M.; Drachen, A. & Canossa, A. (Eds.). Springer Publishers, 365-402 [URL: http://www.springer.com/computer/hci/book/978-1-4471-4768-8%5D.
  52. Drachen, A.; Thurau, C., Yannakakis, G., Togelius, J. and Bauckhage, C. (2013). Game Data Mining. In: Game Analytics – Maximizing the Value of Player Data, Seif El-Nasr, M.; Drachen, A. & Canossa, A. (Eds.). Springer Publishers, 205-253.
  53. Bauckhage, C., Sifa, R., Drachen, A., Thurau, C. and Hadiji, C. F. (2014). Beyond heatmaps. spatio-temporal clustering using behavior-based partitioning of game levels. Proceedings of the IEEE Computational Intelligence in Games, Bonn, Germany.
  54. Rioult, R., Metivier J.-P., Helleu, B., Scelles, N., Durand, C. et al. (2014). Mining tracks of competitive video games. Proceedings of AASRI Conference on Sports Engineering and Computer Science, London, England.
  55. N. Hoobler, G. Humphreys, and M. Agrawala (2004). Visualizing Competitive Behaviors in Multi-User Virtual Environments. Proceedings of the Visualization Conference.
  56. Drachen, A. & Canossa, A. (2009). Analyzing Spatial user Behavior in Computer Games using Geographic Information Systems. In Proceedings of the 13th MindTrek 2009, 182-189.
  57. Miller, J. L. and Crowcroft, J. (2010). Group movement in world of warcraft battlegrounds. International Journal of Advanced Media and Communication, 4(4), 387–404.
  58. Drachen, A. and Canossa, A. (2011). Evaluating motion: Spatial user behaviour in virtual environments. International Journal of Arts and Technology, 4(3), 294–314.
  59. Vukovic, M.; Lovrek, I. and Kraljevic, H. (2012). Discovering shoppers´journey in retail environment by using RFID. In Grana, M. (Ed.) Advances in Knowledge-Based and Intelligent Information and Engineering SYstems, IOS Press. DOI: 10:3233/978-1-61499-105-2-857.
  60. Dodge, R.; Weibel, R. and Lautenschütz, A.-K. (2008). Towards a Taxonomy of Movement Patterns. Information Visualization, 7(3-4), 240-252, DOI:  10.1057/palgrave.ivs.9500182
  61. Andrienko, G.; Andrienko, N. (2008). Spatio-temporal aggregation for visual analysis of movements. In Proceedings of the IEEE Symposium on Visual Analytics Science and Technology, 2008. VAST ’08. pp. 51 – 58. DOI: 10.1109/VAST.2008.4677356
  62. Golub. G. and van Loan, J. (1996). Matrix Computations. John Hopkins University Press (3rd ed.).
  63. Han, J. and Kamber, M. (2006). Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers.
  64. Kersting, K.; Wahabzada, M.; Thurau, C. and Bauckhage, C. (2010). Hirerarchical Convex NMF for CLustering Massive Data. In Proceedings of ACML.
  65. Thurau, C.; Kersting, K. and Bauckhage, C. (2009). Convex Non-Negative Matrix Factorization in the Wild. In Proceedings of IEEE Conference on Data Mining.
  66. Jollie, I. (1986). Principal Component Analysis. Springer Verlag.
  67. Bartle, R. (1996). Hearts, clubs, diamonds, spades: players who suit MUDs. JOurnal of MUD Research 1(1), 1996.
  68. Bateman, C. M. and Boon; R. (2006). 21st Century Game Design. Charles River Media.
  69. Yee, N. (2015). The Proteus Paradox: How ONline Games and VIrtual Worlds Change Us – and How They Don´t.
  70. Yee, N. Ducheneaut, N. (2015). The Gamer Motivation Model in Handy Reference Chart and Slides. Online. URL: http://quanticfoundry.com/2015/12/15/handy-reference/
  71. Luton, W. (2013). Free-to-Play: Making Moeny From Games You Give Away. New Riders.
  72. Aggarwal, C.; Reddy, C. (2013) (eds.) Data Clustering: Algorithms and Applications. Chapman & Hall/CRC.
  73. C. Fraley and A.E. Raftery. How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis. The Computer Journal, 41(8), 1998.
  74. Canossa, A.; Sørensen, J. R. M. and Drachen, A. (2011): Arrrgghh!!!: Blending Quantitative and Qualitative Methods to Detect Player Frustration. In Proceedings of Foundations of Digital Games, 2011.
  75. Harshman, R. (1978). Models for Analysis of Asymmetrical Relationships Among N Objects or Stimuli. Paper presented at the First Joint Meeting of the Psychometric Society and The Society for Mathematical Psychology. Hamilton, Canada.
  76. Harshman, R.; Green, P.; Wind, Y. and Lundy, M. (1982). A Model for the Analysis of Asymmetric Data in Marketing Research. Marketing Science 1(2), 205-242.
  77. Chew, P. A.; Bader, B. W. and Rozovskaya, A. (2009). Using DEDICOM for Completely Unsupervised Part-of-Speech Tagging. Proceedings of the NAACL HLT Workshop on Unsupervised and Minimally Supervised Learning of Lexical Semantics, pp. 54-62.
  78. Southey, F:; Xiao, G.; Holte, R. C.; Trommelen, M. and Buchanan, J. (2005). Semi-Automated Gameplay Analysis by Machine Learning. In Proceedings of the 2005 AAAI Artificial Intelligence Conference.
  79. Aranganayagi, S.; Thangavel, K. (2007). Clustering categorical data using silhouette coefficient as a relocating measure. In IEEE Proceedings of the International Conference on Computational Intelligence and Multimedia APplications, 13-17. DOI:10.1109/ICCIMA.2007.328
  80. Rousseeuw, P. J. (1987). Silhouettes: a graphical aid to the interpretation and validation of cluster analysis.  Journal of Computational and Applied Mathematics, 20, 53-65.
  81. Tibshirani, R.; Walther, G. and Hastie, T. (2001). Estimating the number of clusters in a data set via the gap statistic. Journal of the R. Stastistics Society, B., 63(2), 411-423.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s