DATA STUFF - EP. 1: First Behavioral Analysis

CyrilCyril REGISTERED, Developers Posts: 3 Developer
edited July 2016 in Science!
Hi everyone,

As @Ino stated in another post, the data science team at Nerd Kingdom is starting to build up, and we thought we would share with you some of the work that has been recently done regarding TUG and data analytics (more particularly behavioral analysis), as well as discuss our goals for the next months and the next release by the end of the year.

Thanks to @marceloSR for his work on this study. Please keep in mind that those are preliminary results, aiming to help us build better and more refined models over time. Data were collected during 45 days, consisting in 315,470 events generated by 87 unique players that had to install a specific mod for that purpose. After pre-processing and filtering, we ended up with 459 unique sessions played by 79 unique players.

For each session, several types of events have been collected: mining, building, harvesting, crafting, farming, exploring, dying, hunting, fighting, consuming. Besides those events, other metrics have been defined such as the total length (time played), number of unique biomes visited, traveled distance or total number of events.

The first idea was to look at the engagement of those players based on the number of events generated over time. Using clustering techniques, we ended up getting three levels of engagement:

- Low engagement sessions (69.06%): ~ 39 min for ~192 events
- High engagement sessions (25.92%): ~155 min for ~1,175 events
- Very high engagement sessions (5.01%): ~363 min for ~4,945 events




Low (red), High (green), Very High Engagement (blue)


This step is important to better define player profiles/types/archetypes by engagement level, i.e. what defines the different player types for those that have a low engagement level and how we can improve their overall experience by refining the game based on their interests (e.g. personalized generated world/content).

In order to achieve this goal, we need to breakdown the previously defined low and high engagement sessions into different behaviors observed. Using clustering again for this behavioral analysis, the following results have been obtained:




- Short Play: Sessions with very low gameplay length and almost no engagement in the gameplay actions studied (shortest traveled distance, few explore events, etc.).

- Visit: Sessions with diversified gameplay behavior, where no action stands by itself and most actions are moderately performed.

- Build: High values for mining and building events. All their other parameters are considered low or moderate.

- Explore: Significant distances traveled in the game world: high values for unique biomes visited, distance traveled, exploring, and hunting events.




- Pioneering: Demonstrates a diversified gameplay style where no action stands by itself (similar to the Visit cluster from the low engagement behaviors).

- Explore: High values for unique biomes visited, travel distance, exploring and crafting events. Players tend to travel in order to harvest and craft items using the collected resources.

- Build: High values for building and farming events. On the other hand, they are the group with the lowest number of exploration and death events.


Again, those are preliminary/proof-of-concept results, based on a reasonably small amount of data from a limited number of users. Player types will certainly evolve as we collect more data, however, this still allows us to interpret how the users engage and behave during game sessions, as well as study the transitions between behaviors that we can observe.

Future works, and what we are already working on within the data science team, include, among others:

- Design and improve our data capture strategy by accessing a broader spectrum of events containing information about gameplay experience, in order to improvand refine our models.
- Qualitative analysis is also being conducted to define player archetypes. We will update our archetypes based on the quantitative analysis results every iteration
- Evaluate the correlation of event consequences by player profile, in other words can we trace a chronological relation between some types of events occurring and a potential engagement shift?
- Determine if based on incoming player behavior data, whether or not an engagement level for the current session could be predicted after a certain amount of time/events.

To finish with this first data related post, I’ll just add that one of our final goals with those data collection and statistical analysis is to build an online data analytic platform that will not only allow us to get insights on our side as a company, but will also provide the users a broad vision of the community behavior as well as deep insights from their own gaming sessions. We made a good start and a lot needs to be done by the end of the year, but as we show progress, we are really excited about the possibilities that we are exploring with the data science team. We will keep you updated on next the advancements!

Comments

  • Hoppa_JoelHoppa_Joel REGISTERED Posts: 191 Seed
    Most of us, worked together building towns, then we'd satellite around with other bases for build purposes. some fun times :)
  • WingidonWingidon REGISTERED Posts: 1,128 Seed
    So, basically, this is showing what players tend to do depending of how long their session is, did I understand it right?
  • ekohrmanekohrman REGISTERED Posts: 87 Seed
    Many of the Explore sessions were probably one of getting lost and desperately trying to find our way back home! :) I know I did that plenty.
    Some call me... Terella.

  • CyrilCyril REGISTERED, Developers Posts: 3 Developer
    @Wingidon The level of engagement is not only defined by the session length, but also by the number of events by session. And then yes when we breakdown those sessions by engagement level, we can derive patterns on how users tend to play.

    @ekohrman That may be a possibility. Actually, a student is currently working on spatial analysis with those data, to determine for example if Explore sessions are users that really explore the map or if they tend to go back and forth from a specific area.
  • DapperHamsterDapperHamster REGISTERED Posts: 58 Seed
    That would be interesting to look at the explore sessions; I know I made many treks from my base to the village area to look at people's building progress, treks that might have gotten interpreted as exploring events.

    What is meant by player archetypes? Are you thinking that players fall into certain categories? And can you do 'longevity' studies by looking at particular players behaviors over time? I feel like most of us went through both an explore and build phase, perhaps several times.
  • WingidonWingidon REGISTERED Posts: 1,128 Seed
    @DapperHamster By archetypes I assume they mean the kinds of stuff that players tends to do. For example, some people like to build stuff, while others preffer to go murder things around.
  • Hoppa_JoelHoppa_Joel REGISTERED Posts: 191 Seed
    Okay, so DooyDan and I did a lot of exploration to find seeds for the last server seed too, the idea was to have as many biomes converged for the town. So a lot of the exploration on the last server build was done outside the online bit of the game.
    our test which didn't pan out was that if we were all together the server crashes would be less, which to a point is correct, but there is a objects count when you exceed it in an area it will over load the server causing crashes when a new person loads in and things tried to 'make themselves known' to everyone all over again.
    The end result, was we had started crashing around the start where we all were building, and it seemed to happen at a certain number of objects int he debugging screen ( forget now what it was. ) so I ended up taking down my house to lessen counts, and we started building in other locations to test, and for the most part no more crashes happened just so long as an area block did not exceed a certain number of objects.

  • inoino REGISTERED Posts: 131
    @DapperHamster @Wingidon , yes, that is correct. An archetpye is a generalization of a players habits/behaviors broken down into categories.. Explorer, Adventurer, Farmer, etc. Some of those bits may evolve, or some subsets may exist.

    There are interesting things that could occur with gameplay with this sort of data as well. Perhaps some god blessing to people who do those things. Or triggering some kind of event... but more specifically, it will eventually impact the SORT of worlds you may see. Or the types of creatures, companions, or bosses you may encounter... its finding the balance so we can challenge someone, but also introduce the types of content that are important to them, but also see how we can encourage them to try or experience new types of things.
    I am a Dev on TUG, and I does teh science

    Follow me on the twitters, why not? @inoritewtf
Sign In or Register to comment.