Feedback What has GMS 2 analytics done for us?

[FrostyCat]

I have recently learned that GDPR compliance was one of the reasons why 2.1.4 was a forced update, and it has made me revisit an old issue regarding analytics on GMS 2.

From day 1 I have been a fierce opponent of burned-in analytics on GMS 2. In my argument, I cited legal issues, inflexibility in updating the runtime and a lack of developer-end utility as potential problems. In my honest opinion, all three of the above have come true.

YoYo staff claimed that the analytics data may one day be shared in an aggregated report and/or available on an individual basis, such that it's useful for us on a personal level. Neither has come to fruition. YoYo needs to recognize that when we use GM to develop commercial games, we are developers first, GMS 2 users second. We need and want data pertaining to ourselves individually, as well as that of the market as a whole or a specific segment. The way GMS 2 telemetry is done satisfies neither needs: We can't get at our own stats with it, and it is neither big enough to be reflective of the market as a whole nor specific enough to describe the segment we're individually interested in.

YoYo staff also claimed that telemetry helps them understand usage and improve GMS 2. Yet from the way YoYo responds to breakers in monetization and analytics plugins in iOS/Android, or major crashes or pain points in the IDE, I see little evidence of action on collected data. The analytics is not used proactively, in fact most if not all of the bug-fixing activity is still passively driven by the helpdesk. Either the analytics data is being allowed to pile up without doing anything, or it is collected in a way that cannot be acted upon (e.g. failing to capture exceptions that can be caught). All while tickets still sit in the attic for months at a time.

Quite frankly, if the analytics collected by YoYo is not being used productively, then it should either be improved or removed. As far as an end-user developer like me is concerned, it is a technical and legal liability that has not been offset by tangible assets elsewhere. Please create that asset, or provide evidence of that asset if there's something behind the scenes I don't know of.

YoYo should stop asking what we can do for their analytics, and start asking what their analytics can do for us.

 

[HW.]

It has been almost a month since May 25th and it looks like the impacts of the avalanche of GDPR don't stop, ... yet.

However, i prefer and i think it is better that YoYo will gather the data themselves on their official upcoming games (from their recently announced Publishing section), rather than IF (also known as IN CASE, which we don't hear about it so far, fortunately.)~ that we might be asked as GMS users in the case of there might be a need to ask such a GDPR-compliant consent (again? after those other cookies consents of our own analytics, oh noooo please don't add any more burdens, we are still struggling doing consents for our own analytics and also those traffic reports or cookies) from the end user (of our own games) for the telemetry (IF && in case GDPR demands it) for the use of 3rd party (YoYo).

So, perhaps, if the telemetry data isn't useful anymore, because of the last year's complaints (optional on/off) , in addition to the recent GDPR impacts and also the upcoming 2019 ePrivacy Directive and bla bla bla more privacy laws.

I think if YoYo removes the telemetry altogether from the engine, it will be efficient. And then YoYo can do the analytics "only" from their games of publishing section of the company (which can gather consents from the end users directly in the name of the YoYo/its partner publisher).

I think that by YoYo doing bug fixing, and be more active to monitor this GMS community forum will also help to improve the GMS2 product. There are many feedbacks here which aren't realized yet such as BUGGG REPORTS, requested features, finishing roadmaps, etc (compared to the data telemetry, we are not sure as we don't know exactly what it is actually except YoYo team).

I am sorry if i am wrong, but it seems since GDPR, many things are considered as personal now in related to data, tracking, telemetry, and so on.

 

[appymedia]

Until yesterday I honestly hadn't realised there was analytics being sent without the option to disable it. Every other bit of software I know lets you opt out of things like this. 99% of users will still opt in but giving the option is important. How can I see the exact content of this data, does anyone know?

Also it would be nice to have some input from YoYo on here guys. This thread and some related ones on bugs forced updates deserve a little attention IMO. I know your busy but helping us understand stuff might help as theres always 3 sides to every argument / discussion. Our side, your side and then what matters / the compromise in the middle

 

[ATY@Y@]

There are two types of analytics we employ, player analytics, which is optional in all licences except Creator (it will shortly be optional in Creator). The second is IDE use analytics, which we are seeking to improve.


While analytics is covered by GDPR, it’s not the reason we require everyone to re-confirm their accounts, and the reason we needed to employ a mandatory update to ensure this happened. The main reason for this is that our previous processes for identifying whether a user is a child were inadequate. Children are not able to provide explicit consent under GDPR.


The improvements we are making to our IDE analytics won’t require a mandatory product update. They will be used to better understand how new users use the system, so we can provide better tutorial content, and to support our helpdesk and product development teams in determining whether problems are being widely experienced by users, so we can more effectively prioritise.

 

[JeffJ]

Still doesn't remedy the fact that making the IDE and runtime updates separate was useless in this case, removing our options for rollback.

Also - if the runtime update for GDPR compliance had to be forced, why didn't you make it so that the only thing that update had was the compliant stuff? By making a forced runtime update with other stuff you were basically asking for trouble, knowing fully well your users would not be able to rollback from any mistakes made in said runtime. The collision errors introduced had nothing to do with GDPR. Shouldn't have been covered in the same update under any circumstances. No matter how you put it, your users got burned for your bad decisions (again) and there is no excuse.

Because of these bad decisions I am legitimately worried about the future. What can you tell the community to assure that this carelessness will not be executed in the future?

Also - you completely omitted the question of giving users access to any analytics, be it IDE or game analytics. This is something that was also suggested and ignored back when it was initially discussed. What are the current plans to share this data? For example, if I choose to opt in to player analytics in my games, will I have access to that data? If not, why should I choose to opt in?

 

[gnysek]

I wonder what happens, if according to GDPR, I will ask YYG to give me export of ALL data they have about me (so it need to include IDE stats)...

 

[FrostyCat]

There are two types of analytics we employ, player analytics, which is optional in all licences except Creator (it will shortly be optional in Creator). The second is IDE use analytics, which we are seeking to improve.

The first type begs two questions:

• Why is it that when we turn off IDE analytics (and thus sending no data), it is still not GDPR-compliant?
• If the reason is because the analytics code is still in the runtime and would show up in app store scanners, why is the option for disabling analytics not getting rid of that code in the first place? Why is the analytics designed in a way that it cannot be easily uncoupled from the runtime, when the use case for removing it completely is foreseeable by anyone who isn't lovestruck with big data?

The second one I'll cover in more detail in the rest of the post. I have several serious issues to raise about the way YoYo is using it.

While analytics is covered by GDPR, it’s not the reason we require everyone to re-confirm their accounts, and the reason we needed to employ a mandatory update to ensure this happened. The main reason for this is that our previous processes for identifying whether a user is a child were inadequate. Children are not able to provide explicit consent under GDPR.

And what makes you think the new process is actually adequate? If it turns out not to be adequate, are we going to repeat this with another mandatory update that we can't backtrack unrelated bugs with?

If you read the posts around here complaining about the mandatory update breaking time-sensitive projects, the people most affected by it are full-on professionals, not kiddies playing with GMS 2 for the first time. Every one of these professionals is capable of providing the explicit consent, they are all over the age of majority. Why are they then not allowed access to the version predating the mandatory update, at least by request?

The improvements we are making to our IDE analytics won’t require a mandatory product update. They will be used to better understand how new users use the system, so we can provide better tutorial content, and to support our helpdesk and product development teams in determining whether problems are being widely experienced by users, so we can more effectively prioritise.

Except YoYo isn't prioritizing effectively at all with this data, and professional users are feeling the brunt of it.

YoYo's analytics is failing to capture offline and pre-login usage experience, and that has been a weak point in GMS 2 since day one. For example, one of the splash screens on GMS 2 has been crashing on my system for months (so I only have an 80%-something shot at getting it to start up properly). Yet this won't show up in analytics because it isn't initialized yet. In fact, a significant portion of complaints about GMS 2 is failing to start up, log in or keep its session properly. Is analytics data telling YoYo not to put effort there?

YoYo's analytics also doesn't seem to be sufficiently segmented, so it lumps both beginners AND professionals in the same basket even when they have polar-opposite usage patterns. If you target efforts this way, you end up making the "average person with one ovary and one testicle" mistake. Another problem happens if you mis-classify professionals as beginners, or vice versa. I genuinely hope that if this segmentation is happening, it isn't based on something flaky and naive like cumulative usage hours.

Targeting efforts strictly by popularity is also problematic. While YoYo is at a point where additional growth in professional use is the most important, most of its user base is still non-professional. You can see in topics like this and others on the GMS 2 section that professionals are increasingly disaffected about being neglected and adversely affected by decisions that only rookies can tolerate. If this trend of failing to cater to professionals continue, it could be a vicious cycle that GMS 2 can die from: Fewer professionals using GMS 2 - Analytics tell YoYo to cater to professionals less - more professionals leave.