• Hey Guest! Ever feel like entering a Game Jam, but the time limit is always too much pressure? We get it... You lead a hectic life and dedicating 3 whole days to make a game just doesn't work for you! So, why not enter the GMC SLOW JAM? Take your time! Kick back and make your game over 4 months! Interested? Then just click here!

Marketplace reviews exchange

csanyk

Member
I posted an appeal asking for reviewers for my Marketplace assets yesterday, originally in the Marketplace forum, but it got taken down by the moderator because they felt that the posts promoting your Marketplace asset is itself an invitation for reviews.

So, I'm trying again. This is a little bit different.

Since the Marketplace Beta launched, I've published 9 Marketplace assets so far, many of which are free, and have a total of (I think) just 5-6 reviews across all of them. I'd love to get more feedback on my work.

https://marketplace.yoyogames.com/publishers/145/csanyk

I'm looking for fair, accurate, unbiased reviews.

In return, I'll review your asset(s). Comment below with a link to your publisher page, and I'll check out your assets, and post a review within a few days.
 

chance

predictably random
Forum Staff
Moderator
When I removed your other topic from the Marketplace Forum, I didn't expect you to re-post it here. haha... but that's OK. Perhaps there's some benefit in discussing the Marketplace "review process" itself. So here are my thoughts on this matter:

I don't feel your approach is a good idea. I agree that Marketplace assets should get unbiased reviews, but setting up a quid pro quo "review exchange" won't always produce unbiased reviews. The problem is that reviewers expecting a return review of their own assets may be reluctant to say anything too negative about your asset. Ultimately, this leads to reviews that don't really help users that much. They may omit comments about deficiencies that users need to know about.

I'm not saying you would produce dishonest reviews. But this could produce reviews that were very neutral, and not much help to users.
 

Nocturne

Friendly Tyrant
Forum Staff
Admin
In return, I'll review your asset(s). Comment below with a link to your publisher page, and I'll check out your assets, and post a review within a few days.
Personally I really dislike this idea... it's a "i'll scratch your back if you scratch mine" system, and doesn't lead to objective reviewing in the least. Say I review someone's asset and I think it's rubbish... poorly documented, and hard to use. Now, if I leave a bad review, then they might take umbrage and in turn give something of mine a bad review. Alternatively, I might feel obliged to omit this part of things and give a neutral/favourable review.

Don't get me wrong, I'd love to see more comments and ratings on ALL my marketplace assets, but I don't think this is the way to get it...
 

csanyk

Member
Admittedly there are problems due to human nature being what it is.

How about a web application that scrapes the Marketplace site for assets, ranks them by number of reviews published (ascending), sorts them by free and paid assets, and you go there and sign up to review assets, and the app assigns you a selection of the least-reviewed assets that are either free, or paid and you've already bought them?

Alternately, I'd like to just try to set up an event, like a "review jam" or something like that, where the goal is for people to post reviews on assets.

However you do it, review quality will always be a concern, but that's what moderation systems (was this review useful?) are usually for.
 

Nocturne

Friendly Tyrant
Forum Staff
Admin
Alternately, I'd like to just try to set up an event, like a "review jam" or something like that, where the goal is for people to post reviews on assets.
Sounds good. I'm all for anything that helps build the community. Work out the details and PM me with them and we'll discuss it if you want. ;)
 

Mike

nobody important
GMC Elder
Anything that is a quid-pro-quo doesn't work, and actually corrupts reviews. People will rate positive to get positive results back. So this can't happen.


A site to specifically review assets seems odd.
First, it's not like they can't just go through the current Market place and get free assets and review them.
Second, if you get an asset to just review, your simply not doing it justice. Getting some script (or asset) and then complaining about it simply because you don't need it - or like it, is bad - it may not be aimed at them. Take a beginner who is after some simple running animation. The anims may not be amazing, but they may fill the hole that user needs.

People reviewing for the sake of it means they'll end up with stuff they think they could write or create easily and think it's crap. If that were the case, should I go and just give everything in the marketplace a bad review? no...that would be stupid.

Don't like this.....sorry.
 

csanyk

Member
Anything that is a quid-pro-quo doesn't work, and actually corrupts reviews. People will rate positive to get positive results back. So this can't happen.
Quid pro quo is problematic, I agree with you. Yet, I would like to see my own assets get more reviews, and I don't know what else to do.

Personally, I want honest feedback. I do the best work I know how when I make my assets, and I'm proud of the work I do, and of course I love it when I hear that people found my work to be of good quality. But if someone has a problem with it, telling me about it will only help me to make the asset better. Constructive feedback would be better appreciated as a private exchange between developer and customer, rather than part of a review, but if the publisher isn't responsive, I think it would be appropriate to put it in a public-facing review. I want all kinds of feedback: feature requests, bug reports, thanks, advice, you name it. The main purpose for a product review is to give other users an idea of whether the purchase was worth it, and I'd like for the content of a product review to primarily reflect that. I imagine that people being people, they'll use and misuse the review form for whatever their own purposes are, of course.

I really don't want to tell everyone how to review assets, though. People can tell if someone's putting the effort into writing a quality review that will help other Marketplace shoppers decide whether a given asset is likely to be worth their time or not. Trashing an asset for some personal vendetta would most likely backfire on the reviewer. But I haven't really seen that many reviews on other assets that were unhelpful. I just would like to see more reviews, on my own assets especially of course, but across the whole marketplace. Maybe once reviews are very commonplace we'll have a problem of bad reviews. But right now, I barely have any reviews at all, so bad reviews aren't really a problem I'm concerned with. I'm focused on upping the number of reviews I receive.

A site to specifically review assets seems odd.
First, it's not like they can't just go through the current Market place and get free assets and review them.
Second, if you get an asset to just review, your simply not doing it justice. Getting some script (or asset) and then complaining about it simply because you don't need it - or like it, is bad - it may not be aimed at them. Take a beginner who is after some simple running animation. The anims may not be amazing, but they may fill the hole that user needs.
I don't see why you would call promoting the act of writing reviews "odd". It's not odd; it's promoting community.

Why do I suggest that it would be helpful to have a web app to help promote reviewing? Sure, it's right there on the page where the asset is for sale. However, out of around 1000+ purchases of 9 published assets, I've only managed to receive 8 reviews since publishing my first asset, about a month after the Marketplace went live. So having the review link on the asset page doesn't seem to entice people to put the time in to do a review. Maybe 0-1% is actually a "good" response rate, despite my perception that it is not. But I'm trying to come up with ideas for ways to boost the review rate.

If there was a way to search the Marketplace for "most reviewed" and "least reviewed" assets, it'd facilitate Marketplace users who are particularly interested in writing reviews. If we collectively sought to promote reviewing as a positive activity, it'd benefit the community of Marketplace users, publishers and buyers alike. So a web app that was geared toward promoting that would be a good thing.

I got the idea of there being a web app that was geared toward promoting under-appreciated assets to give them some attention from the way the Ludum Dare game jam promotes game submissions that haven't been reviewed enough to be scored in the rankings. LD48 has a 20 day period to review games, and there used to be a problem of games not getting noticed, and so many games didn't get enough reviews to get ranked. The LD48 community responded to this problem by developing the site to specifically highlight under-reviewed games. It worked -- or at the very least, it's helped.

People reviewing for the sake of it means they'll end up with stuff they think they could write or create easily and think it's crap. If that were the case, should I go and just give everything in the marketplace a bad review? no...that would be stupid.

Don't like this.....sorry.
People will have all sorts of reasons for participating in reviewing assets. Some of them won't get it, and will write crap reviews. That's a given. That's not a reason not to promote more participation in the review system. Good reviews will rise to the top.

The best way to promote writing better quality reviews is to discuss what makes a review a "quality" review, and for the community to debate and promote those values, and raise the bar.

Active participation is the key. The current system is under-utilized (from what I can see, at least) and I'd like to see individuals in our community who are so inclined to work together to do something about that. Having a more laissez fair attitude about it is fine (if, Mike, you're busy developing GM:Next, I wouldn't expect you to be focused everywhere) but someone who is focused on the Marketplace should be taking the time to do things that will make the Marketplace better. I think promoting participation in the review feature qualifies.
 

csanyk

Member
Yeah, I don't like the idea of this either.
Ok, new rule on this thread:

If you don't like my idea, say why, and also offer a better way to promote more reviews. Otherwise you're just threadcrapping.

Let's have a productive discussion on how to boost participation in the review system. If you're not interested in that, don't participate.

Thanks.
 

csanyk

Member
Well unless you have recently been promoted to Mod, you don't get to make the rules. ;)

But since you asked...

Nocturne, summed it up with the 'you scratch my back...' comment. Which will turn the MP into a populatiry contest and nothing more than that.

Even if you get a few reviews the way things stand, you at least know that they are genuine reviews.

The only problem with the current system is that, if you have insecurity issues and you need people to constantly pat you on the back and say 'good job', then you are pretty screwed.
OK, so you're still stuck on my initial offer, to review other people's assets in return for them reviewing mine.

I already acknowledged that human nature will make this approach problematic at scale. So, can we agree to stop beating that horse?

OK. Look. I'm after one thing: more reviews. I'm trying to find a way to achieve that. I'm not trying to fix all the flaws inherent with review systems. I just want to get some feedback so I can get a better idea of what sort of things I should try to offer on the marketplace.

Wanting to generate more reviews doesn't address all the flaws inherent in the review system, and it's not intended to. It never was my intention to fix the review system. I'm simply trying to get more reviews for my assets.

All I'm doing with my original offer is saying, "Hey, if anyone would like more reviews, I'd like more reviews too, so let's review each other's stuff." I'm not asking for 5-star reviews. i'm just trying to get more feedback, and interact with fellow GM:S users.

But yeah, as I've acknowledged from the moment it was brought up, at large scale I'm sure that there would be some bad actors who'd use the offer to try to game the system by writing shill reviews for each other. I don't imagine that any bad actors are actually waiting for an invite to shill, though, do you? They have more efficient ways to do that, by generating fake accounts and just doing it.

And as well, I'm sure there might be some immature publishers who would get offended at something a reviewer wrote, and rather than address it by flagging or commenting on the review, or taking the feedback to heart and trying to make their asset better, would instead take it out on the reviewer by posting negative reviews for their assets. But, really, are we so afraid of that being the dominant use of the review system that a simple, earnest appeal for more reviews is met with such hostile skepticism?

I guess that says a lot about the average user of the software. Pretty sad, really.

If anyone would like to take me up on my offer, it's still a good offer: a fair, honest review of what I thought of your asset, for a fair, honest review of what you thought of mine.
 

csanyk

Member
Sorry, I did t realise that there was more to this thread, as you tend to ramble on a bit. (TLDR)



Write decent assets that make people go - wow!

Honestly, there doesn't seem to be any assets in your selection that scream 'Magic!'. People are downloading and either saying in their mind 'meh' or 'not bad'. Why would they go out of their way to write a review after that?
That's a catch-22. How am I supposed to get better if I can't hardly get any kind of feedback on my work?

I do the best I can to write high quality code and documentation, and I don't seem to have any problem getting downloads, especially for the free stuff, but reviews elude me. The few reviews I've gotten have been mostly very positive, I just would like to see better than a 0.6% ratio of purchases:reviews.

And, for a lot of assets I look at, many of them don't have many reviews, either.

One could conclude that one should simply try harder and do better, and then reviews will come flowing. I don't think it's quite that simple. I don't think that you should have to write something mind-blowingly amazing to get reviewed. There's plenty of stuff that's not flashy, but is useful, or well done, that deserves more attention than it's getting.

It could also be I'm the only user in the entire marketplace who cares at all about reviews, but I don't think that's likely either. In general, the more social participation the marketplace gets, the better it will be for everyone. It's been said that markets are conversations, and I'd like our marketplace to be a vibrant one.

TL;DR: If you don't feel like reviewing anything I've made, then don't. But if you'd like something of yours reviewed, one way to get one more review is to review something of mine, and then let me know. I'll check out your thing and give an fair and honest review of it.
 

csanyk

Member
BTW when you mentioned 'fake accounts', I am sure YYG would be all for this. As it means each time you want to make a new account, you need to purchase another copy of 'Pro'.

Yeah, some of us have more than one copy due to sales etc. but there is going to always be a limit to how many accounts you can hold.
I wasn't sure if the marketplace was still only open to Pro users or not, but that does in large part mitigate the problem. Not only does it prevent large numbers of fake accounts being created, it potentially gives YYG the power to revoke licenses being used to game the review system. Since reviews have to be attached to a username, and the username to a license, it's a powerful check that incentivizes reviewers' integrity. So I'm having a hard time seeing how it'll become a problem.

Sidebar: Really, though, I'd like to see the marketplace opened up so that I can sell assets to Standard users. I'm sure that's a vast market compared to the people who have paid for a license. And it'd be a good way for YYG to make some money (30% of sales ain't nothin!) from users who are too cheap to pay up for a Pro license. I'm all for keeping publishing access to paid users only, but I'd love to be able to sell assets to the millions of users who've downloaded GM:S for free.
 

csanyk

Member
That's all part of 'you get what you pay for'.

You really think that people who won't pay for a license, will buy your assets?
Maybe, maybe not. But I imagine there may be some people who can't justify dropping $100 one-time on a license, but would spend $0.99 100 times on marketplace assets. It might not be a large number, but if it were 0.01% out of millions, it'd still be a couple hundred more than if none of them could buy anything.
 
P

ParodyKnaveBob

Guest
I admire the efforts and mindset behind all this. I haven't been able to use the Marketplace in a couple years now (long sad story), thus I don't know if YYG has already addressed my small(-in-concept) idea, but what if the IDE itself helped nudge people to review?

I sometimes play games on Kongregate, and after you've played for awhile, the sidebar opens a tab asking if you want to rate 1-5 stars; if you don't, and you play later, same thing, after awhile, it'll open, asking if you want to rate it; meanwhile, the tab also gives an option to never be asked to rate games. It's pretty unintrusive, and it serves as a nice reminder for the people who do actually show some sort of interest in the game.

If the IDE detects you using some asset or another, perhaps it could give a reminder next time you open the Marketplace -- or even an arbitrary project, or a project that already uses the asset -- to go write a review. Whatcha think? I figure it'd be a bunch of work for YYG just on the design end, but the programming of it shouldn't be anywhere nearly difficult for them (unless Delphi rears its ugly head again, ha ha ha ha).

Also, concerning unpaid users obtaining assets, don't forget that free assets can work as an advertising gateway toward the paid assets. Even if unpaid users only obtained free assets, reviews of those assets can still help the creator in the long run.

Regards,
Bob
 
  • Like
Reactions: Yal

csanyk

Member
I admire the efforts and mindset behind all this. I haven't been able to use the Marketplace in a couple years now (long sad story), thus I don't know if YYG has already addressed my small(-in-concept) idea, but what if the IDE itself helped nudge people to review?
I love it, but I've lost hope for anything substantial in terms of IDE features to be delivered until GM:Next. And the sooner that's out, the better. The UX for My Library is atrocious, and hasn't been improved since they launched the Marketplace Beta.
 
  • Like
Reactions: Yal

Yal

šŸ§ *penguin noises*
GMC Elder
The UX for My Library is atrocious, and hasn't been improved since they launched the Marketplace Beta.
I concur on this. I've actually saved empty projects with the coolest assets I could find just so that I could start working on them instantly when I felt the urge instead of having to spend 15 minutes looking through the slowest scrolling list I've ever seen in a GUI. Reviewing an asset at the moment requires you to open a webpage in the internal browser from said list and it's pretty much a trainwreck, especially since you'd want as little to do with the list as possible to begin with.
 
I

icuurd12b42

Guest
Amazon asks me to write reviews and I'm usually happy to oblige. the marketplace should be the same though I hold no hopes anything will be done about this...
 

chance

predictably random
Forum Staff
Moderator
Amazon asks me to write reviews and I'm usually happy to oblige. the marketplace should be the same... (snip)...
I was going to suggest the same thing. I get emails after an Amazon purchase, asking that I contribute a review. And since I rely on other customers' reviews, I try to contribute my own whenever possible. Personally, I think this is the best solution.

The most trustworthy reviews usually come from customers. And customers may be more comfortable getting "encouragement emails" from the Marketplace (YoYo), instead of from individual vendors.
 

ShaunJS

Just Another Dev
GMC Elder
It's a very interesting discussion here. I am very interested in us finding ways to to encourage more use of MP reviews. Perhaps there's more we can be doing on our end as Chance suggests (Reminders/encouragement on purchase? list of owned but unreviewed assets per user?)

I also agree that My Library has many UX problems in general right now.
 
I

icuurd12b42

Guest
It's a very interesting discussion here. I am very interested in us finding ways to to encourage more use of MP reviews. Perhaps there's more we can be doing on our end as Chance suggests (Reminders/encouragement on purchase? list of owned but unreviewed assets per user?)

I also agree that My Library has many UX problems in general right now.
Yes I think Chance's Idea is absolutely brilliant!!!
 
A

Aura

Guest
Apart from that, I guess that the overall rate (and number) of quality reviews on the MP can be increased with the help of a community competition (more of an event).

If you're old enough, some of you might remember the GMC Review Jam -- something similar could be held for MP assets. We could ask people to review assets of their choice and then conduct voting. ^^"
 
N

NPT

Guest
I'd always follow up with a 'thanks for your purchase,

...

Then NPT (from memory) kicked up about 'privacy' concerns and YYG stopped supplying email addresses.

But every single person I emailed were very appreciative, which started some strong relationships.
Yeah, that's right.

I made a fuss about 'privacy' concerns because I didn't want you sending Thank you emails.
 

csanyk

Member
I concur on this. I've actually saved empty projects with the coolest assets I could find just so that I could start working on them instantly when I felt the urge instead of having to spend 15 minutes looking through the slowest scrolling list I've ever seen in a GUI. Reviewing an asset at the moment requires you to open a webpage in the internal browser from said list and it's pretty much a trainwreck, especially since you'd want as little to do with the list as possible to begin with.
There really should be a way to write a review from the site. If there isn't, that's just not good design. Admittedly it's been a while since I've posted a review myself, but I thought you could post them directly on the sale page.

Apart from that, I guess that the overall rate (and number) of quality reviews on the MP can be increased with the help of a community competition (more of an event).

If you're old enough, some of you might remember the GMC Review Jam -- something similar could be held for MP assets. We could ask people to review assets of their choice and then conduct voting. ^^"
I'm in favor of an event, and have been talking privately with Nocturne about ideas for it. Nocturne has asked me to come up with suggestions on how to do it, and I'm thinking about it, but I'd love to hear everyone else's thoughts on how to do it right.

Initially we did get access to the emails of the purchasers. I'd always follow up with a 'thanks for your purchase, if you need any support don't hesitate to contact me' type email.

Now the purchaser is buying a 'box' and there is absolutely no relations anymore. This makes for a cold stony transaction. And there is no real way it can be any more than that in the present form.
I agree with this wholeheartedly. I was very disappointed when YYG decided that Marketplace sellers were "third parties" who should not have access to customer information from their own buyers. That was, and is, a terrible decision, and had a chilling effect on what could have been a friendly community-driven market. The result of that decision is that the Marketplace is just a cold place where you can buy and sell stuff without any real interaction between buyer and seller.
 
J

jackhigh24

Guest
id disagree there, i would not want someone emailing me at all, if i want to email you i would as there is a contact link on the market place plus you can add your email to the project as well, even ask for them to remember to leave a review, also there is a rate it button in the library for all that you have downloaded, as for asking for review for review i agree with mike and chance on that and have turned down thousands of people and companies asking me to join there review for review programs as it just does not give you the feedback that you really need.
 
Top