After reading about the suspiciously low Metacritic user score of 6.3 for Media Molecule’s LittleBigPlanet, I decided to dig into the story a little deeper. For those unfamiliar with the situation, allow me to fill you in.

Metacritic, who by and large is considered the main source for aggregated scoring on the web, offers two score flavors (one for critics and one for users). This is the case across most sites. The problem is, when you are considered the “go to” resource for pretty much anything, there’s a chance that people will make a big deal out of something on your site that they would normally overlook on another.

That said, this specific instance pertains to the scores currently presented on Metacritic for the LittleBigPlanet game. The first, a 9.5 at the time of this article, represents the aggregated score based on several well accepted gaming review sites such as GamePro, IGN, and EuroGamer. The second, more controversial score, is a 6.3, and is compiled via the average score given by a pool of 2,225 user votes.

Gaming news site N4G, which pulls related news from various websites around the web, features a blog post titled “Fanboys pull LittleBigPlanet’s Metacritic score down”, discussing this very topic. Reading some of the furious responses to the score piqued my interest, so I visited the Metacritic site to see first hand what all the fuss was about. Clicking on the “Read user comments” link near the overall user score brings you to the bottom of the page, where the following statement by Marc Doyle, the sites founder, awaits:

“My advice for our faithful users is to focus your attention on the Metascore for this game and not the thousands of user votes, most of which have been submitted before said users have played the game. This is a gaming community, and if people want to stuff the ballot box, there’s not much I can do at this point. When we upgrade the registration requirements for participation on the site in the near future, this type of thing won’t happen. We’ll post the full legitimate user reviews upon the game’s release. As always, thanks for using the site.”

This prompted me to contact Marc directly in order to get a more complete picture of the situation from his end. Specifically, I felt that gamers would want some clarity regarding the portion of his comment stating that most of the user reviews submitted for LBP were made “before said users [had] played the game.”

Firstly, I wanted to know if he would consider eliminating user reviews submitted before the full release of the game, i.e. reviews clearly submitted based on the open beta alone, or even worse, no experience with the game at all, to which he responded:

” … we won’t be clearing those old scores from LBP when it launches, but we will only post reviews from people who have clearly played the game. All the reviews from fanboys and haters who haven’t played it will just be left out. That is true for all products on Metacritic. Humans actually read all of those user reviews (so we can delete profanity, spoilers, etc.), and those humans are told not to post reviews from people who clearly haven’t played the game.”

He then added that the burden of proof they demand from users submitting reviews prior to a game’s release date is be much higher than those doing so after the fact.

Slightly confused, I asked Marc for some clarification as to how the game could have such a low score, if his team was manually looking over each user review submitted. He stated:

“All the [numeric] scores “count” in the overall user average and total votes tally, but we only post a subset of those as [written] user reviews.”

When talking about the future of the site, as well as other things they are doing to help eliminate this type of fanboy nonsense in the future, Marc had the following to say:

“There will be a Metacritic re-design in the future (can’t say exactly when), and as a part of that process, we’re going to beef up our user registration and community functions so that “stuffing the ballot box” won’t be nearly as easy.

The original idea behind allowing users to write user reviews (and vote on them) was to allow those people (like me) who see advanced screenings of movies and sometimes play games ahead of their true release date to write reviews and have them posted on Metcritic. That’s still the case now.

Over the years, people have, by and large, not exploited the fact that you can vote early. Only recently have people started voting en masse for (and against) games like they’ve done with LBP.

Metacritic’s primary “product” is the Metascore, which is the weighted average of scores from professional critics. We include user reviews for obvious reasons, but because that score can never be 100% reliable, it’s given secondary importance on the site. For example, you can’t sort or search by user scores for any products. That will likely change with the redesign.”

Marc obviously has a solid understanding of the space, its evolution, and the challenges that lie ahead. The fact of matter though, is that currently, anyone can visit Metacritic and leave an abnormally low user score, skewing the average, without necessarily saying anything of value at all. I asked him if this was something the site would be revisiting in the immediate future.

” … nothing should change in this regard until we do the redesign. That tally will remain a raw tally, and only the printed user reviews will be “vetted” as useful reviews from people that we think actually played the game … “

He closed the topic by adding that the specifics of “whatever [they] do with the voting system moving forward [have] yet to be established.”

With October 27th just around the corner, LittleBigPlanet is four LittleBigDays away from being officially released. So while the current 6.3 Metacritic user score has been getting most of the attention on the net, I (like Marc himself), kindly ask that you shift your focus to the 95 out of a 100 Metacritic score the game is receiving from the mainstream media. Everything else will sort itself out in due time.

Tags: aggregated scoring, controversy, fanboys, founder, LittleBigPlanet, Marc Doyle, Media Molecule, Metacritic, N4G, SackBoy, SCEA, scee, user score

• PS3 Fanboys Strike Back: Gears 2 Metacritic User Score Lowered to 4.1
• Resistance 2 Latest Victim in Metacritic Douchey User Flames
• Despite Fanboy Accusations, Metacritic Does Not Exclude In-Network Review Scores
• Sessler Psycho-Analyzes the Fanboy Condition
• N4G Being Bombarded with Gears 2 Spoiler Articles Posing as News
• N4G Cares Not for Contributors, Only Bottom Line
• Joker’s T-Rated Gag Gun Fatality All Gag No Gore
• Chris Satchell Talks XNA Community, Security and Distribution