Home

 › 

Articles

 › 

How You’re Being Fooled By Video Game Surveys

How You’re Being Fooled By Video Game Surveys

The information is out! GlobalWebIndex has put together a very official looking graph going over how much time console owners spend gaming. Right at the top is the Xbox One, followed by the Wii U, PlayStation 4, Xbox 360, PlayStation 3, and Wii. I guess we should go ahead and add this to our list of evidence for whenever we want to try and prove which console is best, right? Wrong. While this may seem insightful and valuable, these survey results aren’t the best means of proving anything about this generation’s three systems.

Let’s start with the survey itself. GlobalWebIndex asked 17,990 console owners a this question, “Roughly how many hours do you spend playing on games consoles during a typical day?” The people were between 16 and 64 years old, and the graph shows the response options were Xbox One, Wii U, PS4, Xbox 360, PS3, and Wii. Back in January 2016, EA said the lifetime sales of the Xbox One and PS4 were 55 million. Over 13 million Wii Us have been sold. In their hayday, there were over 101 million Wiis, 84 million Xbox 360s, and 83 million PS3s sold. You have to wonder if 17,990 is an adequate sample size. Technically, it is. About 9,000 would probably be enough to come up with a 1% margin of error. But given the range of systems and biases that come with consoles, bigger may have been better.

GlobalWebIndex didn’t say anything about when and where the survey was conducted. Over what period was it held? Where was it held? How did it gather the results? You’re going to get a very different outcome if the survey link was offered on say, Gamefaqs or on Yahoo’s main page. Even between different gaming sites, player makeup can differ. A lot of Sony fans visit N4G, which means results from visitors to that site could skew toward the PS3 and PS4. It doesn’t say if the people polled owned only one console or multiple systems. We don’t have all the background information.

Taking a close look at the actual figures is important too. Think about this graph GlobalWebIndex has prepared. It starts with a clickbait headline. “Xbox One users spend the most time gaming.” That sounds definitive, right? Except when you scroll down, the actual figures aren’t so clear cut. Based on this company’s results, Xbox One users spend about two hours and twenty seven minutes per day playing. That’s substantial, sure. But the Wii U and PS4 numbers aren’t far behind. According to this survey’s results, people with Wiis spent about two hours and twenty six minutes on Nintendo’s console each day, while PlayStation 4 owners clocked in at about two hours and fifteen minutes per day.

Let’s say there is a 95% confidence level with results and a 1% margin of error with the results. The information provided is self-recorded. The people polled are giving an estimate. They aren’t using timers to actually track their gaming over a set period. These are ballpark figures. There’s no guarantee that the people reporting on their habits are honest or accurate. Don’t you think that could mean there’s at least a chance of a one minute discrepancy? If true, this could very easily have been a survey proving Wii U owners spend the most time gaming.

How You’re Being Fooled By Video Game Surveys

This survey also says nothing about console sales figures. OK. Let’s say that this is 100% accurate and true. Xbox One owners spend the most time gaming. That doesn’t mean gaming trends will immediately shift and we’ll start seeing drastic alterations in our hobby’s landscape. It doesn’t take into account how many consoles have been sold. You can’t use it to say this system won. You can’t use it to prove the game library is the best. It doesn’t look at review scores or customer satisfaction.

GlobalWebIndex’s survey shouldn’t impress you much. It shouldn’t make people try to say their system is better or another is worse. It offers information that says people probably spend just about two hours a day using their Xbox Ones, Wii Us, and PS4s. With a variation of one to twelve minutes between total times, the difference is negligible. We don’t have enough data to show how consequential all this is and, even if we did, everything is too close to unquestionably declare one system best or worst from it.

To top