Josh Millet and Eric Loken write:
You may recall the frenzy last summer when a phony report circulated that Internet Explorer users were less intelligent. That hoax should have been easily spotted because the fabricated data suggested a massive gap on the IQ scale. Anyone with any knowledge of psychological testing should have immediately understood that the data were impossible.
They then report on a new study with real results:
The CCAT is a timed test with maximum score of 50. Scores greater than 40 are rare, and the mean for our sample is in the low to mid twenties. (Keep in mind that on our website, this test is selected by employers trying to fill jobs with greater than average complexity and responsibility, so the pool is tilted to the upper end of the distribution already.) . . . Internet Explorer users are lagging Chrome, Safari and Firefox users by approximately a ¼ standard deviation. The difference is highly statistically significant . . . and it is of modest practical significance (it would correspond to something like 3 or 4 points on an IQ scale). . .
When we isolate the Mac OS X users from the rest of the pack (almost exclusively Windows), we see a similar and slightly stronger difference. The mean score for Mac users was 25.26 and the mean for the rest 22.83.
Millet and Loken conclude:
But let’s not get carried away with any of this. By definition Google and Apple are hungry for market share, and that means selling to everyone. Care to bet how things will look 15 years from now? It’s possible we’ll be saying “Google Chrome is the new IE,” and the tables in this post will look awfully out of date.
Does anyone know what the sample is representative of? Besides the selection issues (clearly), I have major concerns with this. Also notice the face that with the Mac operating system, the mean is higher. Presumably this might make sense, but also note that the Safari browser (MAC OS for the most part I am assuming) is about inline with the other browsers. HOWEVER once we look at the MAC vs. WINDOWS we get the sexy result that the Mac users are higher. So is there any shot we can see the distributions by the two operating systems. One of these browsers is pulling up the MAC (I’m betting Chrome here) while the Windows is being drawn down by the IE users. On average if we compared the two groups of Chrome users by operating system I bet we find no difference. But we did need the sexy result didn’t we?
Another feature for measuring intelligence would be finding out whether people think ‘Mac’ is an acronym, and spell it in all caps, or an abbreviation for Macintosh.
So the interpretation is that early adoption of technology is correlated with better performance on standardized intelligence tests? Seems like a non-controversial finding, given what our social construct of intelligence includes (things like inquisitiveness).
Joseph:
I don’t think it’s a surprise. The reason for reporting this is, as noted by Millet and Loken in the linked post, awhile ago there had been a hoax on this topic, so it was worth noting the results of a more careful study.
That’s confirmation bias and whishful thinking… It’s like all those other studies saying that vegetarians have higher IQ, but wait, if you’re Vegan you actually have lower IQ…. Football fans have higher IQ than Soccer fans… But wait…. And so on.
This is just silly.
Ps. Just for the record, I use Mac and safary. Hehe
It’s a timed test, right? Any chance the effect is due to which browser is faster?
That was my question as well.
Just submitted this to their site too: For a dataset like this, it could be relevant to report the distribution shapes and not just the means and standard deviations. The sample sizes show that many many more people in the sample are using IE and Windows. My guess is that the median scores for each population are closer but the means are skewed by a long tail of lower values among IE users. If that’s the case, then at every score level, most people stick with their default system browser, but people are less likely to switch browsers if they have lower scores. You can easily show whether this is true by overlaying raw number histograms from each group.
Similar to my point. Good call.
Requests for raw data are acknowledged. Now at the bottom of the original blog post you can find a link to some tables and figures.
Excuse the coarse appearance. I just made them quickly in SPSS to answer the points raised. I think the story stands as originally reported – a modest but clearly present difference across browsers and platforms.
http://blog.criteriacorp.com/2012/03/07/chrome-and-mac-users-are-smarter-than-ie-users/#comments
Seems like age is a fairly reasonable explanation of what’s going on here. Older people will have trouble on timed tests (well, any test, but timed ones especially) and IE users are known to be older than the users of other browsers. Same applies to the Mac gap.
maybe this implies that IE is easier to use than Chrome or Firefox? that’s not an non-trivial issue. or maybe it implies that people who test higher on these tests have a bias towards shiny new things? who knows. it certainly doesn’t imply much else, however.
Eric, let’s suppose we could collect IQ measures of everybody who uses whatches and everybody who does not use watches.
It’s very likely that the mean IQ of the two groups is not exactly the same.
Then, suppose you find out it’s not the same.
Would you really try to rationalize it? “oh, smart people need have more tight schedules and need watches… or, oh, not very smart people forget about the time a lot and need watches”.
The fact is that it’s just not the same, period. And it’s very very probable that it has nothing to do with intelligence.
Rewriting:
“oh, smart people probably have more tight schedules and need watches…
or, oh, people that are not very smort always forget about the time and need watches”.