In my discussion of dentists-named-Dennis study, I referred to my back-of-the-envelope calculation that the effect (if it indeed exists) corresponds to an approximate 1% aggregate chance that you’ll pick a profession based on your first name. Even if there are nearly twice as many dentist Dennises as would be expected from chance alone, the base rate is so low that a shift of 1% of all Dennises would be enough to do this. My point was that (a) even a small effect could show up when looking at low-frequency events such as the choice to pick a particular career or live in a particular city, and (b) any small effects will inherently be difficult to detect in any direct way.
Uri Simonsohn (the author of the recent rebuttal of the original name-choice article by Brett Pelham et al.) wrote:
In terms of the effect size. I [Simonsohn] think of it differently and see it as too big to be believable.
I don’t find it plausible that I can double the odds that my daughter will marry an Andrew if I renamed her Andrea.
Less even that I can multiply by 5 the odds of her marrying a Smith if I changed her last name to Smith (see Figure 1 in my paper).
I guess like always it depends on what the question is. If it is: could implicit egotism account for a large share of our decisions? (R-squared question) then I am with you, even the most naïve estimates which will grossly over-estimate its potential impact will lead to tiny r-square effects.
Although with that logic not wearing seat belts may not be that bad, since most people don’t have serious accidents and hence the change in likelihood of dying, even if twice as likely, is still pretty slim. Right?
To which I replied:
I actually think that the effect sized are plausible (even if they’re perhaps not real). For example, marrying a Smith is pretty rare, so even a small boost in that direction could be a large factor.
From a statistical point of view, though, knowing that something is a small effect is also telling us that it will be difficult to study.
Finally, yes, I agree that not wearing seat belts is not that bad, since indeed it is unlikely to kill or seriously injure you. On the other hand, seat belts don’t cause much discomfort, so the cost-benefit decision for me is to wear them (during the rare times that I’m riding in a car or plane). And from a public health standpoint, seat belts are a cheap and uncontroversial way to save thousands of lives.
I even wear a bike helmet, which is a lot more of an irritant than wearing seat belts. Then again, I have a friend who fell when riding without a helmet and suffered a serious brain injury.
Simonsohn then shot back:
I think we need a benchmark to make a more informed judgment if the effect is small or large.
For example, the Dennis/dentist effect should be much smaller than parent-dentist/child-dentist. I think this is almost certainly true but it is an easy hurdle.
The J marries J effect should not be much larger than the effect of, say, conditioning on going to the same high-school, having sat next to each other in class for a whole semester.
I have no idea if that hurdle is passed.
These are arbitrary thresholds for sure, but better I’d argue than both my “100% increase is too big”, and your “pr(marry smith) up from 1% to 2% is ok”
P.S. It’s fun to have this blog, which is such an open place for discussion, a place that falls between the dry formality of journals and gee-whiz-isn’t-that-cool venues such as Freakonomics and the BPS research digest (both of which are great but don’t seem to lend themselves to open-ended discussion). I don’t know of many places like this where a discussion can truly go back and forth.