Skip to content
 

What’s the motivation to do experiments on motivation?

Bill Harris writes:

Do you or your readers have any insights into the research that underlays Dan Pink’s work on motivation and Tom Wujec’s (or Peter Skillman’s) work on iterative development?  They make intuitive sense to me (but may be counterintuitive to others), but I don’t know much more about them.

Pink’s work is summarized in a TED talk (yeah, okay, cred problem already, eh?) on The puzzle of motivation (there’s a shorter RSA animation here).  The basic claim?  Once one gets past routine tasks and once one has enough financial reward to take money off the table, then extrinsic motivation worsens performance instead of improving them.  That’s largely consistent with Deming’s views, too.

Wujec’s and Skillman’s work is summarized in another TED talk, called Build a tower, build a team, and on The Marshmallow Challenge Web site.  If you listen to the end, you get the message that iteration trumps planning, that extrinsic motivation alone degrades performance seriously, and that knowledge about effective (iterative, collaborative processes) combined with financial rewards trumps even that knowledge by itself.  In a way, that’s consistent with Tukey’s EDA, with your emphasis on starting small and iterating (still not quite expressly consistent with your “throw everything into a model and test it all at once”), and with the iterative stance of action research and continual process improvement.

Both claim to be supported by studies.  There’s an interview with Pink in HBR called What Motivates Us?  but everything I find online says there is research but doesn’t provide it (I read the book, but it was when it was new, and I no longer have it easily available).  So far, the Wujec / Skillman position seems supported mostly by experiments they’ve run in workshops they’ve conducted.

They also seem to conflict in one key sense.  Pink’s thesis seems to be that financial rewards degrade performance, period, while the Wujec / Skillman thesis seems to be that there’s an interaction between knowledge and financial reward, as if financial rewards amplify the effect of appropriate knowledge.

We may all have our biases, but I’m curious about cutting through those to what useful investigation of the data tells us.  What does the research behind these say?  Is that research good, or does it fall prey to low power and that pesky overgrown garden you keep writing about?

My reply:

I have no idea. It all seems much more worthy than that power pose stuff. I guess I should talk with a serious researcher at Columbia or somewhere who works on organizational psychology. Or maybe someone who does research on sports. Usually the way I could get into such a topic would be to read some papers and check out the data and statistical analyses but here I wouldn’t know where to start.

The topic is interesting in itself and is of particular importance to me because nearly every day I’m thinking about what I can do to keep Stan team members happy and productive, and we do have some mix of structure we can impose and extrinsic rewards we can give out. The idea of iterative improvement makes sense to me, although to some extent that just pushes the question back one step to: How do you set up the iterative improvement process?

Finally, the relation between common sense, individual experiences, and statistical evidence is unclear here. Sure, statistical evidence should be relevant, but maybe the details of how to manage the team vary so much that there is no general recipe for success or even for relative success. Then it would be hard to study the topic using the usual treatment vs. control strategy.

A similar issue arises in education research which may be one reason we don’t practice what we preach, and even in silly things like power pose: if, as seems reasonable, different “poses” work for different people in different settings, then it may be close to hopeless to learn anything useful from a conventional experiment. One could however take one step back and do a controlled trial of meta-advice, for example instead of recommending a specific pose, the person in the experiment could just be asked to choose a pose, or to sit and focus for a few minutes. Then again, that won’t work for everyone either . . .

12 Comments

  1. Peter says:

    Maybe of interest:
    Stefano DellaVigna & Devin Pope: What Motivates Effort? Evidence and Expert Forecasts. NBER Working Paper No. 22193, April 2016
    http://www.nber.org/papers/w22193

    Also, how about checking Annual Review of Sociology, Annual Review of Psychology, Annual Review of Economics, and the like: http://www.annualreviews.org/

  2. LemmusLemmus says:

    I have not watched the Pink talk, but it seems like he’s drawing on the research concerning the “crowding out” of intrinsic motivation. For example, see here:

    http://www.econstor.eu/bitstream/10419/75604/1/cesifo_wp245.pdf

    • Rahul says:

      Reminds me of that Israeli kindergarten study (flawed?) where penalizing parents monetarily for late-pickups actually increased the number of late-pickups.

      Would that be similar?

    • Well, this sounded exactly like Deci, Ryan, and colleagues work on “self-determination theory” which discusses a variety of features of motivation, and has the important implication that extrinsic rewards can lead to lower performance by blunting intrinsic motivation…when I teach this, I do sometimes use the (admittedly loose) professional sports analogy that, “when they start paying you, it stops being fun and starts being a job.”

      I don’t want to say the jury’s out on this, but it is a pretty large body of scholarship within social, personality, and organizational psychology. I was pretty glad to see a good chunk of this work cited in the linked paper.

      Nothing against Pink here, but as someone in the organizational psychology area, I’d would much more strongly associate these ideas with Deci and Ryan than with Daniel Pink or anyone else.

  3. Alex says:

    The sports arena would be interesting, if only perhaps because there’s an empirical resolution to two intuitive ideas that run counter to each other. One is the idea of choking, which would fit in with the kind of motivational research that Bill describes. If there’s a lot on the line, people start to get nervous, over think it, etc, and do worse. But the other is the idea of the ‘contract year performance’, when an athlete is in the last year of a deal and plays really well to earn a good new deal. I know that lab research exists on choking, but I don’t know what conclusions it has reached or if anyone has seriously looked at the contract year idea.

  4. David Condon says:

    “Once one gets past routine tasks and once one has enough financial reward to take money off the table, then extrinsic motivation worsens performance instead of improving them.”

    Here’s a video by Tyler Cowen making nearly the opposite argument: https://www.youtube.com/watch?v=8Pk654J8-5c&feature=youtu.be

    I agree with Seth Spain’s comment that Pink is most likely referring to Deci’s theory based on the quote. There are three main areas of research you can look into when reading about motivation: behavioral lab experiments frequently based on animal research, motivational psychology, and economics. You’ll see different terminology used in each case, and so there’s more research written on stuff like this then you’ll ever have time to sort through.

    I have a low opinion of the research on self-determination theory (the main meta-analysis I’ve cited below). Deci, in developing the theory, took it as a given that extrinsic motivation undermines intrinsic motivation (the overjustification effect), and then argued that it was up to scientific research to figure out when the overjustification effect appears and when it doesn’t. So with his meta-analysis he took a gigantic number of published studies (so a biased data set) and just data-mined them essentially. He overlooked areas which contradicted previous predictions he had made, and focused on areas that confirmed previous predictions that he made. This is a common problem with some meta-analyses in educational psychology in which they’re trying to find any evidence they can of their theories rather than trying to falsify their theories.

    So with these studies, many of them are post-test cross-participant comparisons of less than 50 subjects. This is compounded by the fact that they’re trying to measure this effect during extinction (after removal of the reward) which is known to produce highly variable behavior and is difficult to measure. Eisenberger did several large scale studies which failed to replicate many of the predictions of Deci’s meta-analysis. There is a significant lack of consistency in how researchers talk about this effect although the most common one is that tangible rewards (such as money) undermine intrinsic motivation on tasks which people already enjoy. So maybe avoid doing that although… if they’re having lots of fun doing something, why do you need to motivate them?

    If you’re wondering how to motivate employees then follow Temporal Motivation Theory (value, expectancy, delay) or DISC (deprivation, immediacy, size, contingency) which are really just different ways to talk about the same thing. You identify something your employees enjoy, and then when you need them to work harder you offer that thing to them immediately contingent upon them completing that task. A common way to use this is to offer employees dinner when you need them to stay late. Gambling and video games will often precisely time the delivery of these rewards (there’s a ton of research on schedules of reinforcement).

    As for education, just look at Project Follow Through (https://en.wikipedia.org/wiki/Project_Follow_Through) which is the largest scale study of education methods ever conducted. Use that to figure out the best approach to training which says you should use Direct Instruction or something similar that focuses on developing basic skills at a fast pace based on the learner’s skill level and regular feedback.

    Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological bulletin, 125(6), 627.

    Eisenberger, R., Haskins, F., & Gambleton, P. (1999). Promised reward and creativity: Effects of prior experience. Journal of Experimental Social Psychology, 35(3), 308-325.

  5. Oliver C. Schultheiss says:

    I second David Condon’s comments, but would also point interested readers to a meta-analysis by Eisenberger and Cameron that debunks most of the extrinsic-rewards-kill-motivation myth. Eisenberger also wrote a beautiful theoretical analysis paper on motivation and effort that I’d recommend to anyone interested in the topic of achievement motivation.

    It is important to note that much of self-determination theory research relies on questionnaire measures of motivation. What’s particularly problematic here is the assumption that people have accurate introspective access to what motivates them deep down (i.e., their intrinsic motivation). But 60 years of research on implicit and explicit measures of motivation strongly challenges that view (see Köllner & Schultheiss, 2014; Rawolle et al., 2013) — what people say motivates them corresponds to the actual drivers of their behavior as well as the flip of a coin.

    And if you want to get into the intricacies of what that means for motivating workers in the long run (think job turnover here), things get more complicated quickly — at least more complicated than the notion “Let people choose to do what they intrinsically want, avoid extrinsic rewards”. Part of it has to do with the fact that so-called an extrinsic reward can be just the thing that profoundly satisfies someone’s implicit motives. Part of it has to do with the observation that gratification of implicit motives drives emotional well-being. And part of it has to do with the observation that satisfying implicit motives, whether through “extrinsic” or “intrinsic” rewards, builds skills and competencies more quickly than the satisfaction of explicit motives (including the things one reports as intrinsically likeable).

    My general response to SDT after 20+ years in this area of research is: WTF???? Wasn’t there a reason why Freud warned us against believing the Ego’s fictions? Why Skinner and the behaviorists banned self-reports from the scientific agenda? Why neuropsychologists like Mike Gazzaniga portray the verbal self as an meaning-making mechanism that has no direct access to what drives behavior and instead uses inferences (often faulty) about observed behavior (see also Bem’s work while he still did more reasonable stuff)? Ignoring all hard-won lessons about the fallibility of introspective reports won’t advance behavioral sciences one iota.

    References for interested readers:
    Eisenberger, R. (1992). Learned industriousness. Psychological Review, 99, 248-267.
    Eisenberger, R., & Cameron, J. (1996). Detrimental effects of reward: Reality or myth? American Psychologist, 51, 1153-1166.
    Köllner, M., & Schultheiss, O. C. (2014). Meta-analytic evidence of low convergence between implicit and explicit measures of the needs for achievement, affiliation, and power. Frontiers in Psychology, 5(826). doi: 10.3389/fpsyg.2014.00826
    McClelland, D. C., Koestner, R., & Weinberger, J. (1989). How do self-attributed and implicit motives differ? Psychological Review, 96, 690-702.
    Schultheiss, O. C., & Köllner, M. (2014). Implicit motives and the development of competencies: A virtuous-circle model of motive-driven learning. In R. Pekrun & L. Linnenbrink-Garcia (Eds.), International handbook of emotions in education (pp. 73-95). New York, NY: Taylor & Francis/Routledge.

    • Curious says:

      Oliver:

      I could not agree more with your statement, “things get more complicated quickly”. Consider the oft cited example you give in Schultheiss (2008), Implicit Motives, “achievement-motivated individuals prefer feedback with reference to an individual norm that tells them how well they are doing now relative to how well they did previously.”

      The clear implication of this statement for psychometrics of motivation as compared to ability testing, skills testing and the like is that it is a within person model. From a motivational perspective, the difficulty parameter is a function of skill level and not of the ability of a task to discriminate between person ability levels, but rather within person ability/skill level.

    • To piggyback here – since I teach org psych (well, now “leadership and org behavior”) – I always like to point out that there are lots of tasks in real jobs that almost no one intrinsically wants to do (standing around alone making photocopies is the most dull part of teaching, research, or administration in my opinion, but I have to do some of it most days). Further, it’s basically impossible to staff an organization without paying people, and often would involve payments of a level that (in a lab) *should* activate over-justification mechanisms. So, one take home I like to discuss is, so it’s *possible* that extrinsic rewards (I hate to call what has typically been assessed “extrinsic motivation”) *may* undermine intrinsic interest (again), so what? You’ve gotta pay people, and they’re going to need to do tasks that they aren’t interested in doing, but need to get done, anyway. So, to a degree, no matter what’s going on inside people’s motivational systems regarding rewards versus interests trade-offs, you’re basically stuck with at least part of your “incentive schedule”. What do you do instead? Cue discussion. Answer, “Leadership.” “Ah, but what is leadership? How does it apply here…” Cue endless debate.

      • Rahul says:

        Wouldn’t one pragmatic concern be that, even if we can afford to pay someone more, should we always? i.e. Is there a optimum level of payment that maximizes a person’s output beyond which output declines in absolute terms instead of merely saturating?

        Alternatively, if extrinsic reward crowds out intrinsic interest what’s the optimal reward to get a certain output.

        • I’m a psychologist – I don’t do “optimal”. You’re talking like an economist: “Just design the mechanism and the employees will do as expected.”

          But seriously, under any realistic setting, I don’t think you can get anything close to clean enough estimates of the costs and benefits of various pay versus output levels to determine an “optimum” output. I also think that individual output responses are variable enough (within- AND between-persons) that trying to do so would probably be too costly to be an effective use of managerial/HR time. Of course, otherwise, compensation is usually going to be set by some combination of negotiation and “impressions” of the market. So, that’s probably bad too.

          Beyond that, for most organizations (finance outfits seem to be a *little* different here), there are a variety of social norms, industry standards, pre-existing company policies, and (in some cases) legal restrictions, on how you can set the pay for any given employee. (To be clear, compensation is a long way from my areas of expertise, but it seems likely that simply using pay differentials to “motivate” employees is going to give you subjective fairness/justice problems.)

          I guess the other thing to keep in mind here, and this is really just going back to Elton Mayo’s “Human Relations” movement from the 1930s, but dressed up a bit for today, is that people really do seem to care about things other than pay, from both a satisfaction and a motivational perspective. Having coworkers that you actually like working with, a fair and competent boss, opportunities to learn and develop on the job, and so on, provide additional “psychological” compensation.

          Worthwhile question, but as I’ve said, I don’t know how you might realistically/pragmatically estimate the quantities you’d need to know to effectively do it.

Leave a Reply