Anecdata. How McKinsey’s story became Sheryl Sandberg’s fact

You tried to raise your children gender-neutral but your daughter still loves ballerinas and your son loves cars. Maybe it’s not your fault. Maybe this shows their urges are genetically programed.

You once had a woman as president of your organization but she never could communicate a vision. Perhaps she couldn’t help it. Perhaps this is how we learn that women don’t have what it takes to lead.

Stories are important; they can pique our interest, they can engage us. Stories can lead us to want to know more.

Stories are not studies
But stories themselves are not the path to knowledge. Knowing something requires research.

Research is particularly crucial when we’re talking about differences between men and women. Everyone has opinions about this; everyone has stories.

penguins dreaming

And because those stories are easily passed around, we have to be careful not to let anecdotes about ballerinas or bad presidents become general truths.

Sometimes we can actually see a story get dressed up as research. I recently uncovered an example while trying to find the facts behind a claim in Sheryl Sandberg’s Lean In. Her book is full of inspiring stories, but in this one case, she creates a context by invoking research that neither she nor anyone else can actually put their hands on.

How much do men overestimate their qualifications?
My investigation started after I heard a wonderful story that I wanted to use to illustrate the claim that under-qualified men will more easily apply for jobs than under-qualified women. (Read about men who apply for “women only” jobs here.)

I’d seen research showing that men overestimate their skills, but I was remembering something more specific. I had read in Sandberg’s book the claim that men have a lower threshold for applying for jobs than women do, and I wanted to find the study.

An internal report at Hewlett-Packard revealed that women only apply for open jobs if they think they meet 100 percent of the criteria listed. Men apply if they think they meet 60 percent of the requirements.

Sandberg cites the article A business case for women, published in The McKinsey Quarterly by Georges Desvaux, Sandrine Devillard-Hoellinger and Mary C. Meany. So I turned to their article to learn more. They anticipate Sandberg’s phrasing:

Internal research at HP showed that women apply for open jobs only if they think they meet 100 percent of the criteria listed, whereas men respond to the posting if they feel they meet 60 percent of the requirements.

But that was it. They didn’t give any source for this claim, and my curiosity wasn’t yet satisfied. I wanted to know more about the Hewlett-Packard results. How did HP do their research? What did they ask? Why did they pose the question in the first place? So, I kept looking.

Why you can’t trust internal research
Many magazines and blogs repeat this claim, citing either Sandberg or the McKinsey article. Even serious researchers reference this study; in The Wiley-Blackwell Handbook of the Psychology of Coaching and Mentoring, Jonathan Passmore, David Peterson and Teresa Freire use the same quote and the same source.

I started thinking I would never find the actual research. Where were the methods and numbers? We can’t be satisfied to simply quote someone who tells about their “internal research.” If we don’t see it ourselves, we’re not building on their work, but rather on their description of their work.

Accepting a research conclusion without being able to see the research violates just about all the tools in Carl Sagan’s wise and practical Baloney Detection Kit, recently discussed over at Brainpickings.

Social media and fact-checking
I wasn’t quite ready to give up. I tried to see if Sheryl Sandberg had more on this research by posting a question to her and others at the Lean In fanpage on Facebook. No luck.

Next, I tweeted others who I thought might have more information, but kept getting sent back to the McKinsey article. Still no luck.

I tried the authors of the McKinsey Quarterly paper and quickly got a helpful reply from first author Georges Desvaux:

This came from interviews with senior executives, which we conducted with many companies, including HP. I do not have access to the internal HP document, nor do I have the actual contact, as this was a series of confidential interviews.

I think that’s the end of the road. Unless someone at Hewlitt-Packard or McKinsey can supply more information, it seems reasonable to assume this claim grew out of a simple exchange.

McKinsey interviewer: Do you have any special challenges related to recruiting internally to higher-level positions?

Hewlett-Packard senior executive: Well, as a matter of fact, it’s much more difficult to get women to apply for senior positions than men. It seems like they lack self-confidence. They don’t even think about applying unless they’re 100% confident they’re qualified. The men, if they feel like they’re even 60% there, they go for it.

Do men and women have different thresholds for applying for jobs?
Where does that leave us? At least we know better now what we don’t know and perhaps there’s some comfort in that.

A claim reported to be the result of a study is starting to look less like research and more like gossip.

At this point, we know there is no legitimate, evidence-based foundation for the claim that men apply for jobs when they feel 60 percent qualified while women have to be 100 percent certain. None. Nothing that can be examined, reproduced, reviewed or cited. From a researcher’s perspective, it doesn’t exist.

ConfidenceGap copySpreading this claim is particularly unfortunate because men and women probably do have different thresholds for applying for jobs. (The Atlantic recently presents relevant research in The Confidence Gap. But, yes, they promote the HP “discovery,” too.)

Now, instead of addressing that situation constructively, skeptics can write off this cultural difference between men and women, justifying their indifference by pointing to arguments built on what we now believe to be hearsay.

Sheryl Sandberg and the rest of us should stop repeating this number. We do not know if it is true. We can’t. There’s no identifiable source. Keeping the myth alive hurts the greater research program and the social changes it can provoke.

“And we know that with 100% certainty”
All of us who have repeated this generalization should have seen a red flag right away. The alleged research claims a result in which 100 percent of women have a certain reaction. 100?! We should always view such claims with suspicion.

Are women so unexceptionally homogeneous? Is it really true that they never apply for jobs when they think they’re under-qualified? Not even one woman is willing to apply when she feels 95% confident? That is just too far-fetched to be taken seriously.

The McKinsey Quarterly has published something that looks like a study but in fact, it’s just a story. Stories are important, but they are different than studies. Research is more than just dressing up a story and publishing it in your company magazine.

So what does happen when under-qualified women apply for jobs? I don’t know. Sheryl Sandberg and McKinsey seem to think it doesn’t happen. But I’m sure it does.

In fact, I could tell you a story …

 

Related posts

Why are women so uncooperative?

Gender equality mutilation, or how Elsevier went cringeworthy

And more on my gender equality pages.

For the good folks at McKinsey Quarterly, I also recommend a piece on Open Evaluation as a new approach to quality control in publishing.

About Curt Rice

My interest in leadership development at universities affects most of what I do, whether it’s working on gender balance issues, developing policies about Open Access, promoting research-based education or just about anything else. I'm a professor at the University of Tromsø, where I've spent the last decade serving first as the head of a Center of Excellence (2002-2008) and then as the Vice President for Research & Development (prorektor for forskning og utvikling) (2009-2013). I'm currently a Fellow at the Netherlands Institute for Advanced Study.

Comments

  1. Donald Merritt says:

    I have often told someone that what they said was not fact but their opinion.
    It seems to me that a lot of people have difficulty in knowing the difference.
    You might get someone saying “well the fact is I saw a neighbour with a black eye. She was arguing with her husband and he whacked her one”. I asked “did you see him do it”? No, would be the reply but I heard them arguing?????
    The old saying start a story is true. Listen to the same story after it has circulated and you will be lucky to recognise it.

  2. Thanks, Curt! Gee, I’m happy we “met” while I was writing at Forbes. And thanks for dropping by our company blog today to share this piece with us. You’re absolutely 100% perfectly right that if we push out “facts” about gender or racial or any other kind of bias, we’d better be 100% frigging certain that we have quality research to back it up. The science deniers are pesky enough. It would be best not to give them ammunition for their denials.

  3. I think this is up for doing some empirical work then! I will keep it in mind when we recruit students for their thesis work, and i can vaguely think of methods already (as so much of similar things have been done in Social Psychology.)

  4. Hi Curt – good point. I don’t know where the 100% stat comes from. I had heard that women like to be 80% sure before they take a decision (unable to source that either).

    As a Business School and MBA coach as well as being a head hunter I can only quote my personal and extensive experience is that that women tend not to apply for positions until they meet a higher percentage of the criteria than their male counterparts. They may also withdraw from a process more readily if something changes or if they feel they are under qualified. This is all totally anecdotal.

    Whether this is about lack of confidence or about colouring outside the lines I can’t say. I saw the Delft piece and they indeed had male applicants for female roles.

    • Curt Rice says:

      Hi Dorothy,
      Anecdotes are important and they can inspire research. And that confidence gap may very well be real. It’s important and we should explore it.
      But this 60 vs 100 thing is so over-used and under-justified that it’s almost acquired the status of an urban myth!
      Thanks for stopping by :)

  5. Nancy Law says:

    I would love to see a real study on, when given a choice, how many little girls would pick the ballerina costume over the cars, or vice versa for the boys. The gender stereotypes seem to be true, but maybe they are not as strong as we think. There is probably a study out there somewhere, I haven’t looked, but it would be interesting to see numbers.

  6. Marianne Stenger says:

    Ha, I came across this post while trying to track down the very same Hewlett Packard research for an article I’m working on. I guess I will end my search here :) Thanks for the post!

    • Curt Rice says:

      If you find it, please let me know! But, given the correspondence I had with the authors of the McKinsey article, I’m quite 95% sure it does not exist anywhere and 90% sure it never did. None of the principles can cough it up, that’s for sure. I also was in touch with HP directly. Needless to say, to no avail.

      • Marianne Stenger says:

        Yes, like you, all I can find are a lot of reputable media outlets citing it as “internal research by Hewlett Packard” without any hints as to the specific researchers involved or where the elusive “report” might be found. Quite puzzling really.

Trackbacks

  1. […] Update: I followed up on the research, and it turns out not to be. Read the details in What happens when under-qualified women apply for jobs? (And why Sheryl Sandberg and McKinsey wrongl… […]

  2. […] En lang historie kort: Det fantes ingen tall eller statistikk bak påstanden. Undersøkelsen baserte seg på intervjuer med ledere i flere selskaper, blant annet HP. Den var dessuten anonymisert, så det var umulig å finne ut akkurat hvem som hadde sagt det. Med andre ord: En leders subjektive tolkning er plutselig forvandlet til et forskningsfunn. Men ikke et funn som forteller at ledere i bedrifter har en tendens til å forvente at kvinner er mer beskjedne på egne vegne, enn menn. Funnet «bekrefter» ulike selvforståelser hos menn og kvinner, der forskning får vise at en kvinne i sitt vesen verken er selvsikker, selvironisk eller overmodig i sitt møte med verden. […]

  3. […] Update: I followed up on the research, and it turns out not to be. Read the details in What happens when under-qualified women apply for jobs? (And why Sheryl Sandberg and McKinsey wrongl… […]

  4. […] men apply for jobs when they feel 60% qualified while women feel they have to be 100% qualified – turns out to be unverifiable, but the claim just won’t die. And maybe there’s something to it; maybe more research will show […]

  5. […] if they have 60% of the qualifications, and women will apply only if they have 100%. It’s not citable. There is probably a disparity, but we don’t know for sure what it […]

Speak Your Mind

*