You tried to raise your children gender-neutral but your daughter still loves ballerinas and your son loves cars. Maybe it’s not your fault. Maybe this shows their urges are genetically programed.
You once had a woman as president of your organization but she never could communicate a vision. Perhaps she couldn’t help it. Perhaps this is how we learn that women don’t have what it takes to lead.
Stories are important; they can pique our interest, they can engage us. Stories can lead us to want to know more.
Stories are not studies
But stories themselves are not the path to knowledge. Knowing something requires research.
Research is particularly crucial when we’re talking about differences between men and women. Everyone has opinions about this; everyone has stories.
And because those stories are easily passed around, we have to be careful not to let anecdotes about ballerinas or bad presidents become general truths.
Sometimes we can actually see a story get dressed up as research. I recently uncovered an example while trying to find the facts behind a claim in Sheryl Sandberg’s Lean In. Her book is full of inspiring stories, but in this one case, she creates a context by invoking research that neither she nor anyone else can actually put their hands on.
How much do men overestimate their qualifications?
My investigation started after I heard a wonderful story that I wanted to use to illustrate the claim that under-qualified men will more easily apply for jobs than under-qualified women. (Read about men who apply for “women only” jobs here.)
I’d seen research showing that men overestimate their skills, but I was remembering something more specific. I had read in Sandberg’s book the claim that men have a lower threshold for applying for jobs than women do, and I wanted to find the study.
An internal report at Hewlett-Packard revealed that women only apply for open jobs if they think they meet 100 percent of the criteria listed. Men apply if they think they meet 60 percent of the requirements.
Sandberg cites the article A business case for women, published in The McKinsey Quarterly by Georges Desvaux, Sandrine Devillard-Hoellinger and Mary C. Meany. So I turned to their article to learn more. They anticipate Sandberg’s phrasing:
Internal research at HP showed that women apply for open jobs only if they think they meet 100 percent of the criteria listed, whereas men respond to the posting if they feel they meet 60 percent of the requirements.
But that was it. They didn’t give any source for this claim, and my curiosity wasn’t yet satisfied. I wanted to know more about the Hewlett-Packard results. How did HP do their research? What did they ask? Why did they pose the question in the first place? So, I kept looking.
Why you can’t trust internal research
Many magazines and blogs repeat this claim, citing either Sandberg or the McKinsey article. Even serious researchers reference this study; in The Wiley-Blackwell Handbook of the Psychology of Coaching and Mentoring, Jonathan Passmore, David Peterson and Teresa Freire use the same quote and the same source.
I started thinking I would never find the actual research. Where were the methods and numbers? We can’t be satisfied to simply quote someone who tells about their “internal research.” If we don’t see it ourselves, we’re not building on their work, but rather on their description of their work.
Accepting a research conclusion without being able to see the research violates just about all the tools in Carl Sagan’s wise and practical Baloney Detection Kit, recently discussed over at Brainpickings.
Social media and fact-checking
I wasn’t quite ready to give up. I tried to see if Sheryl Sandberg had more on this research by posting a question to her and others at the Lean In fanpage on Facebook. No luck.
Next, I tweeted others who I thought might have more information, but kept getting sent back to the McKinsey article. Still no luck.
I tried the authors of the McKinsey Quarterly paper and quickly got a helpful reply from first author Georges Desvaux:
This came from interviews with senior executives, which we conducted with many companies, including HP. I do not have access to the internal HP document, nor do I have the actual contact, as this was a series of confidential interviews.
I think that’s the end of the road. Unless someone at Hewlitt-Packard or McKinsey can supply more information, it seems reasonable to assume this claim grew out of a simple exchange.
McKinsey interviewer: Do you have any special challenges related to recruiting internally to higher-level positions?
Hewlett-Packard senior executive: Well, as a matter of fact, it’s much more difficult to get women to apply for senior positions than men. It seems like they lack self-confidence. They don’t even think about applying unless they’re 100% confident they’re qualified. The men, if they feel like they’re even 60% there, they go for it.
Do men and women have different thresholds for applying for jobs?
Where does that leave us? At least we know better now what we don’t know and perhaps there’s some comfort in that.
A claim reported to be the result of a study is starting to look less like research and more like gossip.
At this point, we know there is no legitimate, evidence-based foundation for the claim that men apply for jobs when they feel 60 percent qualified while women have to be 100 percent certain. None. Nothing that can be examined, reproduced, reviewed or cited. From a researcher’s perspective, it doesn’t exist.
Spreading this claim is particularly unfortunate because men and women probably do have different thresholds for applying for jobs. (The Atlantic recently presents relevant research in The Confidence Gap. But, yes, they promote the HP “discovery,” too.)
Now, instead of addressing that situation constructively, skeptics can write off this cultural difference between men and women, justifying their indifference by pointing to arguments built on what we now believe to be hearsay.
Sheryl Sandberg and the rest of us should stop repeating this number. We do not know if it is true. We can’t. There’s no identifiable source. Keeping the myth alive hurts the greater research program and the social changes it can provoke.
“And we know that with 100% certainty”
All of us who have repeated this generalization should have seen a red flag right away. The alleged research claims a result in which 100 percent of women have a certain reaction. 100?! We should always view such claims with suspicion.
Are women so unexceptionally homogeneous? Is it really true that they never apply for jobs when they think they’re under-qualified? Not even one woman is willing to apply when she feels 95% confident? That is just too far-fetched to be taken seriously.
The McKinsey Quarterly has published something that looks like a study but in fact, it’s just a story. Stories are important, but they are different than studies. Research is more than just dressing up a story and publishing it in your company magazine.
So what does happen when under-qualified women apply for jobs? I don’t know. Sheryl Sandberg and McKinsey seem to think it doesn’t happen. But I’m sure it does.
In fact, I could tell you a story …
And more on my gender equality pages.
For the good folks at McKinsey Quarterly, I also recommend a piece on Open Evaluation as a new approach to quality control in publishing.