Is neuroscience a meritocracy?

Meritocratic failure

In a recent article, "Why Elites Fail," Christopher Hayes argues that meritocracies often fail. For example, the Hunter College High School, a prestigious public school in NYC, accepts students solely on the basis of an entrance exam. For decades, the meritocratic admissions process meant that the school had a diverse student body: in 1995, 12% of the school was black, and 6% hispanic. Today, however, when rich parents hire private tutors for their kids, the student body has become less diverse, with 3% black, and 1% hispanic.

Hayes identifies two keys to meritocracy. First, there must be interindividual differences in ability, skill, or what-have-you. Second, people must be rewarded for their performance: high performers get promoted, while low performers are "punished." If you looked at how families perform between generations, variance in individual ability and accountability would cause inter-generational mobility: if a parent is exceptional, and their child average, the family would change positions. In theory, the larger the variance in interindividual differences is, the larger the mobility should be (I am not sure how to state this formally and correctly, but that is Hayes argument).

However, this is not what usually happens. After one round of meritocracy, the winners get to invest in the next round, so their children have advantages. Sometimes the winners, often in authority, get to choose the winners (or losers) of the next round. The meritocracy breaks down, and become an oligarchy. This same basic story has played out in many different fields in the last thirty years, including college admissions ("legacy admissions"), or the decrease in inter-generational income mobility in the US since 1970.

After reading the article, I wondered, has the meritocracy has failed in neuroscience as well?

How to test scientific meritocracy

The scientific meritocracy can fail in many ways. Funding can go to prestigious labs, regardless of how efficient they are. Nobel laureates can publish shitty papers in high profile journals. Prizes can go to PIs at "elite" institutions, to reflect how portentous the prize is. (I use "elite" as a shorthand for the top 5-10 neuroscience programs, based on my rankings, and my perception. Exploring what constitutes an elite institution is for another post.)

Since I pretend to be a scientist, I wanted to measure how meritocratic "science" is. One possibility would be to measure whether labs from elite institutions get preferential treatment by journals, but that would require somehow objectively measuring paper quality, and controlling for funding. Instead, I settled on something simpler, and hopefully more objective: looking at the credentials of PIs at elite institutions.

In a meritocracy, you would expect that as you looked farther into the past, the winners would have increasingly diverse backgrounds. Or, looking forward, you would expect some percentage of people who start at the bottom of the hierarchy, but were talented, could work their way up. In terms of PIs at elite institutions, my expectation is that they would almost exclusively have done post-docs at other elite institutions. In contrast, if you looked at where they went to undergrad, I would expect a more diverse set of schools. To see whether this was true, I looked at 30+ PIs at elite institutions.

The test

The baseball writer Rob Neyer has a gimmick where he presents stats for two anonymous players, "Player A", and "Player B." Sometimes the stats would be similar, and the reader would be shocked to find that Player A was an All-Star, while Player B was a "scrub." Sometimes the stats would be dissimilar, but Players A and B would actually be the same player, playing under different conditions. The point of the gimmick was to look at the world more objectively, without the halo effect of their names.

So in that vein, I present two cohorts of researchers, and their associated institutions:

Cohort A	Cohort B
MIT Santa Barbara Stetson Cal State Chico Williams College Cambridge Lawrence Vassar Duke MIT Harvard MIT UChicago Stanford Bryn Mawr	Yale Harvard CalTech UChicago Vanderbilt Harvard Brown Princeton Berkeley Berkeley UVA MIT UVA Harvard

With all due respect to the universities represented in Cohort A, most people would agree that the schools in Cohort B produce more research. So who are these two cohorts of researchers, and how are they affiliated with the institutions? Both cohorts are professors at elite neuroscience departments. But Cohort A got their bachelors before 1990 while Cohort B got theirs after 1990:

Fogies	Undergrad	Whippersnappers	Undergrad
Barres, Ben Knudsen Newsome, William Moore, Tirin Raymond, Jennifer Shatz, Carla Nicoll, Roger Huganir, Richard Bear, Mark Julius, David Malenka, Rob Tsien, Richard Katz, LC Callaway, Ed Cline, Hollis	MIT Santa Barbara Stetson Cal State Chico Williams College Cambridge Lawrence Vassar Duke MIT Harvard MIT UChicago Stanford Bryn Mawr	Datta, Bob Wilson, Rachel Ehlers, Mike Scott, Kristin Harvey, Christopher Deisseroth Dolmetsch, Ricardo Heiman, Miriam Huberman, Andrew Potter, Chris Shuler, Marshall Tye, Kay Goosens, Kim Sabatini, Bernardo	Yale Harvard CalTech UChicago Vanderbilt Harvard Brown Princeton Berkeley Berkeley UVA MIT UVA Harvard

(Methods: To select fogie professors, I included professors I recognized by name, or who are HHMI investigators. For the whippersnappers, I used the SFN Young Investigators award listing, and scanned the websites of departments for assistant professors. People educated outside the US were excluded. For each professor, I noted their schools for BS, PhD, and post-doc. For some professors, I could not ascertain their undergrad institution, and excluded them. This is by no means exhaustive, but I only spent a few hours doing this. Full spreadsheet.)

I have two general conclusions from this. First, if you want to be a professor at an elite institution today, you need to have gone to an elite undergrad. The "worst" school represented, UVA, is ranked #25 (for whatever rankings are worth). There are no Wisconsin-Madisons on the list, let alone places like Ohio State. Three steps removed from your PI position, where you did your undergrad is a determining factor for whether you can become a PI. As you move the credential window forward to grad school and post-docs, the credential threshold gets even higher.

Second, and more weakly, I think this shows that science has become less meritocratic over time. At first blush, I thought the fogies' schools were just generally worse. A quick Google, however, revealed that Williams, Vassar, and Bryn Mawr are well regarded liberal arts schools. So while it's not fair to conclude that the older cohort went to worse schools, I think it is fair to say they went to a more diverse set of schools, ones that did not necessarily emphasize research.

Questions I ask myself

Isn't this sample size small? Yes, but then again there aren't many professors. If I wanted to spend more time, I should probably quantify rankings of both the PIs' institutions, and their undergrad schools. It would also be helpful to look at non-elite institutions' PIs to see how what is happening there.

Don't elite undergrads reflect high SAT scores/intelligence? And science is a g-loaded occupation, so... I would argue that neuroscience is not as g-loaded as other fields like physics or computer science. Once you reach +1 or +2 SD, other factors like work ethic become important. A quick Google shows that elite institutions have, a ~100 point SAT score gap over other places, ~1SD. However, there are much fewer elite institutions, and they also have fewer students, so by numbers alone, there should be just as many equally smart kids at non-elite institutions as the elite ones, and a much larger group at -1SD.

Ok, if the elite kids aren't smarter, could they have some other trait? I can see this. College admissions are increasingly (and insanely) competitive. Thus, elite colleges may screen for competitiveness. If not competitiveness, it could be some other factor. Elite institutions use extracurriculars as differentiators between applicants, and if you believe this Gladwell piece* the extracurriculars reflect something real.

* What happened to Gladwell? People love to shit on him now, but I thought he did a good job summarizing social science for a wide audience in The Tipping Point and his older magazine articles. But now he's publishing half-baked essays on "slack," and name-checking Tyler Cowen's econ-foodie book?

What about non-Americans? I don't know much about this. My understanding is that the European college system is much more equal in terms of quality, so there is less fighting for spots (with exceptions like the French/Swiss écoles, Oxbridge,  etc.). As for Asia, I think a majority of Chinese PIs in the US come from Tsinghua or Peking University, and the Indians from IITs. But American grad schools may not be equipped to identify good applicants from less famous schools.

Isn't this a lot of words for what amounts to glorified googling? Yeah.

Concluding bloviating

As mentioned at the top, society as a whole is becoming less meritocratic. It would be remarkable for science to resist this trend. I'm not sure what, if anything can be done about it. The PIs at elite institutions are generally smart, motivated people, so from the perspective of funding agencies, why should they care whether the PIs have diverse backgrounds? And the NIH does fund non-elite institutions, just less so, if only to avoid senators asking why Idaho doesn't get any funding.

There is an opportunity for disruption here, in that elite institutions are completely overlooking talented, less credentialed people. Some places, like Washington University, seem to specialize in being less famous, but nearly as productive, and I think it's in large part by finding people the elite institutes can't be bothered with. Of course, they will still lose status contests to elite institutions in publishing and prizes.

On a personal note, I knew coming to Geneva that I would need to do a second post-doc to get a job back in the US. Seeing the credentials of these people made me realize just how important status and political connections are, rather than simple productivity. Hopefully, the status requirement are much lower one step down the ladder. The post-docs I knew at Duke were able to get positions at good schools like UNC, Baylor, and BU. Whether they would have been able to get the positions if they were post-docs at those schools is another question.