Skip to main content

Cornell SC Johnson College of Business

Keeping a Better World in Mind

A Dean's Blog by Andrew Karolyi

Rigor, research, resonance, and rankings

November 27, 2024

Discussion and debate about business school relevance fly around quite a bit these days. My fellow business school deans and I are all working in earnest to balance the stewardship of our institutions while responding to loud-and-clear demands for change. None of us would have accepted the position without expecting to mediate the past and the future, but at times it seems we’re yet another sector of the sandwich generation, responsible to both elders and children, while keeping our own forward momentum going.

Books have been, and will be, written about the evolution underway. (I’ve shared Andrew Hoffman’s 2021 manifesto The Engaged Scholar, as cogent an examination as you’ll find.) He argues, and I agree, broadly, that knowledge needs to be cultivated more intentionally, shared more efficiently, and employed more directly. This is the topic of interest to me in this month’s piece.

Traditional scholarly approaches are on trial. Pure theoretical research, ostentatiously divorced from purpose, was considered trustworthy, and its approval through publication in top-level peer-reviewed journals rewarded a scholar with tenure and respect. These were the standards of rigor and achievement, and we who worked our way devotedly through this system still trust that it has been an honorable pursuit. We also know that it has been in many ways a domain of privilege inaccessible to many. We see this now as an unacceptable limitation of our pursuit of true knowledge.

And so we shift.

Rankings are among the features to reconsider. While they inform prospective students about their peers and their potential livelihoods, this information is insufficient. Fellow deans Ann Harrison (formerly Berkeley Haas), Geoff Garrett (USC Marshall), and I collaborated last March on: Business school rankings must measure the ‘societal value added.’  In this outline, we advocate adding metrics like the debt load graduates carry into the workforce and family advancement statistics. FT education editor Andrew Jack encouraged us in writing and in publishing the piece, as he knew we agreed that existing business school rankings fail to capture the full value of a school’s offering. Efforts are ongoing toward convincing a wider swath of schools to join us, but we maintain that this kind of understanding is valuable. I do not have a greater sense of how well our ideas are gaining traction among students, their families, colleagues in industry. Perhaps regular readers of my postings are interested in sharing their own.  

 Understanding influence

So I’m more than interested to see the FT weighing in substantively, last week dropping its first-ever standalone research ranking among global business schools, the FT Research Insights report. Andrew Jack and Anjali Dalal outline the new ranking in their article, Business school and the pursuit of rigour, resonance and relevance. They explain that the venture means to explore “new ways to measure, showcase, and stimulate greater focus on research which – while still rigorous—is also focused both on relevance to the most pressing problems facing the planet, and has resonance with the world of practice.”

I should disclose that I have had dozens of conversations with Andrew Jack on this topic – in my capacity as a dean, but also as chair of Responsible Research in Business & Management. I am not cited in the article, nor are dozens of others with whom he consults. Truthfully, I am not sure any of the new metrics of broader societal impact of research introduced are good or not. The effort is worthy. And I do believe profoundly in the need for impact to be measured beyond just high citation impact factors.

So here we are.

What I know is that this opening effort is going to stimulate much discussion in B-school circles. Questions will be asked about how this connects to program-specific rankings with which we are very familiar. How will it impact how we reward scholars in business schools? How will journals and journal editors react to knowing their articles are being measured differently with great visibility? Lots of buzz abounds and in many circles.

Keep the commitment. Refine the metrics.

Traditional research protocols, meant to protect integrity of our findings, do not always encourage innovation.

As editor of the Review of Financial Studies, I faced this question directly in 2015, when Accounting for Sustainability (an organization founded by then-Prince of Wales/now-King Charles III) politely ambushed a select group of top-tier journal editors in finance to ask why were we not publishing research on climate finance. We answered honestly: there wasn’t any such research to publish.  Younger faculty advancing through the existing system saw no other examples of research studies on climate finance and so were not encouraged to do so themselves. So at RFS we carefully devised a new protocol for a special issue with a request for proposals on research into climate finance. We chose the most rigorous proposals and encouraged the projects through the research process. Not everyone was happy with the editorial innovation, though several journals in finance and management have adopted it for their own. For the discipline of finance, I venture to say it was successful, spawning hundreds and hundreds of research papers on the topic in published and working paper form over the past decade. This was clearly one way to redefine metrics of impact.

Now fast forward to my world as a college dean. When selecting the faculty research to feature in our annual Research with Impact publication, we take our time, using three specific interpretations of impact and the collected input of area chairs and our faculty dean. We also share up-to-the-minute impactful research through the Cornell SC Johnson College of Business Research Paper Series, posted on SSRN. This is a subjective process without clear metrics, but made with purpose by a group of academic leaders.

And now fast forward again to the FT’s newest effort. The FT has transformed its own concept of “influence,” defining the term as practical relevance and application to real-world business issues. In the “old” system, influence was measured by citation counts in top-tier academic journals, reflecting how often other scholars referenced the research. Now, influence is also assessed through case studies and teaching materials that detail how research has directly benefited or transformed business practices. Expert evaluations and interdisciplinary engagement are considered, including qualitative feedback from industry professionals, by policy makers, and reviews of research with clear evidence of practical applications in the business environment. This transition represents FT’s aim to value research that extends beyond purely academic discussions to create actionable insights for business leaders and practitioners.

How does Cornell and our college come out? In the FT’s new breakdown, we rank #19 overall globally in three-year productivity in FT50 journals, their defined benchmark for “rigour” (or “rigor” using US spelling!). We are not told what the weightings are across the sub-metrics, so this leaves open questions. But we know that:

  • Our rank jumps to #11 and #12 when adjustments are made for different citation “intensity” across disciplines, like Accounting, Finance, Marketing, Operations, Strategy. It has been long known that different disciplines pack their articles with more references to other work than other disciplines. (OpenAlex, Scite are the external vendors for this metric).
  • Another place we seem to do well is in policy relevance of our work at #9 (Overton is the vendor) and in sustainability-linked research, #15 (OpenAlexSDG).
  • The less good news is the impactfulness of our published research in the FT50 for teaching and industry relevance. Our rankings are much lower for work cited in case studies published (no score was rendered, The Case Centre was the reference), for work cited in course syllabi (OpenSyllabus), and for SSRN downloads by industry practitioners (#24).

Silos built careers in the 20th Century. In the 21st collaborations will build communal survival. Once-perfect methods may not keep apace. History will no doubt help us but perhaps more as a line of breadcrumbs than a road map to follow carefully. I look forward to continuing honest conversations among us about what this all means for us and for business schools, in general.

Industry needs knowledge and this arguably makes business education more important than ever.

 And I couldn’t leave without mentioning: