With timing like Ian Bell striking a cover drive, two articles have recently arrived on the challenges of making lists and rankings: the quantitative version, from The Economist (apologies if you can’t read that due to registration requirements, they’re tightening up!), and the qualitative, arty perspective from New York Times classical music critic Anthony Tommasini.
The Economist talks about some of the problems with making rankings based on thin data (which is basically what I’ve been doing with the books):
The hunger for crunchy comparisons of everything from venality to brainpower is huge and growing, not least among media such as this newspaper. It is tempting to try and pin ideas down by turning them into figures. Such measures can be handy. But they have serious flaws.
Crunchy comparisons? I like that. They go on to describe how a lot of international data are based on surveys and opinion polls – tricky, because people in different places will have very different criteria for judging the same things. But other types of data are notoriously unreliable as well, and when they go into ranking systems strange results can emerge. What often happens is that variations in reported scores that would be within the margins of error for the underlying measurements can be large enough to account for a range of tens of places in a ranking table. Further, some key development indices combine a range of other indices, each of which may have its own problems. This all results in countries (or books, or whatever’s being ranked) jumping around wildly in the rankings from year to year.
The NYT article deals with the much more specific task of developing and applying criteria to define greatness in, and/or rank, artists. While Anthony Tommasini is discussing composers, the issues are completely analogous to attempts to rank authors and their books. He starts from a position that rankings are crass and silly, and entirely not his job – but he comes round to the idea eventually:
I don’t do ranking. As I see it, the critic’s job description does not include compiling lists of greats in order of greatness. What I do is champion, demystify and describe the composers, works and artists I admire, and, as appropriate, puncture inflated reputations…
if you were to try to compile a list of the 10 greatest composers in history, how would you go about it? For me the resulting list would not be the point. But the process of coming up with such a list might be clarifying and instructive, as well as exasperating and fun.
It’s worth reading the whole thing for his approach to the problem, even if you’re not interested in his discussion of Bach and Hayden (although he makes them quite interesting). It will be turning into a series in which he aims to produce a list of his top ten composers of all time.