DUMBING DOWN

Faz tudo parte da mesma picadora.

Great minds think (too much) alike 

Digital Libraries: Is the web narrowing scientists’ expertise?

 

Give me the broader prospective, please

ONLINE databases of scientific journals have made life easier for scientists as well as publishers. No more ambling down to the library, searching through the musty stacks and queuing up for the photocopier. Instead, a few clicks of a mouse can bring forth the desired papers and maybe others that the reader did not know of—the “long tail” of information that the web makes available.

Well, that is how it is supposed to work, but does it? James Evans, a sociologist at the University of Chicago, decided to investigate. His conclusion, published in this week’s Science, is that the opposite is happening. He has found that as more journals become available online, fewer articles are being cited in the reference lists of the research papers published within them. Moreover, those articles that do get a mention tend to have been recently published themselves. Far from growing longer, the long tail is being docked.

Dr Evans based his analysis on data from citation indexes compiled by Thomson Scientific (part of Thomson Reuters). In a world in which researchers must publish or perish, such indexes are the firing squads. They record how often one article is cited as a source by others, and thus measure a paper’s influence. Those used by Dr Evans cover 6,000 of the most prominent academic journals, some going back to 1945. By cross-referring these to a database called Fulltext Sources Online, he was able to work out when each of these journals became available on the web—and whether a journal had posted back-issues electronically as well. The result was a set of 34m research papers, which he was able to mine in search of his answers.

For each research paper he looked at, he calculated the average age of the articles cited as references. He then calculated, for each of those cited articles, the number of back-issues of the journal it had been published in which were available on the web at the time when it was cited, and averaged that too. Finally, he looked for correlations between the two averages.

What he discovered was that, for every additional year of back-issues of a journal available online, the average age of the articles cited from that journal fell by a month. He also found a fall, once a journal was online, in the number of papers in it that got any citations at all. Indeed, he predicts that for the average journal today, five extra years’ worth of online availability will cause a precipitous drop in the number of articles receiving one or more citations—from 600 to 200 a year. Rather than measuring the length of the tail, then, it seems that modern science is actually focusing on a tiny bit of it.

Why this should be so remains unclear. It does not seem to have anything to do with economics. The same effect applied whether or not a journal had to be paid for. One explanation could be that indexing works by titles and authors alone, as happened with printed journals, forced readers to cast at least a cursory glance at work not immediately related to their own—or even that the mere act of flicking through a paper volume may have thrown up unexpected gems. This may have led people to make broader comparisons and to integrate more past results into their research.

It is not yet clear whether this change is for good or ill. Electronic searching means that no relevant paper is likely to go unread, but narrowing the definition of “relevance” risks reducing the cross-fertilisation of ideas that sometimes leads to big, unexpected advances. As a wag once put it, an expert is someone who knows more and more about less and less until, eventually, he knows everything about nothing. It would be ironic if that is the sort of expertise that the world wide web is creating.

The Economist, 17 Jul 2008

DOS ALHOS E DOS BUGALHOS

MEASURING MORTARBOARDS

A new sort of higher education guide for very discerning customers

WORKING out exactly what students and taxpayers get for the money they spend on universities is a tricky business. Now the Organisation for Economic Co-operation and Development (OECD), a Paris-based think-tank for rich countries, is planning to make the task a bit easier, by producing the first international comparison of how successfully universities teach.

That marks a breakthrough. At the moment, just two institutions make annual attempts to compare universities round the world. Shanghai’s Jiao Tong University has been doing it since 2003, and the Times Higher Education Supplement, a British weekly, started a similar exercise in 2004. But both these indices, which are closely watched by participants in a fickle and fast-expanding global education market (see chart), reflect “inputs” such as the number and quality of staff, as well as how many prizes they win and how many articles they publish. The new idea is to look at the end result—how much knowledge is really being imparted.

“Rather than assuming that because a university spends more it must be better, or using other proxy measures for quality, we will look at learning outcomes,” explains Andreas Schleicher, the OECD’s head of education research. Just as the OECD assesses primary and secondary education by testing randomly chosen groups of youngsters from each country in reading and mathematics, it will sample university students to see what they have learned. Once enough universities are taking part, it may publish league tables showing where each country stands, just as it now does for compulsory education. That may produce a fairer assessment than the two established rankings, though the British one does try to broaden its inquiry by taking opinions from academics and employers.

There is much to be said for the OECD’s approach. Of course a Nobel laureate’s view on where to study may be worth hearing, but dons may be so busy writing and researching that they spend little or no time teaching—a big weakness at America’s famous universities. And changes in methodology can bring startling shifts. The high-flying London School of Economics, for example, tumbled from 17th to 59th in the British rankings published last week, primarily because it got less credit than in previous years for the impressive number of foreign students it had managed to attract.

The OECD plan awaits approval from an education ministers’ meeting in January. The first rankings are planned by 2010. They will be of interest not just as a guide for shoppers in the global market, but also as indicators of performance in domestic markets. They will help academics wondering whether to stay put or switch jobs, students choosing where to spend their time and money, and ambitious university bosses who want a sharper competitive edge for their institution.

The task the OECD has set itself is formidable. In many subjects, such as literature and history, the syllabus varies hugely from one country, and even one campus, to another. But OECD researchers think that problem can be overcome by concentrating on the transferable skills that employers value, such as critical thinking and analysis, and testing subject knowledge only in fields like economics and engineering, with a big common core.

Moreover, says Mr Schleicher, it is a job worth doing. Today’s rankings, he believes, do not help governments assess whether they get a return on the money they give universities to teach their undergraduates. Students overlook second-rank institutions in favour of big names, even though the less grand may be better at teaching. Worst of all, ranking by reputation allows famous places to coast along, while making life hard for feisty upstarts. “We will not be reflecting a university’s history,” says Mr Schleicher, “but asking: what is a global employer looking for?” A fair question, even if not every single student’s destiny is to work for a multinational firm.

 

Nov 15th 2007 
From The Economist print edition