Library and Information Science

The writer of this article pointed out in his contribution to a 1968 Festschrift honoring Fujio Mamiya, that there exists in Japan and elsewhere an naive admiration and overestimation towards evaluation technics of indexing methods, and in the essay criticized a recent commentary of M. Takahashi rather severely. However, after reading some of the critical comments on C. W. Cleverdon's Aslib-Cranfield Research Project, the writer reflects that his own severe attitude may serve to discourage rather than to encourage the progress in such research.

The writer has a great esteem for Cleverdon's report of 1962. It is most challenging and epoch-making. Introducing the test results of the Aslib-Cranfield Project and the underlying hypotheses, the writer tries to make clear main issues which are attacked and criticized by P. A. Richmond and other critics. Some of the attacks on these issues are open to refutation, others come from overcautious care or from lack of understanding towards the hypotheses. Though C. N. Mooers has defined in 1959 the exact set of relevant documents in the whole store, it was rather only in ideality. In actual fact Cleverdon's report made more concrete and popular the concepts of recall and relevance to experiment designers, who are in search of so-called objective measures for retrieval systems. According to the discussion session following the 1960 Aslib Conference, Cleverdon himself was aware of unavoidable weaknesses which lead to certain controversies. He is still continuing his tests and is trying to supplement and retouche his report.

The most impressive result of his achievement, the writer thinks, is the analysis of unsuccessful retrievals. This result may have been foreseen by an expert librarian, although this fact does not belittle the credit due him. His expression of the quantitative evaluation of the system is quite useful for the future designing of indexes as well as for use in training courses.