Sections, categories and keywords as interest specification tools for personalised news services

Abstract

Through an evaluation of system performance and user satisfaction for the Mercurio system, considers the general applicability and usefulness of different methods of specifying user interest for the general case of digital news services. Outlines the specific characteristics distinguishing such systems from more general information systems and discusses their effect. Proposes an evaluation blueprint for them starting from information retrieval procedures, existing work on search engine evaluation, and a close study of the working principles and the required evaluation according to the particular properties and conditions of the services under consideration. Presents and discusses actual evaluation results for system tests based both on real users and customised test cases. Conclusions cover the nature of the information handling tasks that digital news services are faced with, the relative merits of sections, categories and keywords with respect to this particular set of tasks, and the risks of careless application of recall and precision measures in systems such as these.