Evaluating Semantic Web Query Answering Systems
University:University of Oxford
Room :Teleconference Room (2nd floor NOC)
Time:4:00pm (coffee: 3:30)
Giorgos Stoilos, Information Systems Group, Dept. of Computer Science, University of Oxford
Semantic Web technologies, like the OWL 2 and RDF(S) ontology languages, have gradually started to be used in many research as well as industrial strength applications, a recent example being BBC's World Cup Semantic Website. The pressing need for scalable reasoning has motivated a large number of different Semantic Web reasoners. Many such systems are often incomplete--that is, for some combinations of inputs they fail to derive all answers to a user query, with the goal of gaining in performance; others are based on fragments of OWL 2 (OWL 2 profiles), like OWL 2 QL, for which polynomial reasoning algorithms are known.
Due to the vast number and heterogeneity of Semantic Web systems users and even experts are usually incapable of choosing the right system for their application. To help compare systems and assess which one is the best, a set of methods to evaluate them have been developed. In this talk I will overview recent research results in evaluating Semantic Web reasoning systems. The focus will be on completeness evaluation of incomplete systems, i.e., measuring the level of completeness of systems not intended to be complete.