Paper 225 (Research track)

Navigation in Large Ontologies

Author(s): Aamna Qamar

Full text: submitted version

Abstract: The ever-growing data on the Web has given rise to ontologies reaching the size of 100,000s of concept names. The tools available to manipulate and navigate in ontologies are not competent enough to handle such large ontologies. This paper discusses a new tool prototype that allows the ontology engineers to easily perform navigation and exploration tasks in large ontologies for their sense-making. The navigation tasks, like ontology summary, focusing and zooming can be achieved through search functionality that can be run with or without reasoner. For filtering and extracting modules, the syntactic-locality modularization tools are also incorporated in the prototype. The evaluations presented in this paper demonstrate the significantly positive results obtained by experimentation with real-world large ontologies on this tool.

Keywords: Ontologies; key concepts extraction; ontology engineering; Web Ontology Language (OWL); modularization; ontology navigation

Decision: reject

Review 1 (by anonymous reviewer)

(RELEVANCE TO ESWC) -
(NOVELTY OF THE PROPOSED SOLUTION) -
(CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) -
(EVALUATION OF THE STATE-OF-THE-ART) -
(DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) -
(REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) -
(OVERALL SCORE) This rather short paper reports on an approach to enable navigation in large ontologies.
It starts out with some general remarks on ontologies, ontology languages and tools (which I would assume to be known to the community), then discusses Key Concept Extraction as a useful way of providing the user with a coarse overview of a big ontology. It then describes a tool developed by the author by recalling the creation history (maybe not so important for the reader) and describing the architecture. Thereafter the tool is compared against other ontology management tool in terms of loading time. It follows the description of some conducted case studies.
This contribution would make a nice software demo at ESWC or maybe a workshop contribution. It doesn't have sufficient substance for a conference paper as it remains unclear what new scientific insights it provides. The description of the functionality of the tool is somewhat vague and in order to clearly demonstrate its added value, some sort of user study would be necessary.


Review 2 (by anonymous reviewer)

(RELEVANCE TO ESWC) The paper deals with the problem of sense-making in large ontologies, and is, thus, highly relevant to ESWC.
(NOVELTY OF THE PROPOSED SOLUTION) I see very little novelty in the paper. Or at least, this novelty is not obvious from the text. There are tons of visualisation and summarisation tools, and the only important novelty of the proposed approach seems to be the gradual loading of an ontology (which allows it to handle large ontologies).
(CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) A more elaborate description of the tool and its functionalities should exist.
(EVALUATION OF THE STATE-OF-THE-ART) The study of existing relevant approaches (related work) is incomplete. There has been a lot of work on graph visualisation, RDF graph visualisation, summarisation etc, but only some specific approaches are (partly) mentioned in the paper in Section 3.
(DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) The paper's evaluation is far from complete. In particular, the paper is missing a concrete evaluation of the user satisfaction from the tool. Some vague non-quantified statements exist, but this is not enough, especially for a tool that tries to improve user experience.
(REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) The paper's evaluation is far from complete. In particular, the paper is missing a concrete evaluation of the user satisfaction from the tool. Some vague non-quantified statements exist, but this is not enough, especially for a tool that tries to improve user experience.
(OVERALL SCORE) The paper describes a tool for visual sense-making and navigation in ontologies. The paper is missing crucial information, lacks an appropriate evaluation and is describing the tool's features very briefly.
Table 1: not all of the approaches mentioned there are obvious to the reader. Some discussion should be devoted to explaining the ideas behind these works.
The study of existing relevant approaches (related work) is incomplete. There has been a lot of work on graph visualisation, RDF graph visualisation, summarisation etc, but only some specific approaches are (partly) mentioned in the paper in Section 3.
The abstract mentions that the search functionality of the tool can run "with or without reasoner". In the paper, I could not identify an elaboration of this statement.
I would appreciate some screenshots of the proposed tool in action.
The paper's evaluation is far from complete. In particular, the paper is missing a concrete evaluation of the user satisfaction from the tool. Some vague non-quantified statements exist, but this is not enough, especially for a tool that tries to improve user experience.
Typos:
- "also incorporated provide filtering"
Strong points
- Can work with only a partial loading of the ontology, which allows the tool to load big ontologies.
Weak points
- Unclear approach
- Insufficient evaluation
- Incomplete related work
Questions to the authors
None.


Review 3 (by Christian Mader)

(RELEVANCE TO ESWC) The paper claims to propose an "efficient navigation tool for [...] large ontologies". However, the contributions are not clear at all. No research questions have been framed and no particular navigation and visualization tasks and challenges that are addressed by the proposed prototypical tool are provided. In the paper, not even a screenshot or a link to the tool that the author claims to have developed is given, so it is impossible to assess the work in a realistic way.
(NOVELTY OF THE PROPOSED SOLUTION) The author describes a tool that uses the existing strategy of "key concept extraction" (KCE) to improve ontology loading speed and ease navigation. It builds on existing work, which has also been stated by the author. Particular improvements to the KCE algorithm have not been reported. Furthermore I cannot find any novel contributions with relation to the mentioned navigation and visualization methodologies.
(CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) The evaluation only focuses on loading time of different ontologies by various tools. The author shows that the proposed tool can load 7 out of 8 evaluation ontologies, whereas all other 4 tools fail to load 4 of these ontologies. For the remaining 3 ontologies, the tools with which the author compares the approach give a more mixed picture. One tool fails with 2 more ontologies, the others are for some ontologies faster and for some slower than the solution proposed by the author. Sometimes loading speed is even only 0 (zero) seconds which I find very questionable. I would also have expected an evaluation of how actually implemented navigation tasks supported by the author's tool compare against the existing tools from a usability point of view (not only load time), but this is not provided.
(EVALUATION OF THE STATE-OF-THE-ART) There is no dedicated Related Work or State of the Art section. The author provides a general coverage of ontologies, OWL and related tools (reasoners, OWL API) and shortly covers navigation related tools and methods. The paper fails to provide an overview of recent developments in the field ontology navigation methods and mainly targets on KCE, without making clear any alternative methods or the actual advancements the work contributes.
(DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) Based on the very poor evaluation and non-existent other documentation of the approach (screenshots, source code, live deployment) the properties of the approach cannot be assessed.
(REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) same as above
(OVERALL SCORE) The paper describes a prototypical tool that claims to support efficient navigation in large ontologies. It does so by only loading a subgraph of the ontology (key concepts) in memory and expand/collapse that subgraph depending on the user's zoom and focus. The author shows that the tool can load 7 large well-known large ontologies (e.g., snomed, nci thesaurus) while 4 other compared tools (3 protege-based and KC-Viz) run out of memory with four of these ontologies.
Strong Points:
1) Tool adopts a useful existing method (KCE) to reduce memory consumption on initial ontology loading
2) few typos
3) author got some writing practice
Weak Points: 
1) The contributions are not clearly stated, no research questions are provided. 
2) The proposed tool's actual feature set remains unclear and advancement of the state of the art is not lined out.
3) Paper structure unclear, existing work and approach are somewhat mixed up.
4) No way to reproduce or review the work (no screenshots, link, code or deployment provided)
5) Tool development methodology (sprint scopes) not of interest
6) Poor evaluation: why decision for the mentioned ontologies? Are the zero seconds in the table really true? 3 of the four tools for comparison are protege based. Bar chart provides no additional information.


Metareview by Christoph Lange

The reviewers agree that the novelty of this work is not obvious and that the research contribution is not explicit.  There is no serious evaluation of the concrete user satisfaction, which would be essential here.  The approaches (yours and related ones) need to be presented in a more comprehensive and self-contained way.


Share on

Leave a Reply

Your email address will not be published. Required fields are marked *