Paper 179 (Resources track)

Drammar- ontological resource and prototype annotated corpus

Author(s): Vincenzo Lombardo, Rossana Damiano, Antonio Pizzo

Full text: submitted version

Abstract: This paper reports about the release of an ontological resource on drama called Drammar
and a dataset of items annotated with a metadata schema based on Drammar.
Drammar has been realized through a collaboration of computer scientists and drama scholar
through a wiki platform, for the exchanging of definitional ideas and the encoding of axioms.
A corpus of media items have been encoded in their dramatic qualities through the creation
of instantiated ontologies (one per item) that have been shown to have a clear status in
document engineering.
The ontology has been applied to a few tasks, namely reasoning about characters’ emotions,
graphical display of characters’ intentions, encoding of action analysis for rehearsals.

Keywords: ontology resource; drama; metadata annotation

Decision: reject

Review 1 (by Serena Villata)

*** Thanks for your rebuttal. I really liked the original topic of the paper. ***
The paper presents the Drammar ontology for modeling dramatic qualities. This ontology has been designed to be a vocabulary supporting the linked interchange of drama encodings and automated reasoning tasks over them. The ontology has also been used to annotate a corpus (available upon request) described in the paper.
The paper is well written and clear. The general purpose of the Drammar ontology as well as the tasks that may make use of it are discussed. The paper is very detailed in the description of the ontology. 
On the negative side, the authors should have checked more carefully the requirements for a Resource Track submission at ESWC, as many of the elements indicated therein are unspecified in Drammar ontology.
More details below.
--- Potential impact
- Does the resource break new ground?
Yes, both the ontology and the corpus are highly innovative.
- Does the resource plug an important gap?
In the domain of Digital Humanities, it is nice to see such a kind of contributions bridging the area of Drama (media independent) and Computer Science (or more precisely Semantic Web).
- How does the resource advance the state of the art?
Yes, this is the first comprehensive ontology of the dramatic qualities.
- Has the resource been compared to other existing resources (if any) of similar scope?
Yes, there is a comparison with existing resources as well as alignment with some of them when possible.
- Is the resource of interest to the Semantic Web community?
Yes
- Is the resource of interest to society in general?
Yes
- Will the resource have an impact, especially in supporting the adoption of Semantic Web technologies?
Yes, it is quite original to have semantic web formalisms and languages applied to drama. 
- Is the resource relevant and sufficiently general, does it measure some significant aspect?
Yes, it is sufficiently general.
--- Reusability
- Is there evidence of usage by a wider community beyond the resource creators or their project? Alternatively, what is the resource’s potential for being (re)used; for example, based on the activity volume on discussion forums, mailing list, issue tracker, support portal, etc?
The ontology can definitely be reused and extended in the context of other projects about drama.
- Is the resource easy to (re)use?  For example, does it have good quality documentation? Are there tutorials availability? etc.
The resource is well documented and the paper provides all the details about its conception and its representational goals.
- Is the resource general enough to be applied in a wider set of scenarios, not just for the originally designed use?
Several scenarios come to mind in the field of Digital Humanities.
- Is there potential for extensibility to meet future requirements?
Yes, the resource can easily be extended.
- Does the resource clearly explain how others use the data and software?
This point can be improved in the paper, as it is not totally clear.
- Does the resource description clearly state what the resource can and cannot do, and the rationale for the exclusion of some functionality?
The paper clearly states what the resource can do, it would be good to include some more discussions about the limitations of the proposed ontology.
--- Design & Technical quality
- Does the design of the resource follow resource specific best practices?
The ontology follows DOLCE. 
- Did the authors perform an appropriate re-use or extension of suitable high-quality resources?  For example, in the case of ontologies, authors might extend upper ontologies and/or reuse ontology design patterns.
Yes, the authors aligned classes/relations with existing ones in other ontologies, when possible.
- Is the resource suitable to solve the task at hand?
Yes, it is.
- Does the resource provide an appropriate description (both human and machine readable), thus encouraging the adoption of FAIR principles? Is there a schema diagram? For datasets, is the description available in terms of VoID/DCAT/DublinCore?
Two diagrams are used to clarify the overall structure of the ontology and the alignment with existing resources.
-- Availability
- Is the resource (and related results) published at a persistent URI (PURL, DOI, w3id)?
No, but the URI of the ontology is available and the ontology is accessible.
- Does the resource provide a license specification? (See creativecommons.org, opensource.org for more information)
No.
- How is the resource publicly available? For example as API, Linked Open Data, Download, Open Code Repository.
Yes for the ontology. The corpus is available upon request.
- Is the resource publicly findable? Is it registered in (community) registries (e.g. Linked Open Vocabularies, BioPortal, or DataHub)? Is it registered in generic repositories such as  FigShare, Zenodo or GitHub?
No
- Is there a sustainability plan specified for the resource? Is there a plan for the maintenance of the resource?
Yes
- Does it use open standards, when applicable, or have good reason not to?
Yes, it is based on open standards.


Review 2 (by Christophe Guéret)

UPDATE: I would like to thank the author for their responses. However those won't affect my overall evaluation as it seems the paper would need some work going beyond minor tweaks prior to publication.
The paper introduces an ontology for the description of the different elements of a drama. It is a very interesting resource which is introduced in a detailed and clear way.
Pros:
* Very carefully designed ontology
* Detailed description of the thinking behind the design
Cons:
* Lack of any indication about re-usability and availability (c.f. https://2018.eswc-conferences.org/resources-track/ for a list of expected items).
* Bit of a minor issue but there is also no mention of the different languages supported. Are the concepts only available in English? Is there any extension to other languages planned?
* Previous work such as https://www.bbc.co.uk/ontologies/storyline and the work of Paul Rissen on narratives (see https://twitter.com/ACOSCast) should be mentioned and considered.
* It is very unclear if concepts from other ontologies are re-used. The text only says things such as "List is inspired by a well known ontology" or "corresponds to the class Entity". If the authors decided not to re-use concepts from established ontologies it would be good to explain why.
* Could the conflict listed on page 12 be detected through reasoning? Actually, is there any reasoning rules coming with the vocabulary?
* Why is the content of Table 1 only available on request?
Minor:
* Broken refs on page 4
* typo "corpus of consists of" on page 13


Review 3 (by Mari Carmen Suárez-Figueroa)

The paper presents an ontology for describing the drama domain and a dataset based on such a vocabulary.
The domain is very interesting and the authors describe very well the main characteristics of such a domain.
However, the paper misses the main motivation for the work as well as the gap to be covered by this resource. In addition, it would be better if the introduction includes the schema for the paper.
Since the track is about resources and the paper is about an ontology, authors should include explicitly how the ontology has been developed, the methodology and tool used, resources (ontologies, patterns) reused during the development process, and modelling decisions.
Regarding the reuse of ontologies, it is not clear enough whether the authors reused existing vocabularies. The paper mentions, for example, DOLCE and SUMO, but the document does not show whether such general ontologies were reused.
Authors should include in the paper the URL in which the ontology is available, and what is the licence for the presented resource. The paper should also include information about potential uses for the ontology and how the ontology could be reused (where is the documentation available, for example)
In addition, 
- Figure 1 misses the arrows in lines (no clear which element is the domain and which one the range of the relation). Colors in such a figure should be also explain.
- Figure 2 is not clear. It seems some lines are missing; some elements appear with box and others not; it would be very useful if authors update the figure to be more clear.
Other comments:
- It is not clear the meaning of a German word in page 3
- owl --> OWL (page 3)
- Question marks appear in page 4
- "Next section" (and similar expressions as for example 'above') should be replaced by "Section X"
- Acronyms should be explained (OCC)
- Typo: );such --> ); such (page 7)
------
AFTER REBUTTAL PHASE:
Thank you very much for your response. The idea and the domain are very interesting, so I encourage you to work on that.


Review 4 (by Antoine Isaac)

This paper presents an ontology for representing drama. There is a lot of work on identifying the relevant drama constructs from the state of the art, which could warrant publication at a conference on knowledge acquisition (like EKAW) or a discipline / Digital Humanities one (like DH). However I believe that in its current state it is quite out-of-scope for ESWC.
1. Besides the publication of an OWL ontology, there is not much ‘web’ about this work. In fact it seems to interpret “Linked Data” in a way that is not following the essence of the technology. Bring data on the web, making it ‘of the web’ with web technology. There is not much like this here. 
The method and vocabulary do not seek much to relate to other vocabularies and data. On p11, “interface between the drama domain concepts and the linguistic and common sense types of knowledge” could fit as the authors claim their work is compatible with the LD approach. But it’s not very much substantiated. There are only claims of linking to Framenet or Wordnet, but I couldn’t see much in the ontology and nothing the data (see point 2 below). 
In fact the approach does not seem to even to apply semantic web principles to its own elements. Many properties in the ontologies have datatypes as their ranges, while the allowed values look like concepts that could be represented as concepts (in SKOS or other models) which could be provided with translations and interlinked with other vocabularies. 
On a different level, few aiming at a production-level publication would still in 2017 publish an ontology with a namespace that includes a file format extension: the URL of the OWL file at http://www.cirma.unito.it/drammar/drammar.owl is used also as the ontology’s namespace. 
This could be acceptable for a work in progress. But as it seems the work is quite old, I am not convinced (see point 3 below).
2. The re-usability of the work is low. The ontology is published without any meta-information. There is no project site, no license. The ontology is also fairly complex, and one could question whether it is really doable to scale up the product of data that follows its patterns. For example http://www.cirma.unito.it/drammar/drammar.owl#Schema_type has string values like “not” or ‘done”, which is puzzling. What are these “type of schema for some construct”? There’s no online documentation to answer these questions.
In this context it would be useful for readers to have access to the data produced. But there is not much, looking at the extent of the works studied in table 1 (no statistics are reported, which by the way is also an issue). And it is not openly available. “available on request” is definitely not on par with the current state-of-the-art. For university-based work, it even sounds suspicious.
3. The authors have already published their work at http://www.semantic-web-journal.net/content/coupling-conceptual-modeling-and-rules-annotation-dramatic-media-2 (first submitted in 2013 it seems!)
As far as I can see, the journal paper presented the ontology and then the rules. This ESWC submission, 4 years after, presents only the ontology. What has happened in-between? what’s the novelty of the paper? The authors do not say. It seems actually that the rules have disappeared altogether from what is “released” in this paper.
--- AFTER REBUTTAL
I acknowledge the authors’ response, and appreciate that a lot has been spent on the questions I have asked. I also appreciate that more information is available online. I do however still have many doubts. The authors’ answers make me quite unconvinced that these doubts can be addressed, as it seems the arguments are not in good correspondence and/or that the authors have trouble to communicate clearly about them:
- “This is the first time we write a paper on the Drammar ontological resource.” is clearly in contradiction with the rest of the authors’ answer. There have been publications earlier, and I cannot see any good story that explains the differences - and why we should see a sub-set of previous work re-described extensively in a new paper.
- there is a license (CC-BY) on the wiki but it’s unclear whether that applies to the ontology itself. And the example data (work from the students) cannot be found on the wiki. Unless it is findable via the ‘site map picture’ but I cannot spend time mousing over that picture, when a big proportion of the labels are not clickable and different labels (‘ontology design’ and ‘label’) lead to the same page, which does not include more than the paper. This navigation is quite tedious.
- some pages of the wiki are still very much work in progress since over one year (https://www.di.unito.it/wikidrammar/index.php?title=Tasks&action=history), and the ontology (http://www.di.unito.it/~vincenzo/ontologies/DrammarOntology.html) has not been touched since 2011 apparently.
There may be a good story behind all this, but honestly, looking at the way the material is presented, in the paper, in the response and in the wiki, leaves me unconvinced. It would deserve some good general overhaul.


Share on

Leave a Reply

Your email address will not be published. Required fields are marked *