Paper 139 (In-Use track)

Knowledge-Driven Data Integration and Reasoning for Sustainable Subsurface Inter-Asset Management

Author(s): Lijun Wei, Heshan Du, Quratul-Ain Mahesar, Barry Clarke, Derek R. Magee, Vania Dimitrov, David Gunn, David Enwisle, Helen Reeves, Anthony G. Cohn

Full text: submitted version

Abstract: Sustainable subsurface infrastructure management requires an integrated approach that considers the various city infrastructure assets (e.g. roads, ground, buried utilities), human activities and the environment as a holistic system. To this end, an integrated knowledge-driven decision support system is presented in this paper based on a suite of knowledge models, which describe the main properties/processes of different city infrastructure assets (City Infrastructure Asset Ontologies), the categories/ properties of events that need some decisions to be made in assets management (Trigger Ontology), and the geophysical techniques suitable for measuring different asset properties (Investigation Ontology). Various datasets are sourced, preprocessed, integrated in a spatial database and mapped to these ontologies; and an inference engine is employed to exploit the knowledge and data for automated inference of potential consequences. A prototype of this system has been developed and users’
feedback collected from two workshops showed that the system is widely considered as a potentially useful tool for practitioners.

Keywords: knowledge-driven; data integration; reasoning; decision support system; subsurface infrastructure management; inter-asset management

Decision: reject

Review 1 (by Paul Groth)

Comments after rebuttal: I thank the authors for the additional context around the evaluation and status. I think this would make for a good demo paper.
Knowledge-Driven Data Integration and Reasoning for Sustainable Subsurface Inter-Asset Management
This paper describes a prototype decision support system for integrating information about infrastructure (e.g. roads, utilities) and the context around it (e.g. traffic) to allow for the assessment of the impact of an event. For example, if there is excessive traffic on a road it may have an impact on underground waste water pipes. The prototype integrated data from five different data owners using comprehensive ontology using the geospatial domain as reference. It also provides a rules engine for creating suggestions about impacts.  The domain is very interesting from an in-use perspective.
The prototype is impressive. I thought the video was very good. In particular, I liked the justifications and the ability for users to see how modifying assumptions leads to different consequences for various pieces of infrastructure. In my opinion, this definitely should be a demo paper at the conference.
However, I have a three concerns with the paper:
1) Evaluation
The evaluation criteria/setup were not clearly state upfront in the paper. I think the use of surveys, as done in the paper, is methodologically ok but it's important to be clear what the goal of the assessment was. For the pre-prototype survey, I looked at the referred to questionnaire [1] of the relevant practitioners but it didn't provide any insight into the demographic composition of who was consulted. Likewise, the description of the evaluation of the prototype failed to provide information about the numbers of breakdown of the assessors of the paper. It's important to provide background information about evaluators and I encourage the authors to do this. 
2) Status in-practice
While a prototype can definitely be a contribution to an in-use paper, I would have expected a more rigorous evaluation or at least a point-by-point mapping of the requirements obtained from domain experts into the prototype itself. 
3) Broader contextualization
It would be nice to contextualize this article to other geographies outside the UK as well. Just mention examples or cases stemming from other locations. In particular, I was surprised not to see a mention of several projects on the use of semantic web standards for managing the operation and delivery of infrastructure (e.g. Interlink: https://roadotl.geosolutions.nl and http://www.coinsweb.nl/index_uk.html. While these come from a more building information management view, they touch directly on infrastructure asset management. 
Overall, while I really like the prototype, I think the paper still needs work to effectively demonstrate it's true impact in-use.  
[1] http://assessingtheunderworld.org/wp-content/uploads/2016/11/Annual-Dissemination-Event-Condition-Assessment-Questionnaire-Results.pdf
Minor comments:
* This statement "The semantic approach can help facilitate the domain knowledge sharing among stakeholders in different sectors." should have some support (i.e. a reference or example)
* It would be nice if the ontologies were made available following linked data principles and registered with something like Linked Open Vocabularies (http://lov.okfn.org/)


Review 2 (by anonymous reviewer)

This paper introduces Assessing The Underworld Decision Support System (ATUDSS), which combines several different datasets and provides a forward-chaining reasoning function from an event trigger.
Paper describes the ATUDSS on the whole and would give the insight to readers related, such as to disaster management systems.
However, descriptions of each part of the ATUDSS are relatively shallow; thus, what the technical challenges were to implement each part and how the authors overcame those issues are unclear.
Even in an in-use paper, it should have descriptions of technical problems and solutions, especially from the practical point of view.
Since it was expected by the user feedback, the forward-chaining reasoning functions should be evaluated in this "fully integrated" datasets, in addition to uncertainty and missing facts handling.
However, these may have been described in [15] and should be extended, if any.
Moreover, the reason why about half of users did not interested in this system should be considered.
Since this is a semantic conference, it is unfortunate that the description of the used ontology is poor.
Comment:
- How many people attended two workshops?
--
I acknowledge the authors' comments, but unfortunately do not change my score, since many parts should be improved.


Review 3 (by Rinke Hoekstra)

I thank the authors for their response to my review but see no reason to change my overall evaluation. This is not because the work, and project as a whole isn't interesting or impressive (it is!), but rather because the paper fails to meet the requirements for a scientific publication.
===
This paper presents a platform for surface asset management (e.g. road maintenance) by integrating data from various sources. The paper describes the system itself in some detail, but does not really discuss issues of interest such as:
* data integration: was this just done based on geo-coordinates or did you do something else?
* scalability of the approach: how many rules did you define, and how did this impact system performance?
* correctness of the approach: you mention evaluation against users, but fail to specify how many users, and of what background. Did you test the system against historical scenarios? The answers you report ("potentially useful tool") seem to indicate not much more than that your interviewees were being polite.
* reliability of the approach: you mainly use open data, including open streetmap, even for services that you call "sensitive" (schools, hospitals). Why not use data from more primary sources (e.g. local governments)? 
* comparison of techniques: the paper describes your technology choices, but not the alternatives you could have chosen but discarded. The related works section is limited to systems that try to achieve a similar solution, but does not discuss technology. For instance, you choose JESS rules, but why? What were the alternatives, and how did they compare? (NB this is a SW conference you are submitting to)
As to the last point: you mention that various systems already exist within the UK. Why did you choose to build your own system, rather than integrate across the ones that only focus on a single aspect?
To conclude, I think this is an interesting project, but the paper falls short in discussing related work, the technology choices, evaluation and a comparison with existing work.


Review 4 (by Michele Pasin)

Strong accept. 
I really liked reading this paper. Use case is clear and well documented. System description clear too.
A few points that the authors should address in order to improve the paper quality: 
* Can you provide more info about performance and scale: how long does it take for generating the inferences? how many statements can the system deal with in order to have acceptable response times? does it scale linearly? 
* Can you provide more info on ontology development an maintenance: how were the models initially developed and in order to productionize such a system how many KE would be needed? Is it sustainable long term?
* Re. the automatic collection of data following trigger reporting: this seems a really cool approach, but how is it resilient to (one of the) systems having downtime or falling over? 
* Nice to hear that 56% of the respondents providing positive feedback. However I'd be more interested in knowing why the remaining 44% were not happy with it ie what's missing? did they think the outputs of the system were too trivial? or maybe not trustworthy enough? It'd be nice if you could add more details about that.


Review 5 (by Anna Tordai)

This is a metareview for the paper that summarizes the opinions of the individual reviewers.
The reviewers appreciated the clear description of the system and use cases, and some acknowledged that it is an interesting application. They pointed out some weaknesses, including: details are missing regarding design choices, especially in comparison to existing solutions. In addition, with respect to the evaluation, the paper would benefit from a more objective setup and more detailed analysis into the responses of the users.
Laura Hollink & Anna Tordai


Share on

Leave a Reply

Your email address will not be published. Required fields are marked *