Supporting the Interpretation of Predictive Models results with Semantic Technologies
Author(s): Iker Esnaola-Gonzalez, Jesús Bermúdez, Izaskun Fernandez, Aitor Arnaiz
Full text: submitted version
Abstract: Thermal comfort in tertiary buildings not only has a direct impact on occupants health, morale and satisfaction, but also in their working efficiency and productivity. Therefore, there is a need to establish HVAC (Heating, Ventilation and Air Conditioning) control strategies that ensure comfortable thermal situations in these environments. Since Predictive Models are used to forecasting, they are suitable to identify adequate HVAC control strategies that will ensure thermal comfort within a building in advance. This paper makes use of Semantic Technologies to interpret results obtained from Predictive Models and aids facility managers choosing the most adequate HVAC control strategies in advance. The proposed approach is applied in a real use case and compares obtained outcomes with an already existing solution. Results show that the proposed solution is more scalable, simple and flexible, easing the results interpretation and decision making tasks, as well as opening new possibilities for automatizing the HVAC control strategy selection.
Keywords: Semantic Technologies; Interpretation; Predictive Models; Thermal Comfort
Review 1 (by Vito Bellini)
(RELEVANCE TO ESWC) This work has been submitted to the Machine Learning track of ESWC but it just describe a model that uses semantic web technologies such as RDF and SPARQL to build a predictive model through SPARQL queries. It is not clear how forecasting are made using this model, the example SPARQL query that authors provide is not sufficient enough to understand the forecasting process. In my opinion this paper is not suitable for the ML track. (NOVELTY OF THE PROPOSED SOLUTION) Authors uses a Forecasting for EEPSA model and SEAS Forecasting ontology to build their predictive model. It can't see any important innovation in this. Authors mention a previous work by them about how the predictive model works (Semantic Prediction Assistant Approach applied to Energy Efficiency in Tertiary Buildings) therefore I think this work is not really providing any truly innovation regarding Machine Learning aspects. (CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) Even if this work presents weaknesses (it lacks of novelty for example), the proposed solution seems to be correct. (EVALUATION OF THE STATE-OF-THE-ART) Some related approach from the state of the art are mentioned in the related work, such as "Semantic inference-based control strategies for building HVAC systems using modelica-based physical models." but in "Experiment Results and Analysis" no comparison between any state of the art approach is reported. (DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) This approach is scalable, flexible and it can provide an energy efficiency in buildings domain. (REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) It's not really clear in this paper about the dataset, it not seems to be publicly available. No metric is reported to evaluate their experiments, just saying that the optiomal temperature is within a range is too vague to evaluate a model. (OVERALL SCORE) Authors address the problem of energy efficiency for buildings using semantic web technologies. Weak points: 1. experiment not reproducible 2. this work is not an advancement in the field of ML 3. no comparison with other state of the art approaches Strong Points: 1. scalable and flexible 2. use of semantic web technologies that can be used to build a graph starting from data arriving from sensors and it can be used somehow to make inferences 3. use CEP for alerts and notifications
Review 2 (by anonymous reviewer)
(RELEVANCE TO ESWC) The paper is out of scope of the Research track. (NOVELTY OF THE PROPOSED SOLUTION) The proposed implementation is simple and nothing novel is proposed. (CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) The solution seems complete and correct with respect to the challenge discussed. (EVALUATION OF THE STATE-OF-THE-ART) The related work section is very general and not focused on the topic. (DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) Discussion is limited. (REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) Experiments cannot be easily reproduced. (OVERALL SCORE) The paper presents an approach using semantic information and tools (i.e. SPARQL) for interpreting the results of a predictive model concerning the monitoring of a HVAC system. First of all, I would like to highlight that this contribution is out of scope with respect to the Research track. I am not able to see any research contribution with respect to the state of the art. Only an experience is discussed. For this reason, I suggest that this paper should be moved to the In-Use track. However, I still skeptical that the paper would be accepted even if it would be judged by using the In-Use track criteria. --------------------- I thank the authors for their effort in preparing the rebuttal. After reading their reply, I confirm the score given earlier.
Review 3 (by anonymous reviewer)
(RELEVANCE TO ESWC) In this work, the authors present a System that exploits Semantic Technologies to interpret results obtained from Predictive Models. The System is applied in a real use case and compared to an already existing solution. (NOVELTY OF THE PROPOSED SOLUTION) Existing Ontology EEPSA was extended to fulfill requirements in this paper. However, this is in my opinion not sufficient. Reasoning and rules can be applied to select the adequate HVAC control strategies. However, it was not worked out in detail in the experimental study and shown what the advantage and difference to the existing solution is. (CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) Evaluation is not very detailed. In the abstract it is said that the results show that the proposed solution is more scalable, simple and flexible than the already existing solution. However, no real comparision to the existing solution was made. Overview about the ontology (in a diagram) would be very good. Including a diff from version 1.2 to version 1.3. So people can see which concepts and relationships were added. A diagram is easier to read than the listings. The paper is well written. However, the evaluation and the advantages of the Semantic Web Technologies should be worked out in more detail. (EVALUATION OF THE STATE-OF-THE-ART) State-of-the-Art is given and shows an overview about related work. (DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) The System is applied in a real use case and compared to an already existing solution. However, a more detailed evaluation and comparison to the existing approach would strengthen the work (REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) The work is reproducible. The approach, ontology and the experimental study can also be applied to other buildings. (OVERALL SCORE) **Summary of the Paper In this work, the authors present a System that exploits Semantic Technologies to interpret results obtained from Predictive Models. The System is applied in a real use case and compared to an already existing solution. **Short description of the problem tackled in the paper, main contributions, and results The authors present an approach for supporting the interpretation of predictive models from building thermals and supporting the decision making which HVAC control strategies to set up in order to ensure some predefined thermal comfort conditions. **Strong Points (SPs) - Very nicely written. You can easily follow the structure - Used ontologies follow Linked Data Principles - Reused existing standards and well-known ontologies like ssn **Weak Points (WPs) - Evaluation is not very detailed. In the abstract it is said that the results show that the proposed solution is more scalable, simple and flexible than the already existing solution. However, no real comparision to the existing solution was made. - Overview about the ontology (in a diagram) would be very good. Including a diff from version 1.2 to version 1.3. So people can see which concepts and relationships were added. A diagram is easier to read than the listings. - Work out the advantages of semantics in more detail. Compare the proposed solution to the existing one. **Questions to the Authors (QAs) ** Enumerate the questions to be answered by the authors during the rebuttal process** - Why does this paper fit into Research Track or rather: Where is the research part in this paper? In my opinion this Paper fits more to the In-Use & Industrial Track, since it tackles the application of Semantic Web Technologies in a real use case scenario. **Typos / Comments - I would suggest a Reference to the Directive 89/654/EEC. Same for the UK HSE
Review 4 (by Mathieu D’Aquin)
(RELEVANCE TO ESWC) The paper is related to ESWC in the sense that it uses semantic web technologies in a specific scenario, but it does not, as much as I can tell, show a research contribution to the semantic web. In that sense, it would fit better in the in-use track. In relation to this, the title of the paper is too misleading. It gives the impression that it will contribute in a generic way to the interpretation of predictive models (which is a term that I think is also used wrongly in the paper), while the paper is really about a specific use case, making no conclusion on the general case. (NOVELTY OF THE PROPOSED SOLUTION) As show in the related work section, there has been many approaches to support the interpretation of various processes from KDD and ML. While those are still mostly preliminary, there is no dicussion on the value of the proposed approach compared to those. The process presented appears to me as being related to general decision process, using ontological annotations and rule to classify aspects of the prediction into a more interpretable structure. (CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) The approach presented in relation to the use case appears valid. (EVALUATION OF THE STATE-OF-THE-ART) As already mentioned above, there is no comparison with other approaches with similar objectives. This is understandable as the paper is really focusing on the use case, but at least a qualitative view of the benefits of this particular approach to the considered use case compared to other possible alternatives would have made the paper stronger. (DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) Similarly to the previous comment, only some general properties of the process described are being discussed (scalability), but there is not indication of the general benefits of the approach compated to other options. Even if it was an in-use paper, I would have expected some analysis of the specific contributions of semantic web technologies in the considered scenario. (REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) It seems that the approach could be re-applied in other locations, providing that the base technologies and the model were available, which I'm not sure is the case. (OVERALL SCORE) The paper presents the use of semantic web technologies (ontologies and sparql construct queries) for the purpose of interpreting the results of prediction regarding the indoor conditions in buildings. It is an interesting scenario and the interpretation of predictive model is a timely topic (even if this is arguably not what this paper is presenting). The paper however does not seem to contribute to semantic web research, and appears to be the description of an application mostly. There is also no comparison with other approaches, with similar purposes or processes. I find the context setting also very confusing, as it is not clear what is meant by predictive model here (apparently not what is meant by predictive model in machine learning), how this actually relates to KDD (I don't see a data mining method being used prior to interpretation) and in which way what is done can be called interpretation.
Metareview by Achim Rettinge
The paper describes the use of semantic technology to support the selection of simulation results for a HVAC control strategy. While all the reviewer acknowledge the use of semantic technology and see a clear connection with the conferences topics, the work lacks a clear and strong research contribution which has also relationship to machine learning. In the rebuttal, the author agree with the main findings of the reviewer. Unfortunately, we are not able to move the work into the in-use-track at this late stage of the reviewing process. Further, the paper is a good application but also not that strong, so we can only recommend to reject it.