Monitoring and Executing Workflows in Linked Data Environments
Author(s): Tobias Käfer, Andreas Harth
Full text: submitted version
Abstract: The W3C’s Web of Things working group is aimed at addressing the interoperability problem on the Internet of Things using Linked Data as uniform interface. While Linked Data paves the way towards combining such devices into integrated applications, traditional solutions for specifying the control flow of applications do not work seamlessly with Linked Data. We therefore tackle the problem of the specification, execution, and monitoring of applications in the context of Linked Data. We present a novel approach that combines workflows, semantic reasoning, and RESTful interaction into one integrated solution. We contribute to the state of the art by (1) defining an ontology for describing workflow models and instances, (2) providing operational semantics for the ontology that allows for the execution and monitoring of workflow instances, (3) presenting a benchmark to evaluate our solution. Moreover, we showcase how we used the ontology and the operational semantics to monitor pilots executing workflows in virtual aircraft cockpits.
Keywords: Linked Data; Workflows; Automation; Web of Things; Internet of Things; Processes; REST
Review 1 (by Hideaki Takeda)
(RELEVANCE TO ESWC) The authors assume that Read/Write Linked Data interface as a part of information system and proposed the set of the rules for workflow by Linked Data. But the former and the latter are not well combined, rather separated. (NOVELTY OF THE PROPOSED SOLUTION) The authors claimed that the proposed the set of the rules is defined in RDFS and allows reasoning. But as long as the paper, the rules are presented while no reasoning capability is shown. (CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) The description of the rules is actually a logical form, rather than Linked Data (RDF/RDFS/OWL). It should be formalized under some logic system, i.e., a subset of predicate logic with some axioms and inference rules. Otherwise the set of the rules cannot be evaluated formally. (EVALUATION OF THE STATE-OF-THE-ART) The authors analyzed the stat-of-the-art of methodologies for building large distributed systems and pointed out the combination of Restful interactions, Linked Data and workflow as a challenge to overcome the weakness of the current methodologies. The point shows the new direction for Linked Data research. (DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) he use of the real data is interesting and persuasive to show the validity of the proposal. But there need more descriptions how it is used and how it worked. (REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) The onotlogy, rules and examples are available on line. (OVERALL SCORE) The paper proposed the RDF/S based rule language for describing workflows on RESTful interactions. The authors show the ontology to express workflow models and instances and Condition-Action rules for workflows. They validated the rules as the workflow language by using Petri Nets, and tested it with a real situation. The main contribution is to propose the set of the rules for workflow in a logical way. Since it is defined in a logical form, it can be tested formally and evaluated automatically. The strong points are (1) The proposal of the set of the rules for workflow in RDF. Since it is indeed defined in a logical way, it can be tested formally and evaluated automatically. (2) The authors analyzed the stat-of-the-art of methodologies for building large distributed systems and pointed out the combination of Restful interactions, Linked Data and workflow as a challenge to overcome the weakness of the current methodologies. The point shows the new direction for Linked Data research. (3) The use of the real data is interesting and persuasive to show the validity of the proposal. The weak points are (1) Miss-match of technology: The description of the rules is actually a logical form, rather than Linked Data (RDF/RDFS/OWL). It should be formalized under some logic system, i.e., a subset of predicate logic with some axioms and inference rules. Otherwise the set of the rules cannot be evaluated formally. (2) Miss-match to the goal: Linked Data for describing the rules and Linked Data for expressing data are totally irrelevant in the paper. In section 1, the authors claim the importance of information systems built with workflow of Read/Write Linked Data, but the current solution is just another workflow language for Restful interfaces. It does not seem that the features of Linked Data are embedded in the solution. (3) Miss-Match to the goal: The reasoning issue is explicitly described in the paper while the authors emphasized it in section 1. It looks that the set of the rules has capability of reasoning but it is not described. Minor comment: in rules, there are confusion between “hasChildActivity” and “hasChildActivities”. As long as I understand, “hasChildActivity” in the second rule of V may be “hasChildActivities”, otherwise “l” is not well bound. Similarly, 6th and 8th rules in V look the same.
Review 2 (by Valeria Fionda)
(RELEVANCE TO ESWC) The paper tackles the problem of the execution of workflows of activities in the Linked Data. The paper is interesting for the Semantic Web community. (NOVELTY OF THE PROPOSED SOLUTION) The proposed solution is nice and combines RESTful services with reasoning in linked data environment. (CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) The authors describe in Section 6 the operational semantics of the workflow language proposed in the paper. Even if all the rules are reported in the section, no examples are provided to explain the rules and make the proposal easy-to-understand. I do not understand why Fig. 2 is provided as an UML class diagram and not directly as an RDF/RDFS graph. (EVALUATION OF THE STATE-OF-THE-ART) Related work are discussed in Section 2. Even if the discussion is succinct I'm pretty happy with the content of the section. (DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) The evaluation section is quite short and it actually only reports some running times. However, I appreciated that the workflow models used are available online. (REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) I appreciated that the workflow models used are available online. (OVERALL SCORE) The paper introduces a workflow language for linked data environments. After discussing the preliminaries, the operational semantics of the proposed language is introduced by enumerating the rules for the control of the workflow state, activities and transitions. Then, the paper briefly reports on some experimental evaluation. Strong Points 1- The topic is interesting 2- The data used in the evaluation are available online 3- The proposed language is discussed by detailing all the rules Weak Points 1- The paper could be organized better 2- Section 6 lacks of any example to help the reader 3- The experimental evaluation is not very informative The organization of the paper can be improved in the following directions: - A running example should be provided in the introduction and used later in Section 6 to help the reader in understanding the rules. - Section 3 has no reason to be kept as it is. It would be better to incorporate it in the experimental evaluation section. - Section 5 is too short to be a section by itself. It only counts 8 lines and a figure. Moreover, it would be much better to give Fig 2 as an RDF/RDFS graph. - Section 7.1 has no sense as a subsection of the evaluation section. There is no evaluation in it. Typos: - page 2 line 2: “an uniform” —> “a uniform” - page 2: the footnote 11 is 2 page later, please move it to page 2 - page 12 line 1: “We formally showing” —> “We formally show”
Review 3 (by Daniel Garijo)
(RELEVANCE TO ESWC) The paper is highly relevant to the conference, as it presents a way to execute workflows using restful interactions and reasoning. (NOVELTY OF THE PROPOSED SOLUTION) There has been a lot of work in workflow execution, and plenty of workflow systems exist. However I haven't seen an application such as the one proposed in this paper. That said, there is a previous workshop paper where the approach is described. The contribution of this one is on the ontology and operational semantics, hence the weak accept for novelty. (CORRECTNESS AND COMPLETENESS OF THE PROPOSED SOLUTION) It's difficult to assess the completeness of the approach, for which one has to download, reuse and understand a set of tools and assess the use cases (I haven't had the time to do all this, frankly). If an online instance had been set up, this task would have been easier. Regarding correctness, I think the proposed approach is interesting, but I wonder if it could be possible for anyone to interfere with the workflow instances. By just sending PUT requests to the different instances, one could trigger activities and mess around with the workflow execution. It would seem like the authors would need to devise some sort of security mechanism to make sure that the workflow ran safely end to end. (EVALUATION OF THE STATE-OF-THE-ART) The authors have reviewed a fair amount of methods for workflow execution services. However it remains unclear to me why these existing approaches in the state of the art are not sufficient to model the problem described. For example, workflow managements systems like Taverna and WINGS use ontologies to represent workflows. I don't think the concepts illustrated in the ontology are significantly different from what workflow management systems use, despite the change of paradigm. (DEMONSTRATION AND DISCUSSION OF THE PROPERTIES OF THE PROPOSED APPROACH) [Actually I think the score here is borderline, but there is no such option] The ontology doesn't seem to have been evaluated. It is unclear where are the requirements that were used for its design. The operational semantics seem to have been evaluated, although Table 1 is not discussed in the text at all. I also don't understand what section 7.3 is aiming to achieve. (REPRODUCIBILITY AND GENERALITY OF THE EXPERIMENTAL STUDY) Resources are online, and I have been able to access all links proposed in the paper. My only concerned is that the ontology is not documented (which makes it not very reusable) or properly published (content negotiation), which should be addressed if the paper gets accepted. (OVERALL SCORE) --------after rebuttal--------- I would like to thank the authors for their responses. I am happy to maintain my original score. --------original review-------- This paper introduces an ontology for describing workflows and workflow instances in the context of Linked Data applications. The authors introduce a set of operational rules for ensuring that different workflow execution patterns can be run successfully, and run a small evaluation using intelligent building data. The paper reads well and is relevant for ESWC, although the contribution is incremental with respect to previous work. Summarizing the points explained above, I think the approach has some advantages: novel and original approach, a very easy to extend approach for adding constraint reasoning and having all the resources accessible online (even if not properly documented). On the negative side, it seems a little difficult to design workflows in this fashion. Maybe a user study would help in this regard, or a user interface to facilitate workflow specification. In addition, any additional explanations regarding the potential vulnerabilities of the system would be appreciated. I think the paper would be interesting for the ESWC crowd, and thus I recommend to weakly accept it.
Metareview by Ruben Verborgh
We had a long discussion about this paper, and especially about its added value as a research contribution to ESWC over the existing workshop contribution. The main problem, from a research perspective, is that a clear research methodology with research questions is missing. While the text announces “an empirical evaluation of our approach”, the paper only provides timing information, which does not cover the approach but only a very specific (and likely not the most relevant) parameter of it. The authors measure the average runtime of workflows, but it does not seem that this was a problem in the first place. E.g., if the problem were that existing workflow engines are too slow or have the wrong scaling behavior, and you have an algorithm to improve on this, you could set up the corresponding research questions and validate them with such an experiment. However, in the current paper, it is unclear what exactly the evaluation is supposed to prove. In that sense, it appears too much as an engineering contribution to which an evaluation has been appended for form, but it does not answer any questions (which were not raised either). We would suggest that the authors try to find the right research setup for this contribution, by finding the right research questions and answering them (and I think these questions will not in the first place be about timing). Alternatively, this could evolve into an engineering/in use contribution, or a fully theoretical contribution.