Distributed stream consistency checking

Shen Gao, Daniele Dell’Aglio*, Jeff Z. Pan, Abraham Bernstein

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingPublished conference contribution

2 Citations (Scopus)


Dealing with noisy data is one of the big issues in stream processing. While noise has been widely studied in settings where streams have simple schemas, e.g. time series, few solutions focused on streams characterized by complex data structures. This paper studies how to check consistency over large amounts of complex streams. Our proposed methods exploit reasoning to assess if portions of the streams are compliant to a reference conceptual model. To achieve scalability, our methods run on state-of-the-art distributed stream processing platforms, e.g. Apache Storm or Twitter Heron. Our first method computes the closure of Negative Inclusions (NIs) for DL-Lite ontologies and registers the NIs as queries. The second method compiles the ontology into a processing pipeline to evenly distribute the workload. Experiments compares the two methods and show that the second one improves the throughput up to 139% with the LUBM ontology and 330% with the NPD ontology.

Original languageEnglish
Title of host publicationWeb Engineering
Subtitle of host publicationICWE 2018
EditorsT. Mikkonen, R. Klamma, J. Hernández
Place of PublicationCham
PublisherSpringer Verlag
Number of pages17
ISBN (Electronic)9783319916620
ISBN (Print)9783319916613
Publication statusPublished - 2018
Event18th International Conference on Web Engineering, ICWE 2018 - Caceres, Spain
Duration: 5 Jun 20188 Jun 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference18th International Conference on Web Engineering, ICWE 2018


Dive into the research topics of 'Distributed stream consistency checking'. Together they form a unique fingerprint.

Cite this