The paper “Towards Resilient Data Management for the Internet of Moving Things” by Elena Beatriz Ouro Paz, Eleni Tzirita Zacharatou and Volker Markl was accepted for presentation at the 19. Fachtagung für Datenbanksysteme für Business, Technologie und Web (BTW 2021) on September 20 – 24, 2021. Following the acceptance of a paper on fast CSV loading using GPUS, this is the second paper by researchers from the Database Systems and Information Management (DIMA) group at TU Berlin and the Intelligent Analytics for Massive Data (IAM) group at DFKI that will be presented at BTW 2021.
BTW is the leading database conference in the german-speaking area. For more Information on the conference, please visit https://sites.google.com/view/btw-2021-tud/.
Abstract:
Mobile devices have become ubiquitous; smartphones, tablets and wearables are essential commodities for many people. The ubiquity of mobile devices combined with their ever increasing capabilities, open new possibilities for Internet-of-Things (IoT) applications where mobile devices act as both data generators as well as processing nodes. However, deploying a stream processing system (SPS) over mobile devices is particularly challenging as mobile devices change their position within the network very frequently and are notoriously prone to transient disconnections. To deal with faults arising from disconnections and mobility, existing fault tolerance strategies in SPS are either checkpointing-based or replication-based. Checkpointing-based strategies are too heavyweight for mobile devices, as they save and broadcast state periodically, even when there are no failures. On the other hand, replication-based strategies cannot provide fault tolerance at the level of the data source, as the data source itself cannot be always replicated. Finally, existing systems exclude mobile devices from data processing upon a disconnection even when the duration of the disconnection is very short, thus failing to exploit the computing capabilities of the offline devices. This paper proposes a buffering-based reactive fault tolerance strategy to handle transient disconnections of mobile devices that both generate and process data, even in cases where the devices move through the network during the disconnection. The main components of our strategy are: (a) a circular buffer that stores the data which are generated and processed locally during a device disconnection, (b) a query-aware buffer replacement policy, and (c) a query restart process that ensures the correct forwarding of the buffered data upon re-connection, taking into account the new network topology. We integrate our fault tolerance strategy with NebulaStream, a novel stream processing system specifically designed for the IoT. We evaluate our strategy using a custom benchmark based on real data, exhibiting reduction in data loss and query latency compared to the baseline NebulaStream.
A preprint version of the paper (PDF) is available for download.