Shaping tomorrow’s IT executives
BIFOLD is proud to announce that Nils Schubert, a member of the BIFOLD Graduate School and the Database Systems and Information Management group, has been accepted as one of 44 doctoral students of the recent Software Campus batch. This acceptance will support him for about two years, including BMBF funding, training, mentoring, and networking opportunities. Under the direction of his supervisor, BIFOLD Co-director Prof. Dr. Volker Markl, he will conduct research on an independent project, which was outlined unter the title “Approximative Data Stream Processing with Synopses.” Industry partner of this project is Huawei.
Project Draft “Approximate data stream processing with synopses”
In today's digital world, vast amounts of data are being generated at unprecedented speeds. Processing this data in real time is crucial for many applications, from online shopping platforms to financial markets. However, traditional data processing systems are struggling to keep up with the increasing volume and velocity of this data. To tackle this challenge, innovative strategies have been developed. Two common strategies are scale-out and load-shedding.
Scale-out involves expanding the system by adding more resources, such as servers, to handle the growing data. While effective, this approach can be very costly. On the other hand, load-shedding is a technique where only a portion of the incoming data is processed, and the rest is discarded. This reduces the burden on the system but also lowers the quality of the results, as some data is simply ignored.
A more efficient solution is to process data approximately, meaning that all incoming data is processed, but the results may contain a small, controlled margin of error. This approach allows existing resources to be used more effectively, reducing the need for additional investments in hardware. One way to achieve approximate processing is by using synopses, which are summaries of the data that allow for faster and less resource-intensive analysis. The image shows the four different classes of synopses.
The project aims to develop a system that can process high-speed data streams approximately, without requiring users to have deep knowledge of the underlying techniques. Different types of synopses are being explored to determine which ones work best for specific types of data operations. Additionally, an architecture is being designed that enables users to make approximate queries easily and efficiently, even in distributed environments where data is processed across multiple locations.
Furthermore, the integration of modern hardware, including GPUs (graphics processing units) and FPGAs (field-programmable gate arrays), into the system is being investigated to enhance performance and reduce energy consumption. The utilization of these advanced technologies is intended to develop a powerful and flexible solution for approximate data stream processing that is both user-friendly and resource-efficient.
About Software Campus
The Software Campus (SWC), funded by the German Federal Ministry of Education and Research (BMBF), is an executive development program aimed at shaping tomorrow’s senior IT executives. It is directed at outstanding doctoral students in computer science who are interested in taking on leadership responsibilities in the future. In doing so, the program combines cutting-edge scientific research with hands-on management experience in a new and innovative concept. Awardees plan and lead their own research projects in collaboration with an industry partner over an approximately two-year period. Responsible for a research budget of up to 115.000€ each, SWC participants can accelerate their research and gain leadership experience by hiring research assistants, acquiring specialized hardware, and promoting their work at international conferences.