Institutional Repository
Technical University of Crete
Technical University of Crete
EN
|
EL
| URI: | http://purl.tuc.gr/dl/dias/356CF367-7B21-41D6-BB75-63C6522742C9 | ||
| Year | 2020 | ||
| Type of Item | Diploma Work | ||
| License |
|
||
| Bibliographic Citation | Asterios Ntelmpizis, "Incremental windowed aggregations at Apache Flink", Diploma Work, School of Electrical and Computer Engineering, Technical University of Crete, Chania, Greece, 2020 https://doi.org/10.26233/heallink.tuc.87414 | ||
| Appears in Collections |
Nowadays, stream data are produced at a constant and rapid pace, more and more applications attempt to use the streams in order to receive crucial decisions. This outcome can be achieved by using algorithms and data structures that effectively process large amounts of data. These data are called Big Data and can be generated from different sources (sensors, social media). Processing and analysis of Big Data has become essential. Synopses are used in queries in Big Data because of their quick response times. Synopses summarize data set and provide approximate answers to queries.Apache Flink is one of the dominant systems for processing stream data. On data streams it is very important to calculate aggregated results and usually this is achievable using windows, since the number of streams is infinite. Results are, thus, produced after each window expires. However, Flink supports specific number of built-in implemented functions for windows.The purpose of this work is to extend the number of built-in functions that can be supported by Flink, by allowing synopses to be computed and to then provide approximate results. In addition, to maximize performance, we must ensure that building the synopses is done during the time that data are inserted into their windows. This is very important to avoid the pitfall of processing the tuples of a window after it is closed, which would require a second pass over its elements.
Copyright © DIAS 2013
