Pergamos - Library and Information Center of National and Kapodistrian University of Athens

Unit:

Department of Informatics and Telecommunications
Πληροφορική

Deposit date:

2019-11-08

Year:

2019

Author:

GRIGORIOU EVANGELOS

Supervisors info:

Δελής Αλέξης
Βαθμίδα: Καθηγητής
Τμήμα Πληροφορικής και Τηλεπικοινωνιών
Εθνικόν και Καποδιστριακόν Πανεπιστήμιον Αθηνών

Original Title:

An Efficient Decentralized Streaming Model

Languages:

English
Greek

Translated title:

An Efficient Decentralized Streaming Model

Summary:

Recently, increasingly large amounts of data are generated from a variety of sources. Streaming frameworks for Big Data applications help to store, analyze and extract useful informationfromsuchcontinuouslygenerateddata.Thereareseveralexistingstreaming frameworks, like Apache Storm, Apache Spark and Apache Flume. Inthisthesis,wepresentadecentralizedstreamprocessingmodel.ItusesaDHTprotocol toachieveamanymasters-manyworkersarchitectureandassigneachjobitsownmaster. Evengroupsarecreatedforeachjobbyutilizingthesystem’sroutingproperties,resulting in a hierarchical tree formation, consisted of agents that are participating in the network. The root of this tree acts as the master of the group and is responsible for synchronizing the group’s members. Each agent consumes live data logs, which are parsed into mini batches and stored in a memoryefficientdatastructure.Theagentsaggregatetheirlocaldataandtheresultsare rolled up the the aggregation tree.

Main subject category:

Science

Keywords:

big data, cross job coordination, compressed buffer tree, distributed systems, many masters architecture

Index:

Yes

Number of index pages:

Contains images:

Yes

Number of references:

Number of pages:

File: