The "Βig Data in Production: Architectural and Security Challenges for Large Enterprise Big Data Implementations " event was a success

On Thursday, 22 November 2018, a one-day event on "Βig Data in Production: Architectural and Security Challenges for Large Enterprise Big Data Implementations" hosted by the School of ECE of the NTUA as an activity of the IEEE NTUA Student Branch and in cooperation with EY Greece.

The event took place at the Multimedia amphitheater of the NTUA Central Library.

Specifically, EY engineers with multi-year experience on Big Data presented the general principles of Lambda-type architecture along with applications on streaming and micro-batching implementations for the productive operation of Machine Learning algorithms. A cybersecurity solution for network anomaly detection was also presented.

Lambda architecture is a data-processing architecture designed to handle massive quantities of data by attempting to balance latency, throughput, and fault-tolerance using batch processing to provide comprehensive and accurate views of batch data, while simultaneously using real-time stream processing to provide views of online data. These two view outputs may be joined before presentation to the end user. The rise and importance of lambda architecture are correlated with the growth of Big Data, real-time analytics, and live marketing campaigns, which require processing pre-stored data and handling them real-time.

The event's success was confirmed by the wide participation. Students learned, among others, the fundamental security design principles of Large Systems, based on Machine Learning algorithms, and priority-designing for real-time Big Data applications.

The event time-schedule was:

14:30 - 14:40 Introduction: EY and Big Data
14:40 - 15:10 Cybersecurity anomaly detection tool based on Big Data
15:10 - 15:20 Break
15:20 - 15:50 Lambda architecture application in Big Data
15:50 - 16:30 Open discussion (networking event)

Photo credit: IEEE NTUA SB