Introduction of digital technologies into various fields of human activities has resulted in an explosive growth of information. Many tasks require not only processing of large quantities of data but also a prompt reaction to them. These tasks include the followings:
  • Fraud prevention in the financial sector.
  • Predictive maintenance in the manufacturing industry.
  • Recommendation services for Ad Tech.
In order to deal with new tasks, new approaches and new technologies capable of processing millions of messages every second are required. The Fast Data Architecture is used to meet these challenges.

The Fast Data Architecture is a set of technologies and architectural solutions:

  • Microservices ensure the interaction with the outside world: devices, external systems, and etc.
  • Akka — Reactive Programming Framework. Akka-streams, Akka-Http help to implement high-performance TCP/IP and HTTP services to receive and process messages from external devices and messages.
  • Event Broker is a platform of guaranteed delivery of messages.
  • Apache Kafka ensures a high and linear scalable bandwidth.
  • CEP (Complex Event Processing) is a platform for stream processing of messages with low latency.
  • Apache Flink is a high-performance CEP platform.
  • Stream Analytics ensures an online analysis of a data stream with the help of machine learning methods.
  • Apache Spark is a platform for mass processing of data supporting machine learning algorithms.
  • Data Store helps to store large quantities of input data and derived data.
  • Apache Cassandra is a NoSQL database with distributed data storage. It ensures a high bandwidth in case of data recording.
  • Ad-hoc Analytics is a "sandbox" for ad-hoc data analysis.
  • Apache Zeppelin is a notebook for interactive data analysis with the help of machine learning algorithms. It allows the usage of various interpreters: Spark, Cassandra and many others.