Data Streaming

Interactive Extreme-Scale Analytics

Project Overview

The target of the INFORE project is to tackle the challenges posed by the processing and analysis of extreme scale data flows which occurs with an increasing rate today both in industrial as well as scientific applications. The INFORE project aims to combine state-of-the art data science and optimization techniques for an effective and optimized analysis of the massive data flows, while abstracting the design of the streaming analytic processes and thereby leveraging the possibility for streaming analytics to a broader user base.

Live Expo Room

During the conference you can access the expo room and meet project members for discussions, Q&A and more.

Technical Description

To tackle the complexity and huge volume of the data sources explored in the three Use Cases of INFORE and to increase the gathering of insights in streaming analytics, the INFORE project will explore the usage and combination of state-of-the-art data approximation techniques, interactive machine learning methods and complex event detection and forecasting methods. The INFORE architecture will be developed to offer an easy integration of such techniques and the possibility to combine these.

Currently under development is a new RapidMiner extension, that will allow users to quickly build streaming analytical processes and easily deploy them on either Spark Streaming or Apache Flink streaming clusters. As in between communication Apache Kafka is used to send and receive messages from streaming sources. The standalone application of Kafka is also possible for processing streamed data inside the RapidMiner platform.

Use Cases

The INFORE project will explore the extreme-scale streaming analytics scenarios on the example of real-world use cases from the life sciences, the financial and the maritime domains. The developed software architecture will be designed, developed, tweaked and tested alongside the application in this use cases, but designed with a broad extreme-scale analytics application in mind.

The Life Sciences Use Case of INFORE will advance the state-of-the-art in multi-cellular simulation for the study of the effects of drug synergies in tumor growth in a number of ways. These involve the volume and velocity of the simulated data (we will simulate the full size and complexity of cell systems found in in-vivo tumors), using novel methodologies aiming to enhance the efficiency and robustness of the simulations, including the incorporation of dynamic cell-cycle models in the simulation process, techniques for forecasting various events of interest that occur during the simulation, as well as interactive learning techniques to assist the calibration of the in-silico models.

The Financial Use Case of INFORE will use massive amounts of historical and near real-time market data to allow for forecasting, systemic risk estimation and decision support for the finance industry stakeholders. In particular, the goal is to provide a radar for international trading opportunities and risk assessment.

The aim of the Maritime Use Case of INFORE is to enhance Situational Awareness (SA) in the maritime domain, like the ability to perceive and reason with activities, events and threats at sea while projecting them into the future, through large- scale data fusion and real-time analytics. To accomplish this, the use case targets constant fusion of “global views” of areas of interest, obtained from existing maritime surveillance systems (such as the Automatic Identification System (AIS) and Sentinel-1 Synthetic Aperture Radar (SAR) satellite imagery), with “local” views, obtained from autonomous maritime vessels, acting as on-site sensing devices.

Download the talk: here

Project Information

Project Duration

January 1st, 2019 - December 31, 2021

Funding Grant Number


Project Sponsor

European Union Horizon 2020 research and innovation program

Project Partners

Academic Partners

Athena - Research and Innovation Center in Information, Communication and Knowledge Technologies

BSC - Barcelona Super Computing Center

CRG - Centre for Genomic Regulation

CMRE - Centre for Maritime Research and Experimentation

Demokritos - National Center for Scientific Research

Industry Partners

Spring Techno




RapidMiner Infore Team:

Project Page:

Twitter: @InforeProject