Complex SQL Queries on Java Streams

[Mohammed Tabreaz]

Recently we moved from Oracle to Cassandra. In Oracle we were using advance analytical functions such as lag, lead and Macth_Recognize heavily.

I was trying to identify equivalent functionality in java, and came across Apache Flink, however I'm not sure if I should use that library in stand-alone java applications using their CollectionsEnvironment.

Anyone have ever used Apache Flink for such purpose, what is overhead of using this library in Web Services.

[Jörn Franke]

Flink is merely StreamProcessing. I would not use it in a synchronous web call. However, I would not make any complex analytic function available on a synchronous web service. I would deploy a messaging bus (rabbitmq, zeromq etc) and send the request there (if the source is a web app potentially with stomp) to receive feedback asynchronously from the function (implemented in Flunk etc) the result. In this way you can better control your Computational resources, do not block the calling application, can easily recover from connection interruptions etc.

Am 28.10.2019 um 08:11 schrieb Mohammed Tabreaz <[hidden email]>:



Recently we moved from Oracle to Cassandra. In Oracle we were using advance analytical functions such as lag, lead and Macth_Recognize heavily.

I was trying to identify equivalent functionality in java, and came across Apache Flink, however I'm not sure if I should use that library in stand-alone java applications using their CollectionsEnvironment.

Anyone have ever used Apache Flink for such purpose, what is overhead of using this library in Web Services.

[Mohammed Tabreaz]

Thanks for the feedback, clear about non blocking interfaces. 

However, can you clarify or guide me to any other libraries which can be used with java collections for complex analytics.

On Mon, Oct 28, 2019, 11:29 Jörn Franke <[hidden email]> wrote:

Flink is merely StreamProcessing. I would not use it in a synchronous web call. However, I would not make any complex analytic function available on a synchronous web service. I would deploy a messaging bus (rabbitmq, zeromq etc) and send the request there (if the source is a web app potentially with stomp) to receive feedback asynchronously from the function (implemented in Flunk etc) the result. In this way you can better control your Computational resources, do not block the calling application, can easily recover from connection interruptions etc.

Am 28.10.2019 um 08:11 schrieb Mohammed Tabreaz <[hidden email]>:



Recently we moved from Oracle to Cassandra. In Oracle we were using advance analytical functions such as lag, lead and Macth_Recognize heavily.

I was trying to identify equivalent functionality in java, and came across Apache Flink, however I'm not sure if I should use that library in stand-alone java applications using their CollectionsEnvironment.

Anyone have ever used Apache Flink for such purpose, what is overhead of using this library in Web Services.