Model serving in Flink DataStream

[Adarsh Jain]

Hi,

I have a Flink Streaming system and I need to use couple of models trained on other systems with Flink Streaming i.e. need to do model serving.

Is PMML the best way to do it? Any inputs on Flink-JPMML performance?

Any other suggested alternatives?

Regards,

Adarsh

[Andrea Spina]

Hi Adarsh,
we developed flink-JPMML for streaming model serving based on top of the
PMML format and of course Flink: we didn't release any official benchmark
numbers yet. We didn't bump into any performance issue along the library
employment. In terms of throughput and latency it doesn't require more
effort than using Flink streaming APIs by itself.

What it can happen is high memory usage if you're deploying thousands of
(fatty) models at a time within the same pipeline, but this was a design
choice (you can see explanation here
https://www.youtube.com/watch?v=0rWvMZ6JSD8&t=17s).

AFAIK the lib is already deployed in a couple of projects. Don't hesitate to
write on Github issues if you have more questions.

https://github.com/FlinkML/flink-jpmml

Cheers,

Andrea



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/