Flink with Hibernate ORM doesn't release memory after job is canceled

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Flink with Hibernate ORM doesn't release memory after job is canceled

Hung
Hi Flink users,

Does anyone use Flink with Hibernate ORM and have some examples?

We have a job which reads a list of topics from Postgres accessed by Hibernate ORM as parameter for Kafka consumer. The Mapper also reads information from Postgres with Hibernate ORM, meaning, both master and worker access Postgres.

I found out that this Flink job doesn't release memory when the Flink job is canceled, both master and worker.
(There is another issue that in this Flink job use lots of memory even with all possible performance tuning of Hibernate ORM, e.g.
c3p0 connection pool, ehcache size limit, batch size and so on).

The data accessed by this Flink job is quite small. I would estimate it to be no more than 50 MB, and each node has 2 GB RAM. Is there any suggest I could work on?

Best,

Sendoh
Reply | Threaded
Open this post in threaded view
|

Re: Flink with Hibernate ORM doesn't release memory after job is canceled

Aljoscha Krettek
Hi Sendoh,

Could you please go into some details about your job. Maybe show some representative code snippets or how the ORM is used exactly.

Best,
Aljoscha

> On 5. May 2017, at 18:10, Sendoh <[hidden email]> wrote:
>
> Hi Flink users,
>
> Does anyone use Flink with Hibernate ORM and have some examples?
>
> We have a job which reads a list of topics from Postgres accessed by
> Hibernate ORM as parameter for Kafka consumer. The Mapper also reads
> information from Postgres with Hibernate ORM, meaning, both master and
> worker access Postgres.
>
> I found out that this Flink job doesn't release memory when the Flink job is
> canceled, both master and worker.
> (There is another issue that in this Flink job use lots of memory even with
> all possible performance tuning of Hibernate ORM, e.g.
> c3p0 connection pool, ehcache size limit, batch size and so on).
>
> The data accessed by this Flink job is quite small. I would estimate it to
> be no more than 50 MB, and each node has 2 GB RAM. Is there any suggest I
> could work on?
>
> Best,
>
> Sendoh
>
>
>
> --
> View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-with-Hibernate-ORM-doesn-t-release-memory-after-job-is-canceled-tp13033.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.