In-Memory data grid

[nragon]

I would like to now if there is anyone who used an imdg with flink. Trying to see whether apache ignite, apache geode or hazelcast might be a good choice.

Thanks,
Nuno

[nragon]

Just to add a scenario.

My current arquitecture is the following:
I've deployed 4 ignite node in yarn and 5 task managers with 2G and 2 slots each.
As cache on ignite I have on record in key/value (string, object[])
My thoughput without ignite is 30k/sec when I add the lookup i get 3k/sec

My question is, has anyone used ignite as fast lookup or any other in memery data grid? If so, did you have any major performance impact?

Thanks

[Steve Jerman]

I had a DB look up to do and  used com.google.common.cache.Cache with a 10 sec timeout:

private static Cache<String, String> locationCache = CacheBuilder.newBuilder().maximumSize(1000).expireAfterAccess(10, TimeUnit.SECONDS).build();

Seemed to help a lot with throughput….

Steve

On Apr 3, 2017, at 10:55 AM, nragon <[hidden email]> wrote:

Just to add a scenario.

My current arquitecture is the following:
I've deployed 4 ignite node in yarn and 5 task managers with 2G and 2 slots
each.
As cache on ignite I have on record in key/value (string, object[])
My thoughput without ignite is 30k/sec when I add the lookup i get 3k/sec

My question is, has anyone used ignite as fast lookup or any other in memery
data grid? If so, did you have any major performance impact?

Thanks



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/In-Memory-data-grid-tp12494p12510.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

[nragon]

My concern and my requirements are that the cache must be shared with other jobs or any other application.
If a user or job changes the value in ignite, it must be updated in every flink job it uses that cache.

Thanks