Beam quickstart

[Gyula Fóra]

Hey,

Is there somewhere an end to end guide how to run a simple beam-on-flink application (preferrably using Gradle)? I want to run it using the standard per-job yarn cluster setup but I cant seem to get it to work.

I always end up having strange NoSuchMethod errors from protobuf and have spent hours trying to find a setup that works but I still fail.

Thank you!

Gyula

[Ufuk Celebi]

Hey Gyula,

including Aljoscha (cc) here who is a committer at the Beam project.
Did you also ask on the Beam mailing list?

– Ufuk


On Wed, Apr 25, 2018 at 3:32 PM, Gyula Fóra <[hidden email]> wrote:

> Hey,
> Is there somewhere an end to end guide how to run a simple beam-on-flink
> application (preferrably using Gradle)? I want to run it using the standard
> per-job yarn cluster setup but I cant seem to get it to work.
>
> I always end up having strange NoSuchMethod errors from protobuf and have
> spent hours trying to find a setup that works but I still fail.
>
> Thank you!
> Gyula

[Jörn Franke]

Tried with a fat jar to see if it works in general ?

> On 25. Apr 2018, at 15:32, Gyula Fóra <[hidden email]> wrote:
>
> Hey,
> Is there somewhere an end to end guide how to run a simple beam-on-flink application (preferrably using Gradle)? I want to run it using the standard per-job yarn cluster setup but I cant seem to get it to work.
>
> I always end up having strange NoSuchMethod errors from protobuf and have spent hours trying to find a setup that works but I still fail.
>
> Thank you!
> Gyula

[Gyula Fóra]

For some reason it only seems to work if I put my jars in the Flink lib folder. I am not sure why though...

Gyula

Jörn Franke <[hidden email]> ezt írta (időpont: 2018. ápr. 25., Sze, 16:50):

Tried with a fat jar to see if it works in general ?

> On 25. Apr 2018, at 15:32, Gyula Fóra <[hidden email]> wrote:
>
> Hey,
> Is there somewhere an end to end guide how to run a simple beam-on-flink application (preferrably using Gradle)? I want to run it using the standard per-job yarn cluster setup but I cant seem to get it to work.
>
> I always end up having strange NoSuchMethod errors from protobuf and have spent hours trying to find a setup that works but I still fail.
>
> Thank you!
> Gyula