Failed to stream on Yarn cluster

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Failed to stream on Yarn cluster

patcharee
Hi,

I am testing the streaming wiki example -
https://ci.apache.org/projects/flink/flink-docs-master/quickstart/run_example_quickstart.html

It works fine from local mode (mvn exec:java
-Dexec.mainClass=wikiedits.WikipediaAnalysis). But when I run the jar on
Yarn cluster mode, nothing happened. Any ideas? I tested the word count
example from hdfs file on Yarn cluster and it worked fine.

Best,
Patcharee


Reply | Threaded
Open this post in threaded view
|

Re: Failed to stream on Yarn cluster

Maximilian Michels
Hi Patcharee,

What do you mean by "nothing happened"? There is no output? Did you
check the logs?

Cheers,
Max

On Thu, Apr 28, 2016 at 12:10 PM, patcharee <[hidden email]> wrote:

> Hi,
>
> I am testing the streaming wiki example -
> https://ci.apache.org/projects/flink/flink-docs-master/quickstart/run_example_quickstart.html
>
> It works fine from local mode (mvn exec:java
> -Dexec.mainClass=wikiedits.WikipediaAnalysis). But when I run the jar on
> Yarn cluster mode, nothing happened. Any ideas? I tested the word count
> example from hdfs file on Yarn cluster and it worked fine.
>
> Best,
> Patcharee
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Failed to stream on Yarn cluster

patcharee
Hi again,

Actually it works well! I just realized from looking at Yarn application
log that the Flink streaming result is printed in taskmanager.out. When
I sent a question to the mailing list I looked at the screen where I
issued the command, and there was no streaming result there.

Where is the taskmanager.out and how can I change it?

Best,
Patcharee


On 28. april 2016 13:18, Maximilian Michels wrote:

> Hi Patcharee,
>
> What do you mean by "nothing happened"? There is no output? Did you
> check the logs?
>
> Cheers,
> Max
>
> On Thu, Apr 28, 2016 at 12:10 PM, patcharee <[hidden email]> wrote:
>> Hi,
>>
>> I am testing the streaming wiki example -
>> https://ci.apache.org/projects/flink/flink-docs-master/quickstart/run_example_quickstart.html
>>
>> It works fine from local mode (mvn exec:java
>> -Dexec.mainClass=wikiedits.WikipediaAnalysis). But when I run the jar on
>> Yarn cluster mode, nothing happened. Any ideas? I tested the word count
>> example from hdfs file on Yarn cluster and it worked fine.
>>
>> Best,
>> Patcharee
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Failed to stream on Yarn cluster

Maximilian Michels
You can get the Yarn logs via the Yarn web interface or from the
command-line using "./bin/yarn logs <application_id>". In the next
Flink version you will also be able to get them from the Flink web
interface.

On Thu, Apr 28, 2016 at 1:29 PM, patcharee <[hidden email]> wrote:

> Hi again,
>
> Actually it works well! I just realized from looking at Yarn application log
> that the Flink streaming result is printed in taskmanager.out. When I sent a
> question to the mailing list I looked at the screen where I issued the
> command, and there was no streaming result there.
>
> Where is the taskmanager.out and how can I change it?
>
> Best,
> Patcharee
>
>
>
> On 28. april 2016 13:18, Maximilian Michels wrote:
>>
>> Hi Patcharee,
>>
>> What do you mean by "nothing happened"? There is no output? Did you
>> check the logs?
>>
>> Cheers,
>> Max
>>
>> On Thu, Apr 28, 2016 at 12:10 PM, patcharee <[hidden email]>
>> wrote:
>>>
>>> Hi,
>>>
>>> I am testing the streaming wiki example -
>>>
>>> https://ci.apache.org/projects/flink/flink-docs-master/quickstart/run_example_quickstart.html
>>>
>>> It works fine from local mode (mvn exec:java
>>> -Dexec.mainClass=wikiedits.WikipediaAnalysis). But when I run the jar on
>>> Yarn cluster mode, nothing happened. Any ideas? I tested the word count
>>> example from hdfs file on Yarn cluster and it worked fine.
>>>
>>> Best,
>>> Patcharee
>>>
>>>
>