Re: Connecting workflows in batch

Posted by Till Rohrmann on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/Connecting-workflows-in-batch-tp11900p12009.html

Hi Mohit,

StreamExecutionEnvironment.execute() will only return giving you the JobExecutionResult after the job has reached a final stage. If that works for you to schedule the second job, then it should be ok to combine both jobs in one program and execute the second job after the first one has completed.

Cheers,
Till


On Thu, Mar 2, 2017 at 2:33 AM, Mohit Anchlia <[hidden email]> wrote:
It looks like JobExecutionResult can be used here by using the accumulators?

On Wed, Mar 1, 2017 at 8:37 AM, Aljoscha Krettek <[hidden email]> wrote:
I think right now the best option is the JobManager REST interface: https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/rest_api.html

You would have to know the ID of your job and then you can poll the status of your running jobs.

On Mon, 27 Feb 2017 at 18:15 Mohit Anchlia <[hidden email]> wrote:
What's the best way to track the progress of the job?

On Mon, Feb 27, 2017 at 7:56 AM, Aljoscha Krettek <[hidden email]> wrote:
Hi Mohit,
I'm afraid there is nothing like this in Flink yet. As you mentioned you probably have to manually track the completion of one job and then trigger execution of the next one.

Best,
Aljoscha 

On Fri, 24 Feb 2017 at 19:16 Mohit Anchlia <[hidden email]> wrote:
Is there a way to connect 2 workflows such that one triggers the other if certain condition is met? However, the workaround may be to insert a notification in a topic to trigger another workflow. The problem is that the addSink ends the flow so if we need to add a trigger after addSink there doesn't seem to be any good way of sending a notification to a queue that the batch processing is complete. Any suggestions? One option could be track the progress of a job and on a successful completion add a notification. Is there such a mechanism available?