Cannot see optimizer plan in web client
Locked
Cannot see optimizer plan in web client
 
|
Hi all, when using the web client to upload and run a jar, the optimizer plan for my program does not appear. I have checked the "Show optimizer plan" box and provided args in the Arguments field. When I continue to submit the job, it runs correctly. There is no problem when using the plan visualizer tool. Checking the webclient logs, I see this exception in the .log file: org.apache.flink.client.program.ProgramInvocationException: The program plan could not be fetched. The program silently swallowed the control flow exceptions. at org.apache.flink.client.program.PackagedProgram.getPreviewPlan(PackagedProgram.java:243) at org.apache.flink.client.web.PactJobJSONServlet.doGet(PactJobJSONServlet.java:90) at javax.servlet.http.HttpServlet.service(HttpServlet.java:734) at javax.servlet.http.HttpServlet.service(HttpServlet.java:847) at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:532) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:965) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:388) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:187) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:901) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:47) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113) at org.eclipse.jetty.server.Server.handle(Server.java:352) at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:596) at org.eclipse.jetty.server.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:1048) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:549) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:211) at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:425) at org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:489) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:436) at java.lang.Thread.run(Thread.java:745) and the usage message in the .out file, as if I did not provide the correct number of arguments (which I did). If it helps, here's the program I was trying to visualize: https://github.com/vasia/incubator-flink/blob/semimetric-preprocessing/flink-examples/flink-java-examples/src/main/java/org/apache/flink/examples/java/preprocessing/Jaccard.java Anything I'm missing here? Thanks, V. |
Re: Cannot see optimizer plan in web client
|
|
Hi! The web Frontend needs to "intercept" the execution plan when the call to "execute()" comes, which is done with a special exception. The error message says it failed, which is typically because that exception was caught. However, in your code I see nothing that should prevent that plan interception. Is the problem reproducible? Greetings, Am 02.11.2014 16:36 schrieb "Vasiliki Kalavri" <[hidden email]>:
|
Re: Cannot see optimizer plan in web client
|
|
Hi, yes it's reproducible. I get the same behavior with the following dummy program: public class TestOptimizerPlan implements ProgramDescription { @SuppressWarnings("serial") public static void main(String[] args) throws Exception { if (args.length < 2) { System.err.println("Usage: TestOptimizerPlan <input-file-path> <output-file-path>"); return; } ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment(); DataSet<Tuple2<Long, Long>> input = env.readCsvFile(args[0]) .fieldDelimiter('\t').types(Long.class, Long.class); DataSet<Tuple2<Long, Long>> result = input.map( new MapFunction<Tuple2<Long,Long>, Tuple2<Long,Long>>() { public Tuple2<Long, Long> map(Tuple2<Long, Long> value){ return new Tuple2<Long, Long>(value.f0, value.f1+1); } }); result.writeAsCsv(args[1], "\n", "\t"); env.execute(); } @Override public String getDescription() { return "TestOptimizerPlan <input-file-path> <output-file-path>"; } } Cheers, Vasia. On 3 November 2014 09:23, Stephan Ewen <[hidden email]> wrote:
|
Re: Cannot see optimizer plan in web client
|
|
Hey! Just to be sure I understand the problem correctly: Stephan On Mon, Nov 3, 2014 at 12:29 PM, Vasiliki Kalavri <[hidden email]> wrote:
|
Re: Cannot see optimizer plan in web client
|
|
I don't get any of the 2 previews :/ On 3 November 2014 12:40, Stephan Ewen <[hidden email]> wrote:
|
Re: Cannot see optimizer plan in web client
|
|
Hey! here is what I see: - I do not get the error stack trace that you describe when I submit the program and supply two arguments to the program (as required), but I do not get the plan preview either. Using the web developer tools in chrome, I see that the produces JSON is malformed. I will look into that tomorrow... Stephan On Mon, Nov 3, 2014 at 12:46 PM, Vasiliki Kalavri <[hidden email]> wrote:
|
Re: Cannot see optimizer plan in web client
|
|
Hi, thanks for looking into this! It's not urgent for me, I just wanted to make sure there was nothing wrong with my code. If you're busy with other issues, I can look into it myself next week :) Cheers, V. On 3 November 2014 22:02, Stephan Ewen <[hidden email]> wrote:
|
Re: Cannot see optimizer plan in web client
|
|
Hey, it seems you've fixed this. I've just tried and the plan appears without problem :) Thanks! V. On 3 November 2014 23:19, Vasiliki Kalavri <[hidden email]> wrote:
|
Re: Cannot see optimizer plan in web client
|
|
Yes, I fixed it as a side effect of another issue. Turns out it was a problem with non-escaped characters in the function names, which made the JSON malformed. Greetings, Stephan On Tue, Nov 11, 2014 at 5:54 PM, Vasiliki Kalavri <[hidden email]> wrote:
|
«
Return to (DEPRECATED) Apache Flink User Mailing List archive.
|
1 view|%1 views
| Free forum by Nabble | Edit this page |