Hi,
Would it be available to control the supersteps in Flink Spargel? For example, a master controls the basic graph algorithm having 5 phases and the master can switch between the phases. In the given example of Spargel those are send msg and update msg sequentially. Would it be possible to switch to a specific super step? |
Hi, currently, there is no such built-in master compute class, but you can easily have the equivalent functionality it as follows: - If your algorithm has a fixed pattern of superstep types, e.g. an initialization superstep, a main phase and a finalization superstep, then you can simply chain these types of Spargel iterations together, by feeding the output of one as input to the next. For example: Graph inputGraph = ... Graph graph1 = inputGraph.runVertexCentricIteration(new UpdateType1(), new MessangerType1(), maxIter1); Graph graph2 = graph1.runVertexCentricIteration(new UpdateType2(), new MessangerType2(), maxIter2); Graph graph3 = graph2.runVertexCentricIteration(new UpdateType3(), new MessangerType3(), maxIter3); and so on. - If your algorithm needs to switch between superstep types based on some control flow that is not defined beforehand, you can emulate the master functionality by using an iteration aggregator. An aggregator can accumulate values from all vertices during an iteration and then, the aggregated value can be read in the beginning of the next iteration. This way you could keep track of different phases inside your algorithm. For example, your VertexUpdateFunction could look like this: VertexUpdater extends VertexUpdateFunction { int phase; @Override public void preSuperstep() { phase = this.getIterationAggregator("phase.aggregator"); } @Override public void updateVertex(key, value, messages) { if (phase == 1) { // do type 1 superstep .... } else if (phase ==2) { // do type 2 superstep .... } else ... } } Cheers, V. On 14 February 2015 at 22:59, HungChang <[hidden email]> wrote: Hi, |
Free forum by Nabble | Edit this page |