Flink Streaming API

[Siva]

Hello Everyone,

I m new to Flink, wanted to try streaming API using flink-kafka connector in scala.

But there are several versions of it. Please could some one help on below questions

what are the differences between flink-streaming-core and flink-sreaming-scala[java]?

Latest version of flink-streaming-scala of 0.10.2 version, does it mean that streaming development in scala can only use Flink 0.10.2 not the latest one 1.0.3?

what is difference between flink-connector-kafka-base and flink-connector-kafka-0.x.x? Is flink-connector-kafka-0.9-2_11 compatible with Kafka 0.10.0?

Thanks in advance.

Thanks

Siva

[Till Rohrmann]

Hi Siva,

in version 1.0.0 we’ve added the Scala binary version suffix to all Flink dependencies which depend on Scala. Thus, you should look for flink-streaming-scala_2.10 and flink-streaming-java_2.10. For these artifacts you should be able to find a version 1.0.3 on maven central, for example.

The easiest way to setup a project is to use the quickstarts [1]. It will add the correct dependencies. But usually it should be enough to include the flink-client_2.10 and the respective API dependency to your project (e.g. flink-streaming-scala_2.10).

I think that you should be able to use the Flink Kafka connector 0.9 with Kafka 0.10.0 as long as Kafka 0.10.0 still supports the consumer API introduced with Kafka 0.9. But I might be wrong here. So best if you tried it out.

[1] https://ci.apache.org/projects/flink/flink-docs-master/quickstart/java_api_quickstart.html

Cheers,
Till

​

On Mon, Aug 1, 2016 at 3:48 PM, Siva <[hidden email]> wrote:

Hello Everyone,

I m new to Flink, wanted to try streaming API using flink-kafka connector in scala.

But there are several versions of it. Please could some one help on below questions

what are the differences between flink-streaming-core and flink-sreaming-scala[java]?

Latest version of flink-streaming-scala of 0.10.2 version, does it mean that streaming development in scala can only use Flink 0.10.2 not the latest one 1.0.3?

what is difference between flink-connector-kafka-base and flink-connector-kafka-0.x.x? Is flink-connector-kafka-0.9-2_11 compatible with Kafka 0.10.0?

Thanks in advance.

Thanks

Siva

[Siva]

Hi Till,

Thanks for your response. I m able to use flink-connector-kafka-0.9-2_11 with Kafka v10 to produce and consume messages.

Thanks,

On Mon, Aug 1, 2016 at 6:41 AM, Till Rohrmann <[hidden email]> wrote:

Hi Siva,

in version 1.0.0 we’ve added the Scala binary version suffix to all Flink dependencies which depend on Scala. Thus, you should look for flink-streaming-scala_2.10 and flink-streaming-java_2.10. For these artifacts you should be able to find a version 1.0.3 on maven central, for example.

The easiest way to setup a project is to use the quickstarts [1]. It will add the correct dependencies. But usually it should be enough to include the flink-client_2.10 and the respective API dependency to your project (e.g. flink-streaming-scala_2.10).

I think that you should be able to use the Flink Kafka connector 0.9 with Kafka 0.10.0 as long as Kafka 0.10.0 still supports the consumer API introduced with Kafka 0.9. But I might be wrong here. So best if you tried it out.

[1] https://ci.apache.org/projects/flink/flink-docs-master/quickstart/java_api_quickstart.html

Cheers,
Till

​

On Mon, Aug 1, 2016 at 3:48 PM, Siva <[hidden email]> wrote:

Hello Everyone,

I m new to Flink, wanted to try streaming API using flink-kafka connector in scala.

But there are several versions of it. Please could some one help on below questions

what are the differences between flink-streaming-core and flink-sreaming-scala[java]?

Latest version of flink-streaming-scala of 0.10.2 version, does it mean that streaming development in scala can only use Flink 0.10.2 not the latest one 1.0.3?

what is difference between flink-connector-kafka-base and flink-connector-kafka-0.x.x? Is flink-connector-kafka-0.9-2_11 compatible with Kafka 0.10.0?

Thanks in advance.

Thanks

Siva