Making calls to external API wit Data Streams

[Krishna Kalyan]

Hello All,

I need some help making async API calls. I have tried the following code below.

class AsyncWeatherAPIRequest extends AsyncFunction[String, String] {

  override def asyncInvoke(input: String, resultFuture: ResultFuture[String]): Unit = {

    val query = url("<External API URL>")

    val response = Http.default(query OK as.String)

    resultFuture.complete(Collections.singleton(response()))

  }

}

The code below leads to a compilation issue while calling the AsyncDataStream api. 

    val resultStream: DataStream[(String, String)] =

      AsyncDataStream.unorderedWait(userData, new AsyncWeatherAPIRequest(), 1000, TimeUnit.MILLISECONDS, 1)

I would really appreciate some examples in scala to make an external API call with datastreams.

Regards,

Krishna

      

Standorte in Stuttgart und Berlin · Zoi TechCon GmbH · Quellenstr. 7 · 70376 Stuttgart · Geschäftsführer: Benjamin Hermann, Dr. Daniel Heubach. Amtsgericht Stuttgart HRB 759619, Gerichtsstand Stuttgart. Die genannten Angaben werden automatisch hinzugefügt und lassen keine Rückschlüsse auf den Rechtscharakter der E-Mail zu. This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited.

[Dominik Wosiński]

Hey, 
What is the exact issue that you are facing and the Flink version that you are using ?? 


Best Regards,
Dom.

pt., 12 paź 2018 o 16:11 Krishna Kalyan <[hidden email]> napisał(a):

Hello All,

I need some help making async API calls. I have tried the following code below.

class AsyncWeatherAPIRequest extends AsyncFunction[String, String] {

  override def asyncInvoke(input: String, resultFuture: ResultFuture[String]): Unit = {

    val query = url("<External API URL>")

    val response = Http.default(query OK as.String)

    resultFuture.complete(Collections.singleton(response()))

  }

}

The code below leads to a compilation issue while calling the AsyncDataStream api. 

    val resultStream: DataStream[(String, String)] =

      AsyncDataStream.unorderedWait(userData, new AsyncWeatherAPIRequest(), 1000, TimeUnit.MILLISECONDS, 1)

I would really appreciate some examples in scala to make an external API call with datastreams.

Regards,

Krishna

      

Standorte in Stuttgart und Berlin · Zoi TechCon GmbH · Quellenstr. 7 · 70376 Stuttgart · Geschäftsführer: Benjamin Hermann, Dr. Daniel Heubach. Amtsgericht Stuttgart HRB 759619, Gerichtsstand Stuttgart. Die genannten Angaben werden automatisch hinzugefügt und lassen keine Rückschlüsse auf den Rechtscharakter der E-Mail zu. This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited.

[Krishna Kalyan]

Thanks for the quick reply Dom,

I am using flink 1.6.1. 

Error: Type Mismatch expected AsyncFunction actual AsyncWeatherAPIRequest

On Fri, 12 Oct 2018 at 16:21, Dominik Wosiński <[hidden email]> wrote:

Hey, 
What is the exact issue that you are facing and the Flink version that you are using ?? 


Best Regards,
Dom.

pt., 12 paź 2018 o 16:11 Krishna Kalyan <[hidden email]> napisał(a):

Hello All,

I need some help making async API calls. I have tried the following code below.

class AsyncWeatherAPIRequest extends AsyncFunction[String, String] {

  override def asyncInvoke(input: String, resultFuture: ResultFuture[String]): Unit = {

    val query = url("<External API URL>")

    val response = Http.default(query OK as.String)

    resultFuture.complete(Collections.singleton(response()))

  }

}

The code below leads to a compilation issue while calling the AsyncDataStream api. 

    val resultStream: DataStream[(String, String)] =

      AsyncDataStream.unorderedWait(userData, new AsyncWeatherAPIRequest(), 1000, TimeUnit.MILLISECONDS, 1)

I would really appreciate some examples in scala to make an external API call with datastreams.

Regards,

Krishna

      

Standorte in Stuttgart und Berlin · Zoi TechCon GmbH · Quellenstr. 7 · 70376 Stuttgart · Geschäftsführer: Benjamin Hermann, Dr. Daniel Heubach. Amtsgericht Stuttgart HRB 759619, Gerichtsstand Stuttgart. Die genannten Angaben werden automatisch hinzugefügt und lassen keine Rückschlüsse auf den Rechtscharakter der E-Mail zu. This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited.

--

Standorte in Stuttgart und Berlin · Zoi TechCon GmbH · Quellenstr. 7 · 70376 Stuttgart · Geschäftsführer: Benjamin Hermann, Dr. Daniel Heubach. Amtsgericht Stuttgart HRB 759619, Gerichtsstand Stuttgart. Die genannten Angaben werden automatisch hinzugefügt und lassen keine Rückschlüsse auf den Rechtscharakter der E-Mail zu. This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited.

[Dominik Wosiński]

Hey, 

It seems that You have written Async function that takes String and returns String. But in execution you expect the result of the function to be the tuple (String, String).  That's where the mismatch occurs, the function itself is ok :) 
If you will change DataStream[(String,String)] to Datastream[String] it should work smoothly.

Best Regards,
Dom.

pt., 12 paź 2018 o 16:26 Krishna Kalyan <[hidden email]> napisał(a):

Thanks for the quick reply Dom,

I am using flink 1.6.1. 

Error: Type Mismatch expected AsyncFunction actual AsyncWeatherAPIRequest

On Fri, 12 Oct 2018 at 16:21, Dominik Wosiński <[hidden email]> wrote:

Hey, 
What is the exact issue that you are facing and the Flink version that you are using ?? 


Best Regards,
Dom.

pt., 12 paź 2018 o 16:11 Krishna Kalyan <[hidden email]> napisał(a):

Hello All,

I need some help making async API calls. I have tried the following code below.

class AsyncWeatherAPIRequest extends AsyncFunction[String, String] {

  override def asyncInvoke(input: String, resultFuture: ResultFuture[String]): Unit = {

    val query = url("<External API URL>")

    val response = Http.default(query OK as.String)

    resultFuture.complete(Collections.singleton(response()))

  }

}

The code below leads to a compilation issue while calling the AsyncDataStream api. 

    val resultStream: DataStream[(String, String)] =

      AsyncDataStream.unorderedWait(userData, new AsyncWeatherAPIRequest(), 1000, TimeUnit.MILLISECONDS, 1)

I would really appreciate some examples in scala to make an external API call with datastreams.

Regards,

Krishna

      

Standorte in Stuttgart und Berlin · Zoi TechCon GmbH · Quellenstr. 7 · 70376 Stuttgart · Geschäftsführer: Benjamin Hermann, Dr. Daniel Heubach. Amtsgericht Stuttgart HRB 759619, Gerichtsstand Stuttgart. Die genannten Angaben werden automatisch hinzugefügt und lassen keine Rückschlüsse auf den Rechtscharakter der E-Mail zu. This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited.

--

Standorte in Stuttgart und Berlin · Zoi TechCon GmbH · Quellenstr. 7 · 70376 Stuttgart · Geschäftsführer: Benjamin Hermann, Dr. Daniel Heubach. Amtsgericht Stuttgart HRB 759619, Gerichtsstand Stuttgart. Die genannten Angaben werden automatisch hinzugefügt und lassen keine Rückschlüsse auf den Rechtscharakter der E-Mail zu. This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited.