Processing events through web socket

[Abdul Salam Shaikh]

I am trying to read data over a websocket from a source(For example wss://brisbane.unipulse.com.au:443/OutboundInterfaces/outbound/statements). But at the moment the implementation of env.socketTextStream only supports plain TCP and has no support for TLS or higher level protocols​. Is there any other wrapper/third party API implementation which I can use on top of flink to act as a streaming service before transmitting it to flink ? 

--

[Fabian Hueske-2]

Hi,

the TextSocketSink is rather meant for demo purposes than to be used in an actual applications.

I am not aware of any other built-in source that would provide what you are looking for.

You can implement a custom SourceFunction that does what you need.

Best, Fabian

2016-10-05 9:48 GMT+02:00 Abdul Salam Shaikh <[hidden email]>:

I am trying to read data over a websocket from a source(For example wss://brisbane.unipulse.com.au:443/OutboundInterfaces/outbound/statements). But at the moment the implementation of env.socketTextStream only supports plain TCP and has no support for TLS or higher level protocols​. Is there any other wrapper/third party API implementation which I can use on top of flink to act as a streaming service before transmitting it to flink ? 

--

[Maximilian Michels]

Like Fabian said, there is no built-in solution. If you want to use
encryption over a socket, you will have to implement your own socket
source using an encryption library.

On Wed, Oct 5, 2016 at 10:03 AM, Fabian Hueske <[hidden email]> wrote:

> Hi,
>
> the TextSocketSink is rather meant for demo purposes than to be used in an
> actual applications.
> I am not aware of any other built-in source that would provide what you are
> looking for.
> You can implement a custom SourceFunction that does what you need.
>
> Best, Fabian
>
> 2016-10-05 9:48 GMT+02:00 Abdul Salam Shaikh <[hidden email]>:
>>
>> I am trying to read data over a websocket from a source(For example
>> wss://brisbane.unipulse.com.au:443/OutboundInterfaces/outbound/statements).
>> But at the moment the implementation of env.socketTextStream only supports
>> plain TCP and has no support for TLS or higher level protocols. Is there any
>> other wrapper/third party API implementation which I can use on top of flink
>> to act as a streaming service before transmitting it to flink ?
>>
>> --
>> Thank
>> s.
>>
>

[Maximilian Michels]

You don't have to wrap the existing socket source. You will have to
create your own source function which directly reads from the web
socket and forwards the results to the next Flink operator. Then you
will call env.addSource(new YourWebSocketSource());

To get started, you might want to read the code of the existing
SocketTextStreamFunction as a basis for your implementation.

Cheers,
Max

On Mon, Oct 10, 2016 at 1:58 PM, Abdul Salam Shaikh
<[hidden email]> wrote:

> Thanks for responding.
> I want to read json streams from web sockets but at the moment socketStream
> method of Flink only supports plain TCP sockets.
>
> What could be the probable solution for this ?
> All I can think of so far is a wrapper which listens to from the web sockets
> and sends it to the TCP sockets.
>
> Please let me know if there is anything more efficient than that.
>
> Thanks.