Does Queryable State only support K/V queries not SQL?

[kant kodali]

Hi All,

1) Does Queryable State support SQL? By which I mean I can do issue a full-fledged sql query like say ("select * from table where foo='hello' group by name")  

2) Does Queryable state support offset and limit? Because if I have a million rows I don't want to get all at once.

Sorry if these are naive questions I am new to flink but I have used other streaming processors before.

Thanks!

[Fabian Hueske-2]

Hi,

Queryable state only supports key point queries, i.e., you can query a keyed state for the value of a key.
Support for SQL is not on the roadmap.

Best, Fabian

2018-02-25 14:26 GMT+01:00 kant kodali <[hidden email]>:

Hi All,

1) Does Queryable State support SQL? By which I mean I can do issue a full-fledged sql query like say ("select * from table where foo='hello' group by name")  

2) Does Queryable state support offset and limit? Because if I have a million rows I don't want to get all at once.

Sorry if these are naive questions I am new to flink but I have used other streaming processors before.

Thanks!

[kant kodali]

Hi Fabian,

Does it make sense to have it on the roadmap? that way external applications can do ad-hoc queries for the most recent stream data and avoid writing to any external database?

Thanks!

On Mon, Feb 26, 2018 at 12:14 AM, Fabian Hueske <[hidden email]> wrote:

Hi,

Queryable state only supports key point queries, i.e., you can query a keyed state for the value of a key.
Support for SQL is not on the roadmap.

Best, Fabian

2018-02-25 14:26 GMT+01:00 kant kodali <[hidden email]>:

Hi All,

1) Does Queryable State support SQL? By which I mean I can do issue a full-fledged sql query like say ("select * from table where foo='hello' group by name")  

2) Does Queryable state support offset and limit? Because if I have a million rows I don't want to get all at once.

Sorry if these are naive questions I am new to flink but I have used other streaming processors before.

Thanks!

[Renjie Liu]

Hi, kant:

Full support for sql is not easy tuo support. A simple KV storage is not enough to build a RDBMS.

On Sun, Mar 4, 2018 at 1:42 AM kant kodali <[hidden email]> wrote:

Hi Fabian,

Does it make sense to have it on the roadmap? that way external applications can do ad-hoc queries for the most recent stream data and avoid writing to any external database?

Thanks!

On Mon, Feb 26, 2018 at 12:14 AM, Fabian Hueske <[hidden email]> wrote:

Hi,

Queryable state only supports key point queries, i.e., you can query a keyed state for the value of a key.
Support for SQL is not on the roadmap.

Best, Fabian

2018-02-25 14:26 GMT+01:00 kant kodali <[hidden email]>:

Hi All,

1) Does Queryable State support SQL? By which I mean I can do issue a full-fledged sql query like say ("select * from table where foo='hello' group by name")  

2) Does Queryable state support offset and limit? Because if I have a million rows I don't want to get all at once.

Sorry if these are naive questions I am new to flink but I have used other streaming processors before.

Thanks!

--

[Hequn Cheng]

Hi kant, 

To avoid writing to external databases, some relevant issues are on the roadmap, see FLINK-6968[1]. When this issue is finished, you can store your updating tables in queryable state which is available to query(key point queries). Hope it helps you.

Best, Hequn

[1] https://issues.apache.org/jira/browse/FLINK-6968

On Sun, Mar 4, 2018 at 1:41 AM, kant kodali <[hidden email]> wrote:

Hi Fabian,

Does it make sense to have it on the roadmap? that way external applications can do ad-hoc queries for the most recent stream data and avoid writing to any external database?

Thanks!

On Mon, Feb 26, 2018 at 12:14 AM, Fabian Hueske <[hidden email]> wrote:

Hi,

Queryable state only supports key point queries, i.e., you can query a keyed state for the value of a key.
Support for SQL is not on the roadmap.

Best, Fabian

2018-02-25 14:26 GMT+01:00 kant kodali <[hidden email]>:

Hi All,

1) Does Queryable State support SQL? By which I mean I can do issue a full-fledged sql query like say ("select * from table where foo='hello' group by name")  

2) Does Queryable state support offset and limit? Because if I have a million rows I don't want to get all at once.

Sorry if these are naive questions I am new to flink but I have used other streaming processors before.

Thanks!