Orc Sink Table

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Orc Sink Table

jose farfan
Hi

I am checking the documentation


Register a TableSink

A registered TableSink can be used to emit the result of a Table API or SQL query to an external storage system, such as a database, key-value store, message queue, or file system (in different encodings, e.g., CSV, Apache [Parquet, Avro, ORC], …).

Flink aims to provide TableSinks for common data formats and storage systems. Please see the documentation about Table Sources and Sinks page for details about available sinks and instructions for how to implement a custom TableSink.


You can read that we can define different encodigos CSV, ORC, etc.


But in the source code, I can only find CsvTableFlink

How I can get a OrcTableFlink? Do I need to extend the TableSinkBase, of there is another place to find that implementation


BR

Jose

 


Reply | Threaded
Open this post in threaded view
|

Re: Orc Sink Table

Dawid Wysakowicz-2

Hi Jose,

Flink provides only OrcTableSource that is BatchTableSource only as of now. You can read more here[1]. As of now there is no support for orc sink file. The community is working extensively though on extending the set of supported connectors for Table API/SQL, so it might change in the future.

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/table/connect.html#orctablesource


On 13/09/18 11:50, jose farfan wrote:
Hi

I am checking the documentation


Register a TableSink

A registered TableSink can be used to emit the result of a Table API or SQL query to an external storage system, such as a database, key-value store, message queue, or file system (in different encodings, e.g., CSV, Apache [Parquet, Avro, ORC], …).

Flink aims to provide TableSinks for common data formats and storage systems. Please see the documentation about Table Sources and Sinks page for details about available sinks and instructions for how to implement a custom TableSink.


You can read that we can define different encodigos CSV, ORC, etc.


But in the source code, I can only find CsvTableFlink

How I can get a OrcTableFlink? Do I need to extend the TableSinkBase, of there is another place to find that implementation


BR

Jose

 




signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Orc Sink Table

Timo Walther
In reply to this post by jose farfan
Hi Jose,

you have to add additional Maven modules depending on the connector/format you want to use. See this page [1] for more information.

Feel free to ask further questions if the description is not enough for you.

Regards,
Timo

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/table/connect.html#further-tablesources-and-tablesinks


Am 13.09.18 um 11:50 schrieb jose farfan:
Hi

I am checking the documentation


Register a TableSink

A registered TableSink can be used to emit the result of a Table API or SQL query to an external storage system, such as a database, key-value store, message queue, or file system (in different encodings, e.g., CSV, Apache [Parquet, Avro, ORC], …).

Flink aims to provide TableSinks for common data formats and storage systems. Please see the documentation about Table Sources and Sinks page for details about available sinks and instructions for how to implement a custom TableSink.


You can read that we can define different encodigos CSV, ORC, etc.


But in the source code, I can only find CsvTableFlink

How I can get a OrcTableFlink? Do I need to extend the TableSinkBase, of there is another place to find that implementation


BR

Jose

 



Reply | Threaded
Open this post in threaded view
|

Re: Orc Sink Table

jose farfan
Hi Timo, David

Thx for your quick answer

BR
Jose

On Thu, 13 Sep 2018 at 12:41, Timo Walther <[hidden email]> wrote:
Hi Jose,

you have to add additional Maven modules depending on the connector/format you want to use. See this page [1] for more information.

Feel free to ask further questions if the description is not enough for you.

Regards,
Timo

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.6/dev/table/connect.html#further-tablesources-and-tablesinks


Am 13.09.18 um 11:50 schrieb jose farfan:
Hi

I am checking the documentation


Register a TableSink

A registered TableSink can be used to emit the result of a Table API or SQL query to an external storage system, such as a database, key-value store, message queue, or file system (in different encodings, e.g., CSV, Apache [Parquet, Avro, ORC], …).

Flink aims to provide TableSinks for common data formats and storage systems. Please see the documentation about Table Sources and Sinks page for details about available sinks and instructions for how to implement a custom TableSink.


You can read that we can define different encodigos CSV, ORC, etc.


But in the source code, I can only find CsvTableFlink

How I can get a OrcTableFlink? Do I need to extend the TableSinkBase, of there is another place to find that implementation


BR

Jose