There is no classloader in TableFactoryUtil using ConnectTableDescriptor to registerTableSource

[Joshua Fan]

Hi

As known, TableFactoryService has many methods to find a suitable service to load. Some of them use a user defined classloader, the others just uses the default classloader. 

Now I use ConnectTableDescriptor to registerTableSource in the environment, which uses TableFactoryUtil to load service, but TableFactoryUtil just use the default classloader, it is not enough in my case. Because the user may use kafka 0.8 or 0.9, the jars can not be put together in the lib directory. 

Is there a proper way to use ConnectTableDescriptor to registerTableSource at a user defined classloader?

I know SQL Client has their now implementation to avoid use TableFactoryUtil, but I think TableFactoryUtil itself should also provide a method to use user defined classloader.

Yours sincerely

Joshhua

[Hequn Cheng]

Hi Joshua,

Could you use `TableFactoryService` directly to register TableSource? The code looks like:

final TableSource<?> tableSource = TableFactoryService.find(StreamTableSourceFactory.class, streamTableDescriptor, classloader)
.createStreamTableSource(propertiesMap);
tableEnv.registerTableSource(name, tableSource);

Best, Hequn

On Tue, Jan 15, 2019 at 6:43 PM Joshua Fan <[hidden email]> wrote:

Hi

As known, TableFactoryService has many methods to find a suitable service to load. Some of them use a user defined classloader, the others just uses the default classloader. 

Now I use ConnectTableDescriptor to registerTableSource in the environment, which uses TableFactoryUtil to load service, but TableFactoryUtil just use the default classloader, it is not enough in my case. Because the user may use kafka 0.8 or 0.9, the jars can not be put together in the lib directory. 

Is there a proper way to use ConnectTableDescriptor to registerTableSource at a user defined classloader?

I know SQL Client has their now implementation to avoid use TableFactoryUtil, but I think TableFactoryUtil itself should also provide a method to use user defined classloader.

Yours sincerely

Joshhua

[Joshua Fan]

Hi Hequn

Yes, the TableFactoryService has a proper method. As I use StreamTableDescriptor to connect to Kafka, StreamTableDescriptor actually uses ConnectTableDescriptor which calls TableFactoryUtil to do service load, and TableFactoryUtil does not use a user defined classloader, so I can not use `TableFactoryService.find(StreamTableSourceFactory.class, streamTableDescriptor, classloader)` in StreamTableDescriptor directly.

One solution for me is:

1.add method to TableFactoryUtil to use user defined classloader.

2.add method to ConnectTableDescriptor accordingly.

3.add method to StreamTableDescriptor accordingly.

But I wonder if there is a current solution to register TableSource from StreamTableDescriptor using user defined classloader.

Your sincerely

Joshua

On Tue, Jan 15, 2019 at 8:26 PM Hequn Cheng <[hidden email]> wrote:

Hi Joshua,

Could you use `TableFactoryService` directly to register TableSource? The code looks like:

final TableSource<?> tableSource = TableFactoryService.find(StreamTableSourceFactory.class, streamTableDescriptor, classloader)
.createStreamTableSource(propertiesMap);
tableEnv.registerTableSource(name, tableSource);

Best, Hequn

On Tue, Jan 15, 2019 at 6:43 PM Joshua Fan <[hidden email]> wrote:

Hi

As known, TableFactoryService has many methods to find a suitable service to load. Some of them use a user defined classloader, the others just uses the default classloader. 

Now I use ConnectTableDescriptor to registerTableSource in the environment, which uses TableFactoryUtil to load service, but TableFactoryUtil just use the default classloader, it is not enough in my case. Because the user may use kafka 0.8 or 0.9, the jars can not be put together in the lib directory. 

Is there a proper way to use ConnectTableDescriptor to registerTableSource at a user defined classloader?

I know SQL Client has their now implementation to avoid use TableFactoryUtil, but I think TableFactoryUtil itself should also provide a method to use user defined classloader.

Yours sincerely

Joshhua

[Joshua Fan]

Hi Hequn

Thanks. Now I know what you mean. To use tableEnv.registerTableSource instead of using StreamTableDescriptor.registerTableSource. Yes, it is a good solution.

If the StreamTableDescriptor itself can use a user-defined classloader, it is better.

Thank you.

Yours sincerely

Joshua

On Wed, Jan 16, 2019 at 10:24 AM Joshua Fan <[hidden email]> wrote:

Hi Hequn

Yes, the TableFactoryService has a proper method. As I use StreamTableDescriptor to connect to Kafka, StreamTableDescriptor actually uses ConnectTableDescriptor which calls TableFactoryUtil to do service load, and TableFactoryUtil does not use a user defined classloader, so I can not use `TableFactoryService.find(StreamTableSourceFactory.class, streamTableDescriptor, classloader)` in StreamTableDescriptor directly.

One solution for me is:

1.add method to TableFactoryUtil to use user defined classloader.

2.add method to ConnectTableDescriptor accordingly.

3.add method to StreamTableDescriptor accordingly.

But I wonder if there is a current solution to register TableSource from StreamTableDescriptor using user defined classloader.

Your sincerely

Joshua

On Tue, Jan 15, 2019 at 8:26 PM Hequn Cheng <[hidden email]> wrote:

Hi Joshua,

Could you use `TableFactoryService` directly to register TableSource? The code looks like:

final TableSource<?> tableSource = TableFactoryService.find(StreamTableSourceFactory.class, streamTableDescriptor, classloader)
.createStreamTableSource(propertiesMap);
tableEnv.registerTableSource(name, tableSource);

Best, Hequn

On Tue, Jan 15, 2019 at 6:43 PM Joshua Fan <[hidden email]> wrote:

Hi

As known, TableFactoryService has many methods to find a suitable service to load. Some of them use a user defined classloader, the others just uses the default classloader. 

Now I use ConnectTableDescriptor to registerTableSource in the environment, which uses TableFactoryUtil to load service, but TableFactoryUtil just use the default classloader, it is not enough in my case. Because the user may use kafka 0.8 or 0.9, the jars can not be put together in the lib directory. 

Is there a proper way to use ConnectTableDescriptor to registerTableSource at a user defined classloader?

I know SQL Client has their now implementation to avoid use TableFactoryUtil, but I think TableFactoryUtil itself should also provide a method to use user defined classloader.

Yours sincerely

Joshhua