Collections as Flink job parameters

[Протченко Алексей]

 

Hello all.

 

I have a question about providing complex configuration to Flink job. We are working on some kind of platform for running used-defined packages which actually cantain the main business logic. All the parameters we are providing via command line and parse with ParameterTool. That’s ok until we have parameters of simple types like String, int etc. But the problem is that we need to add a Map of custom parameters for users to provide configuration variables specific for their code. 

 

Reading documentation and code of ParameterTool I do not see clear possibility to do it. Is using third-party arguments parser is the only option?

 

Best regards,

Alex

 

 

--
Алексей Протченко

[Zhu Zhu]

Hi Протченко,

Yes you cannot get a Map argument from ParameterTool directly.

ParameterTool fetches and stores data in the form of string so it's not feasible to support any types of configuration values which may be set by users.

A workaround is to convert the map to a string in head and parse it later in the main method.

Thanks,

Zhu Zhu

Протченко Алексей <[hidden email]> 于2019年11月19日周二 上午12:29写道：

 

Hello all.

 

I have a question about providing complex configuration to Flink job. We are working on some kind of platform for running used-defined packages which actually cantain the main business logic. All the parameters we are providing via command line and parse with ParameterTool. That’s ok until we have parameters of simple types like String, int etc. But the problem is that we need to add a Map of custom parameters for users to provide configuration variables specific for their code. 

 

Reading documentation and code of ParameterTool I do not see clear possibility to do it. Is using third-party arguments parser is the only option?

 

Best regards,

Alex

 

 

--
Алексей Протченко