Hi Flavio,
Below is my explanation to your question, based on anecdotal evidence:
As you may know, Flink distribution package is already scala version specific and bundles some jar artifacts.
User Flink job is supposed to be compiled against some of those jars (with maven's `provided` scope). For example, it can be Flink CEP library.
In such cases, jar names are usually preserved as is (so you would reference the same artifact dependency name in your application build and when you want to copy it from `/opt` to `/lib` folder).
Some of the jars are not supposed to be used by your application directly, but rather as "plugins" in your Flink cluster (here I mean "plugins" in a more broader sense, than plugins mechanism used by file systems introduced in Flink 1.9).
File systems, metrics reporters are good candidates for this. The reason that original jar artifacts are scala version specific is rather "incidental" (imo) - it just happens that they may depend on some core Flink libraries that still have scala code.
In practice the implementation of those libraries is not scala dependent, but to be strict (and safe) they are built separately for different scala versions (what you see in the maven central).
My understanding, that one of the goals to move scala away from core libraries (to some api level library) - this should make some of the component builds scala independent.
Removal of scala version for those jars in the distribution is probably done with the future plan in mind (so that it stays the same user experience).
Regards,
Aleksey
On Thu, Oct 10, 2019 at 10:59 AM Flavio Pompermaier <
[hidden email]> wrote:
Sorry,
I just discovered that those jars are actually in the opt folder within Flink dist..however the second point still holds: why there's a single influxdb jar inside flink's opt jar while on maven there are 2 versions (one for scala 2.11 and one for 2.12)?
Best,
Flavio
On Thu, Oct 10, 2019 at 10:49 AM Flavio Pompermaier <
[hidden email]> wrote:
Hi to all,
I was trying to configure monitoring on my cluster so I went to the metric reporters documentation.
There are 2 things that are not clear to me:
- In all reporters the documentation says to take the jars from /opt folder..obviously this is not true. Wouldn't be better to provide a link to the jar directly (on Maven Central for example)?
- If you look to influxdb dependency the documentation says to use flink-metrics-influxdb-1.9.0.jar but there's no such "unified" jar, on maven central there are two version: 1 for scala 2.11 and one for scala 2.12
Should I open 2 JIRA tickets to improve those 2 aspects (if I'm not wrong..)?
Best,
Flavio
Locked
