Running Apache Flink on Android

[Alexander Borgschulze-2]

I was trying to run Apache Flink within an Android App. I just want to run a minimum working example, like this:


@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    runFlinkExample();
}

private void runFlinkExample() {
    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    DataStream<Integer> stream = env.fromCollection(Arrays.asList(1, 2, 3, 4, 5));
    stream.print();
    try {
        env.execute();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

 

These are my two .gradle files:

build.gradle (Module)

 

plugins {
    id 'com.android.application'
}

android {
    compileSdkVersion 30
    buildToolsVersion "30.0.3"

    defaultConfig {
        applicationId "com.example.flink"
        minSdkVersion 26
        targetSdkVersion 30
        versionCode 1
        versionName "1.0"

        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }

    packagingOptions {
        exclude 'META-INF/DEPENDENCIES'
        exclude 'reference.conf'
    }
}

dependencies {

    implementation 'androidx.appcompat:appcompat:1.2.0'
    implementation 'com.google.android.material:material:1.3.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.0.4'
    testImplementation 'junit:junit:4.+'
    androidTestImplementation 'androidx.test.ext:junit:1.1.2'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.3.0'

    // Flink
    implementation 'org.apache.flink:flink-streaming-java_2.12:1.12.1'
    implementation 'org.apache.flink:flink-clients_2.12:1.12.1'
}

 

build.gradle (Project)

// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
    repositories {
        google()
        jcenter()
    }
    dependencies {
        classpath "com.android.tools.build:gradle:4.1.2"

        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

allprojects {
    repositories {
        google()
        jcenter()
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

 

The first problem is, that I get the following Error:

Caused by: java.lang.ClassNotFoundException: Didn't find class "org.apache.flink.streaming.api.environment.StreamExecutionEnvironment" on path: DexPathList[[zip file "/data/app/~~DbT_CZ7AhLED2xZgLBk ....

 

In cases there this error doesn't appear, I get Akka-Actor errors, because I must exclude 'reference.conf', otherwise the code wouldn't compile. However, this leads to more exceptions, e.g. missing akka-version.

So my general question is: Is it possible to run Flink within an Android-App? Or is this not possible (recommended)? Perhaps someone knows how to modfiy my gradle files (or something else) to run my example. Or perhaps someone already has successfully used Flink in Android.

[Piotr Nowojski-4]

Hi,

The question would be, why do you want to do it? I think it might be possible, but probably nobody has ever tested it. Flink is a distributed system, so running it on an Android phone doesn't make much sense.

I would suggest you first make your app/example work outside of Android. To make sure that dependencies and project setup is correct. Keep in mind that you also need to start somehow a Flink cluster. I would expect that starting a minicluster with a local execution environment (StreamExecutionEnvironment.createLocalEnvironment(...) instead of StreamExecutionEnvironment.getExecutionEnvironment()) would be the way to go.  Unless you want to run a distributed cluster across multiple Android phones, but in that case I really don't know why you would like to do it :)

Also, Android has its own JDK, which we have never tested. It might cause some problems.

Piotrek

wt., 2 mar 2021 o 16:23 Alexander Borgschulze <[hidden email]> napisał(a):

I was trying to run Apache Flink within an Android App. I just want to run a minimum working example, like this:


@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    runFlinkExample();
}

private void runFlinkExample() {
    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    DataStream<Integer> stream = env.fromCollection(Arrays.asList(1, 2, 3, 4, 5));
    stream.print();
    try {
        env.execute();
    } catch (Exception e) {
        e.printStackTrace();
    }
}

 

These are my two .gradle files:

build.gradle (Module)

 

plugins {
    id 'com.android.application'
}

android {
    compileSdkVersion 30
    buildToolsVersion "30.0.3"

    defaultConfig {
        applicationId "com.example.flink"
        minSdkVersion 26
        targetSdkVersion 30
        versionCode 1
        versionName "1.0"

        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }

    packagingOptions {
        exclude 'META-INF/DEPENDENCIES'
        exclude 'reference.conf'
    }
}

dependencies {

    implementation 'androidx.appcompat:appcompat:1.2.0'
    implementation 'com.google.android.material:material:1.3.0'
    implementation 'androidx.constraintlayout:constraintlayout:2.0.4'
    testImplementation 'junit:junit:4.+'
    androidTestImplementation 'androidx.test.ext:junit:1.1.2'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.3.0'

    // Flink
    implementation 'org.apache.flink:flink-streaming-java_2.12:1.12.1'
    implementation 'org.apache.flink:flink-clients_2.12:1.12.1'
}

 

build.gradle (Project)

// Top-level build file where you can add configuration options common to all sub-projects/modules.
buildscript {
    repositories {
        google()
        jcenter()
    }
    dependencies {
        classpath "com.android.tools.build:gradle:4.1.2"

        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

allprojects {
    repositories {
        google()
        jcenter()
    }
}

task clean(type: Delete) {
    delete rootProject.buildDir
}

 

The first problem is, that I get the following Error:

Caused by: java.lang.ClassNotFoundException: Didn't find class "org.apache.flink.streaming.api.environment.StreamExecutionEnvironment" on path: DexPathList[[zip file "/data/app/~~DbT_CZ7AhLED2xZgLBk ....

 

In cases there this error doesn't appear, I get Akka-Actor errors, because I must exclude 'reference.conf', otherwise the code wouldn't compile. However, this leads to more exceptions, e.g. missing akka-version.

So my general question is: Is it possible to run Flink within an Android-App? Or is this not possible (recommended)? Perhaps someone knows how to modfiy my gradle files (or something else) to run my example. Or perhaps someone already has successfully used Flink in Android.

[Alexander Borgschulze-2]

Hey,

Thanks for your answer :)

For my Master's thesis,I want to test and evaluate the use of CEP technologies for detecting Complex Patterns in Android sensor data (Floating Phone Data). Apache Flink offers a CEP library, so I thought it would be an interesting option. The data sources would be the sensors (Gyroscope and Accelerometer). Then I want to find patterns in this sensor data stream. This would be my usecase. 

But I am struggling with runnning a minimum working example. The execution outside of Android is not the problem. But I thought, that there might be a way to run Flink (CEP) on Android

 

>>> Piotr Nowojski <[hidden email]> 03.03.21 21.22 Uhr >>>

Hi,

The question would be, why do you want to do it? I think it might be possible, but probably nobody has ever tested it. Flink is a distributed system, so running it on an Android phone doesn't make much sense.

I would suggest you first make your app/example work outside of Android. To make sure that dependencies and project setup is correct. Keep in mind that you also need to start somehow a Flink cluster. I would expect that starting a minicluster with a local execution environment (StreamExecutionEnvironment.createLocalEnvironment(...) instead of StreamExecutionEnvironment.getExecutionEnvironment()) would be the way to go.  Unless you want to run a distributed cluster across multiple Android phones, but in that case I really don't know why you would like to do it :)

Also, Android has its own JDK, which we have never tested. It might cause some problems.

Piotrek

wt., 2 mar 2021 o 16:23 Alexander Borgschulze <[hidden email]> napisał(a):

I was trying to run Apache Flink within an Android App. I just want to run a minimum working example, like this:@Overrideprotected void onCreate(Bundle savedInstanceState) {    super.onCreate(savedInstanceState);    setContentView(R.layout.activity_main);    runFlinkExample();}private void runFlinkExample() {    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();    DataStream<Integer> stream = env.fromCollection(Arrays.asList(1, 2, 3, 4, 5));    stream.print();    try {        env.execute();    } catch (Exception e) {        e.printStackTrace();    }}

 

These are my two .gradle files:

build.gradle (Module)

 

plugins {    id 'com.android.application'}android {    compileSdkVersion 30    buildToolsVersion "30.0.3"    defaultConfig {        applicationId "com.example.flink"        minSdkVersion 26        targetSdkVersion 30        versionCode 1        versionName "1.0"        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"    }    buildTypes {        release {            minifyEnabled false            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'        }    }    compileOptions {        sourceCompatibility JavaVersion.VERSION_1_8        targetCompatibility JavaVersion.VERSION_1_8    }    packagingOptions {        exclude 'META-INF/DEPENDENCIES'        exclude 'reference.conf'    }}dependencies {    implementation 'androidx.appcompat:appcompat:1.2.0'    implementation 'com.google.android.material:material:1.3.0'    implementation 'androidx.constraintlayout:constraintlayout:2.0.4'    testImplementation 'junit:junit:4.+'    androidTestImplementation 'androidx.test.ext:junit:1.1.2'    androidTestImplementation 'androidx.test.espresso:espresso-core:3.3.0'    // Flink    implementation 'org.apache.flink:flink-streaming-java_2.12:1.12.1'    implementation 'org.apache.flink:flink-clients_2.12:1.12.1'}

 

build.gradle (Project)

// Top-level build file where you can add configuration options common to all sub-projects/modules.buildscript {    repositories {        google()        jcenter()    }    dependencies {        classpath "com.android.tools.build:gradle:4.1.2"        // NOTE: Do not place your application dependencies here; they belong        // in the individual module build.gradle files    }}allprojects {    repositories {        google()        jcenter()    }}task clean(type: Delete) {    delete rootProject.buildDir}

 

The first problem is, that I get the following Error:Caused by: java.lang.ClassNotFoundException: Didn't find class "org.apache.flink.streaming.api.environment.StreamExecutionEnvironment" on path: DexPathList[[zip file "/data/app/~~DbT_CZ7AhLED2xZgLBk ....

 

In cases there this error doesn't appear, I get Akka-Actor errors, because I must exclude 'reference.conf', otherwise the code wouldn't compile. However, this leads to more exceptions, e.g. missing akka-version.

So my general question is: Is it possible to run Flink within an Android-App? Or is this not possible (recommended)? Perhaps someone knows how to modfiy my gradle files (or something else) to run my example. Or perhaps someone already has successfully used Flink in Android.