Hi all,
I am trying to execute a batch job that gets a list of IDs and perform a loop with a number of steps during each iteration including reading from a MS SQL Server DB.
A sample pseudo-code of our implementation is the following:
List<Long> ids = …
ids.foreach(
id -> executeIteration();
)
void executeIteration {
// 1. Read from source DB
// 2. Perform a number of transformations
// 3. Write data to local file
}
When I am trying to execute the aforementioned job to a local Flink cluster, job fails retrieving the following exception:
java.lang.IllegalArgumentException: open() failed.The query processor could not start the necessary thread resources for parallel query execution.
at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.open(JDBCInputFormat.java:241)
at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:170)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704)
at java.lang.Thread.run(Unknown Source)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The query processor could not start the necessary thread resources for parallel query execution.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)
at com.microsoft.sqlserver.jdbc.SQLServerResultSet$FetchBuffer.nextRow(SQLServerResultSet.java:4869)
at com.microsoft.sqlserver.jdbc.SQLServerResultSet.fetchBufferNext(SQLServerResultSet.java:1781)
at com.microsoft.sqlserver.jdbc.SQLServerResultSet.next(SQLServerResultSet.java:1034)
at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.open(JDBCInputFormat.java:239)
... 3 more
Does anyone have any idea why this is happening?
Thanks in advance,
Konstantinos
Locked
