Using STSAssumeRoleSessionCredentialsProvider for cross account access

[Vinay Patil]

Hi,

I am trying to access dynamo streams from a different aws account but getting resource not found exception while trying to access the dynamo streams from Task Manager. I have provided the following configurations :

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_CREDENTIALS_PROVIDER,AWSConfigConstants.CredentialProvider.ASSUME_ROLE.name());

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_ARN,dynamoDbConnect.getRoleArn());

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_SESSION_NAME,dynamoDbConnect.getRoleSessionName());

In the main class I am able to get the arn of dynamoDb table using STSAssumeRoleSessionCredentialsProvider, so the assume role is working fine . Getting error only while accessing from TM. 

I assume that the credentials are not required to be passed : https://github.com/apache/flink/blob/abbd6b02d743486f3c0c1336139dd6b3edd20840/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/util/AWSUtil.java#L164

[Vinay Patil]

Hi,

Can someone pls help here , facing issues in Prod . I see the following ticket in unresolved state. 

https://issues.apache.org/jira/browse/FLINK-8417

On Thu, Oct 24, 2019 at 11:01 AM Vinay Patil <[hidden email]> wrote:

Hi,

I am trying to access dynamo streams from a different aws account but getting resource not found exception while trying to access the dynamo streams from Task Manager. I have provided the following configurations :

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_CREDENTIALS_PROVIDER,AWSConfigConstants.CredentialProvider.ASSUME_ROLE.name());

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_ARN,dynamoDbConnect.getRoleArn());

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_SESSION_NAME,dynamoDbConnect.getRoleSessionName());

In the main class I am able to get the arn of dynamoDb table using STSAssumeRoleSessionCredentialsProvider, so the assume role is working fine . Getting error only while accessing from TM. 

I assume that the credentials are not required to be passed : https://github.com/apache/flink/blob/abbd6b02d743486f3c0c1336139dd6b3edd20840/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/util/AWSUtil.java#L164

Regards,

Vinay Patil

[Fabian Hueske-2]

Hi Vinay,

Maybe Gordon (in CC) has an idea about this issue.

Best, Fabian

Am Do., 24. Okt. 2019 um 14:50 Uhr schrieb Vinay Patil <[hidden email]>:

Hi,

Can someone pls help here , facing issues in Prod . I see the following ticket in unresolved state. 

https://issues.apache.org/jira/browse/FLINK-8417

Regards,

Vinay Patil

On Thu, Oct 24, 2019 at 11:01 AM Vinay Patil <[hidden email]> wrote:

Hi,

I am trying to access dynamo streams from a different aws account but getting resource not found exception while trying to access the dynamo streams from Task Manager. I have provided the following configurations :

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_CREDENTIALS_PROVIDER,AWSConfigConstants.CredentialProvider.ASSUME_ROLE.name());

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_ARN,dynamoDbConnect.getRoleArn());

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_SESSION_NAME,dynamoDbConnect.getRoleSessionName());

In the main class I am able to get the arn of dynamoDb table using STSAssumeRoleSessionCredentialsProvider, so the assume role is working fine . Getting error only while accessing from TM. 

I assume that the credentials are not required to be passed : https://github.com/apache/flink/blob/abbd6b02d743486f3c0c1336139dd6b3edd20840/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/util/AWSUtil.java#L164

Regards,

Vinay Patil

[Vinay Patil]

Thanks Fabian,

@Gordon - Can you please help here.

On Fri, Oct 25, 2019 at 9:11 PM Fabian Hueske <[hidden email]> wrote:

Hi Vinay,

Maybe Gordon (in CC) has an idea about this issue.

Best, Fabian

Am Do., 24. Okt. 2019 um 14:50 Uhr schrieb Vinay Patil <[hidden email]>:

Hi,

Can someone pls help here , facing issues in Prod . I see the following ticket in unresolved state. 

https://issues.apache.org/jira/browse/FLINK-8417

Regards,

Vinay Patil

On Thu, Oct 24, 2019 at 11:01 AM Vinay Patil <[hidden email]> wrote:

Hi,

I am trying to access dynamo streams from a different aws account but getting resource not found exception while trying to access the dynamo streams from Task Manager. I have provided the following configurations :

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_CREDENTIALS_PROVIDER,AWSConfigConstants.CredentialProvider.ASSUME_ROLE.name());

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_ARN,dynamoDbConnect.getRoleArn());

dynamodbStreamsConsumerConfig.setProperty(ConsumerConfigConstants.AWS_ROLE_SESSION_NAME,dynamoDbConnect.getRoleSessionName());

In the main class I am able to get the arn of dynamoDb table using STSAssumeRoleSessionCredentialsProvider, so the assume role is working fine . Getting error only while accessing from TM. 

I assume that the credentials are not required to be passed : https://github.com/apache/flink/blob/abbd6b02d743486f3c0c1336139dd6b3edd20840/flink-connectors/flink-connector-kinesis/src/main/java/org/apache/flink/streaming/connectors/kinesis/util/AWSUtil.java#L164

Regards,

Vinay Patil