Re: deploying flink cluster in AWS - Containerized

Posted by Chakravarthy varaga on
URL: http://deprecated-apache-flink-user-mailing-list-archive.369.s1.nabble.com/deploying-flink-cluster-in-AWS-Containerized-tp12371p12385.html

Hi,

    I request someone to help here.

Best Regards

CVP

On Thu, Mar 23, 2017 at 10:13 PM, Chakravarthy varaga <[hidden email]> wrote:

I'm looking forward to hearing some updates on this...

Any help here is highly appreciated !!

On Thu, Mar 23, 2017 at 4:20 PM, Chakravarthy varaga <[hidden email]> wrote:

Hi Team,

     We are doing a PoC to deploy Flink cluster on AWS. All runtime components will be dockerized.

    

      I have few questions in relation to discover & security:

      1. How does Job Manager discover task managers? Do they talk to over TCP ?

      2. If the runtime components TM, JM are containerized how are the IPs resolved dynamically? Basically do I have to configure the JM with the hostnames of the TMs. If so, if the TMs are on ephemeral IPs and on restart of TM how does the job manager know the TM's (IP/Host). Before I go into DNS and subnets, I'd like to understand how they disvoer & talk to each other !

       3. I went through some Flink materials on the web on security precisely on kerebros. However how do I ensure that user level authentication is applied on job management. For ex., only certain users are allowed to start/stop jobs ? This question is in relation to if flink is deployed as standalone-cluster

  

Thanks & Regards

CVP