Tuesday, June 23, 2020

Authentication & Authorisation with AWS Cognito - Part 1 Authentication

We have setup our Kubernetes cluster; we have setup the dashboard on how to monitor them; we have implemented a few services;
What next? 

We need to build an application that will use these services - Let us take the first step to build an application - Authentication and Authorisation - The 2A's

Most times, while building an Application we concentrate a lot on the features of the application that the 2A's are an after thought.

I have currently started out with a problem statement and working on an application; I have come to a point where I can build on the function and build a product out of it

However, I have taken a pause! I am focussing on the 2A's as they are very important; they constitute two of the most important things as we start building any product. If the security around the product is not good; in this day and age, it just will not fly

In olden days we used to have all these fancy ways on how do I build the database for user management and spend a lot of time on the hashing of the password - lot has changed. Thanks to AWS Cognito; this helps you weave both the aspects of Authentication and Authorisation into your application.

In this example I am illustrating using Python & Flask - you can use the same concepts along with the nuances of the language you chose to develop your application with.

As I wade through AWS Cognito I find a lot of information scattered all over the place. I am trying to collate all that I found in one single place.

Let us deal with Part 1 - Authentication and follow it up with Authorisation.

There are two components of AWS Cognito - User Pool & Identity Pool
User Pool deals with Authentication and Identity pools deals with Authorisation

Let us consider the following features as part of User Management;
  • Allow the user to signup and gets a confirmation email 
  • The user forgets to confirm and is sent a new confirmation
  • The user confirms his account
  • User has forgotten the password and an email is sent to the user to reset the password
  • User used the code provided to set a new password
  • The user now uses the new password and performs a successful login to the system
Boto3 has the following functions to cater to all of the about with the client "cognito-idp"
  • sign_up
  • resend_verification
  • confirm_sign_up
  • forgot_password
  • confirm_forgot_password
  • initiate_auth
How does this work

The entire component is multi-layered. Following are the component layers
  • User Pools
  • App Client
  • Create an App client 

Before we dive deep into these functions - Let us create a user pool
The user pool has a set of standard attributes 
 given namemiddle name family name name 
 namenick name preferred username  email
 addressbirthdate phone number gender    
 localepicture profile zoneinfo 
 updated at website  

At the time of pool creation we can set any or all of these as required - you cannot change them afterwards

  • We can also define custom fields as needed
  • Specify password characterstics
  • Specify if users can signup or added by an administrator
  • Specify Multi-Factor Authentication
  • Password recovery methods
  • Customise email & SMS messages
  • Define Tags 
  • Indicate remembering user devices
  • Define workflow at various stages of creation of a user
There are a lot more you can do including federation of identities

Once you define the user pool define an App Client for this pool, the app client has options on how authentication can be performed by the app client

You will need the user pool id and the app client id within your code to perform the authentication, if you enable to generate client secret you will need this too

Let us now go back to the functionality we defined as part of authentication initially.

Following choices made for the example
  • Required Attributes
    • email
    • name
    • phone number
  • username
    • email
  • secret generated for app client
Function to create security hash
def get_secret_hash(username):
    msg = username + CLIENT_ID
    dig = hmac.new(str(CLIENT_SECRET).encode('utf-8'), 
        msg = str(msg).encode('utf-8'), digestmod=hashlib.sha256).digest()
    securityHash = base64.b64encode(dig).decode()
    return securityHash
The above function is called by all the boto3 api's when the App Client - generate secret is chosen, we get the has with the username - in our examples we will use email as the username


User Signup
        response = cidpClient.sign_up(
            ClientId=CLIENT_ID,
            SecretHash=get_secret_hash(email),
            Username=email,
            Password=password, 
            UserAttributes=[
                {
                    'Name': "name",
                    'Value': name
                },
                {
                    'Name': "email",
                    'Value': email
                },
                {
                    'Name': "phone_number",
                    'Value': phone
                }
            ],
            ValidationData=[
                {
                    'Name': "email",
                    'Value': email
                }
            ])
Pass in parameters which you have marked as required and also any user defined parameters

Verify Signup
        response = cidpClient.confirm_sign_up(
            ClientId=CLIENT_ID,
            SecretHash=get_secret_hash(email),
            Username=email,
            ConfirmationCode=code,
            ForceAliasCreation=False,
        )
Verification Reminder
        response = cidpClient.resend_confirmation_code(
            ClientId=CLIENT_ID,
            Username=email,
            SecretHash=get_secret_hash(email),
        )
Forgot Password
        response = cidpClient.forgot_password(
            ClientId=CLIENT_ID,
            Username=email,
            SecretHash=get_secret_hash(email),
        )
Confirm Forgot Password
        response = cidpClient.confirm_forgot_password(
            ClientId=CLIENT_ID,
            SecretHash=get_secret_hash(email),
            Username=email,
            ConfirmationCode=code,
            Password=password,
           )
Login
      response = cidpClient.initiate_auth(
                 ClientId=CLIENT_ID,
                 AuthFlow='USER_PASSWORD_AUTH',
                 AuthParameters={
                     'USERNAME': email,
                     'SECRET_HASH': get_secret_hash(email),
                     'PASSWORD': password,
                  })
For each of these functions handle the corresponding exceptions to ensure that we handle all the exceptions.

Next section we will discussion authorisation and refreshing the tokens





    



Wednesday, June 10, 2020

Deploy your first Kubernetes Service

We have setup our own Kubernetes cluster and we have installed the Kubernetes Dashboard. Let us try and install a service.

Nope, we are not going to do Hello World - I am bored with it

I created a small flask application which allows me to Browse S3 Buckets based on permission in IAM roles.

We will split this into two parts 
  1. Create a docker container and have this app running
  2. Create the same as a service in Kubernetes
Part 1 - Create a docker container and have an (Python Flask) app running inside the container 

Prerequisite:
  • Have Python Flask app which is running 
  • Install Docker
Steps to create the docker container
  • Create a folder for the app
    mkdir s3browser
  • Create the requirements file requirements.txt
    • List all the packages you need along with the versions
      Flask==1.1.2
      boto3==1.13.6
      werkzeug==0.16.1
      tzlocal==2.1
  • Create the Dockerfile
    • Choose a base image - Python:3.8.2
    • Specify the port for the container - 5000
    • Create the folder structure required in the container
    • Copy the files required from the local folder to the container
    • Specify the entry point - python
    • Specify the command to run - main.py
      FROM python:3.8.2
      WORKDIR /usr/src/app
      COPY requirements.txt ./
      RUN pip install -r requirements.txt
      RUN mkdir bootstrap fonts download upload
      COPY . .
      COPY ./bootstrap ./bootstrap
      COPY ./fonts ./fonts
      ENTRYPOINT ["python3"]
      CMD ["main.py"]
We now have all the required steps to create the docker image
[ec2-user@ip-10-0-1-38 app]$ sudo docker build -t s3browser .
Sending build context to Docker daemon  500.7kB
Step 1/10 : FROM python:3.8.2
3.8.2: Pulling from library/python
90fe46dd8199: Pull complete
35a4f1977689: Pull complete
bbc37f14aded: Pull complete
74e27dc593d4: Pull complete
4352dcff7819: Pull complete
deb569b08de6: Pull complete
98fd06fa8c53: Pull complete
7b9cc4fdefe6: Pull complete
512732f32795: Pull complete
Digest: sha256:8c98602bf4f4b2f9b6bd8def396d5149821c59f8a69e74aea1b5207713a70381
Status: Downloaded newer image for python:3.8.2
 ---> 4f7cd4269fa9
Step 2/10 : WORKDIR /usr/src/app
 ---> Running in 5d9cb6ae01f2
Removing intermediate container 5d9cb6ae01f2
 ---> 3e9513288f2c
Step 3/10 : COPY requirements.txt ./
 ---> 75b3d1cdab8d
Step 4/10 : RUN pip install -r requirements.txt
 ---> Running in 45827297fb12
Collecting Flask==1.1.2
  Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB)
Collecting boto3==1.13.6
  Downloading boto3-1.13.6-py2.py3-none-any.whl (128 kB)
Collecting werkzeug==0.16.1
  Downloading Werkzeug-0.16.1-py2.py3-none-any.whl (327 kB)
Collecting tzlocal==2.1
  Downloading tzlocal-2.1-py2.py3-none-any.whl (16 kB)
Collecting click>=5.1
  Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
Collecting itsdangerous>=0.24
  Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting Jinja2>=2.10.1
  Downloading Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
Collecting s3transfer<0 .4.0="">=0.3.0
  Downloading s3transfer-0.3.3-py2.py3-none-any.whl (69 kB)
Collecting jmespath<1 .0.0="">=0.7.1
  Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1 .17.0="">=1.16.6
  Downloading botocore-1.16.19-py2.py3-none-any.whl (6.2 MB)
Collecting pytz
  Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB)
Collecting MarkupSafe>=0.23
  Downloading MarkupSafe-1.1.1-cp38-cp38-manylinux1_x86_64.whl (32 kB)
Collecting python-dateutil<3 .0.0="">=2.1
  Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting docutils<0 .16="">=0.10
  Downloading docutils-0.15.2-py3-none-any.whl (547 kB)
Collecting urllib3<1 .26="">=1.20; python_version != "3.4"
  Downloading urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
Collecting six>=1.5
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: werkzeug, click, itsdangerous, MarkupSafe, Jinja2, Flask, six, python-dateutil, docutils, jmespath, urllib3, botocore, s3transfer, boto3, pytz, tzlocal
Successfully installed Flask-1.1.2 Jinja2-2.11.2 MarkupSafe-1.1.1 boto3-1.13.6 botocore-1.16.19 click-7.1.2 docutils-0.15.2 itsdangerous-1.1.0 jmespath-0.10.0 python-dateutil-2.8.1 pytz-2020.1 s3transfer-0.3.3 six-1.15.0 tzlocal-2.1 urllib3-1.25.9 werkzeug-0.16.1
WARNING: You are using pip version 20.1; however, version 20.1.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
Removing intermediate container 45827297fb12
 ---> ac86d48f81e6
Step 5/10 : RUN mkdir bootstrap fonts download upload
 ---> Running in 6fc0e64800a9
Removing intermediate container 6fc0e64800a9
 ---> e5b6c2505c07
Step 6/10 : COPY . .
 ---> 62002b4eefe0
Step 7/10 : COPY ./bootstrap ./bootstrap
 ---> 8909f349d50f
Step 8/10 : COPY ./fonts ./fonts
 ---> 1be3dd0c5262
Step 9/10 : ENTRYPOINT ["python3"]
 ---> Running in bf9e16f104bb
Removing intermediate container bf9e16f104bb
 ---> 4afaaef6b481
Step 10/10 : CMD ["main.py"]
 ---> Running in a5b3ecd0e25d
Removing intermediate container a5b3ecd0e25d
 ---> b63b9f6788b2
Successfully built b63b9f6788b2
Successfully tagged s3browser:latest
Run the docker image we have created
[ec2-user@ip-10-0-1-38 app]$ sudo docker run -d -p 5000:5000 s3browser
4a78858638e5c05e5c5352752ba04e16dbb7914f63ea38f77092060e117ff5b2
[ec2-user@ip-10-0-1-38 app]$ sudo docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS                    NAMES
4a78858638e5        s3browser           "python3 main.py"   18 seconds ago      Up 17 seconds       0.0.0.0:5000->5000/tcp   nostalgic_kalam
Open the app in your browser



Part 2 - Host this service in your Kubernetes Cluster

Steps to host service:
  • Create your free docker account - hub.docker.com - user gavihs
  • Create your repository (make it a public repository) - s3browser
  • Tag your image as <Account>/<Repository> - gavihs/s3browser
    [ec2-user@ip-10-0-1-38 app]$ sudo docker images
    REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
    s3browser           latest              6a56635d563d        20 minutes ago      1.01GB
    python              3.8.2               4f7cd4269fa9        4 weeks ago         934MB
    [ec2-user@ip-10-0-1-38 app]$ sudo docker tag 6a56635d563d gavihs/s3browser
    [ec2-user@ip-10-0-1-38 app]$ sudo docker images
    REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
    gavihs/s3browser    latest              6a56635d563d        21 minutes ago      1.01GB
    s3browser           latest              6a56635d563d        21 minutes ago      1.01GB
    python              3.8.2               4f7cd4269fa9        4 weeks ago         934MB
    
  • Push docker image to docker repository
    [ec2-user@ip-10-0-1-38 app]$ sudo docker login -u gavihs
    Password:
    WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
    Configure a credential helper to remove this warning. See
    https://docs.docker.com/engine/reference/commandline/login/#credentials-store
    
    Login Succeeded
    [ec2-user@ip-10-0-1-38 app]$ sudo docker push gavihs/s3browser
    The push refers to repository [docker.io/gavihs/s3browser]
    36e19271ecd9: Pushed
    a925ec3c1db0: Pushed
    15cd094801a0: Pushed
    9307954483b2: Pushed
    20ae7d89eead: Pushed
    d77ded6c9229: Pushed
    dd64728994ac: Pushed
    508c3f3b7a64: Pushed
    7e453511681f: Pushed
    b544d7bb9107: Pushed
    baf481fca4b7: Pushed
    3d3e92e98337: Pushed
    8967306e673e: Pushed
    9794a3b3ed45: Pushed
    5f77a51ade6a: Pushed
    e40d297cf5f8: Pushed
    latest: digest: sha256:9c579687302b0ba1683b263f9b1fab07080231d598325019c1f0afbf7a2031fb size: 3678
  • Create the deployment file - s3browser-deployment.yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: s3browser
    spec:
      selector:
        matchLabels:
          run: s3browser
      replicas: 1
      template:
        metadata:
          labels:
            run: s3browser
        spec:
          containers:
          - name: s3browser
            image: gavihs/s3browser:latest
            ports:
            - containerPort: 5000
  • Create the deployment
    ubuntu@ip-10-0-1-79:~/flask-app$ kubectl create -f s3browser-deployment.yaml
    deployment.apps/s3browser created
  • Create the service
    ubuntu@ip-10-0-1-79:~/flask-app$ kubectl expose deployment.apps/s3browser
    service/s3browser exposed
    ubuntu@ip-10-0-1-79:~/flask-app$ kubectl get service
    NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
    kubernetes   ClusterIP   100.64.0.1              443/TCP    9h
    s3browser    ClusterIP   100.67.84.177           5000/TCP   4s
  • Access the service from kubernetes
    https://api-k8-shivag-io-covt8s-1234567890.eu-west-1.elb.amazonaws.com//api/v1/namespaces/default/services/https:s3browser:/proxy/browse?folder=