Nope, we are not going to do Hello World - I am bored with it
I created a small flask application which allows me to Browse S3 Buckets based on permission in IAM roles.
We will split this into two parts
- Create a docker container and have this app running
- Create the same as a service in Kubernetes
Part 1 - Create a docker container and have an (Python Flask) app running inside the container
Prerequisite:
- Have Python Flask app which is running
- Install Docker
Steps to create the docker container
- Create a folder for the app
mkdir s3browser - Create the requirements file requirements.txt
- List all the packages you need along with the versions
Flask==1.1.2
boto3==1.13.6
werkzeug==0.16.1
tzlocal==2.1 - Create the Dockerfile
- Choose a base image - Python:3.8.2
- Specify the port for the container - 5000
- Create the folder structure required in the container
- Copy the files required from the local folder to the container
- Specify the entry point - python
- Specify the command to run - main.py
FROM python:3.8.2
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip install -r requirements.txt
RUN mkdir bootstrap fonts download upload
COPY . .
COPY ./bootstrap ./bootstrap
COPY ./fonts ./fonts
ENTRYPOINT ["python3"]
CMD ["main.py"]
We now have all the required steps to create the docker image
[ec2-user@ip-10-0-1-38 app]$ sudo docker build -t s3browser .
Sending build context to Docker daemon 500.7kB
Step 1/10 : FROM python:3.8.2
3.8.2: Pulling from library/python
90fe46dd8199: Pull complete
35a4f1977689: Pull complete
bbc37f14aded: Pull complete
74e27dc593d4: Pull complete
4352dcff7819: Pull complete
deb569b08de6: Pull complete
98fd06fa8c53: Pull complete
7b9cc4fdefe6: Pull complete
512732f32795: Pull complete
Digest: sha256:8c98602bf4f4b2f9b6bd8def396d5149821c59f8a69e74aea1b5207713a70381
Status: Downloaded newer image for python:3.8.2
---> 4f7cd4269fa9
Step 2/10 : WORKDIR /usr/src/app
---> Running in 5d9cb6ae01f2
Removing intermediate container 5d9cb6ae01f2
---> 3e9513288f2c
Step 3/10 : COPY requirements.txt ./
---> 75b3d1cdab8d
Step 4/10 : RUN pip install -r requirements.txt
---> Running in 45827297fb12
Collecting Flask==1.1.2
Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB)
Collecting boto3==1.13.6
Downloading boto3-1.13.6-py2.py3-none-any.whl (128 kB)
Collecting werkzeug==0.16.1
Downloading Werkzeug-0.16.1-py2.py3-none-any.whl (327 kB)
Collecting tzlocal==2.1
Downloading tzlocal-2.1-py2.py3-none-any.whl (16 kB)
Collecting click>=5.1
Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
Collecting itsdangerous>=0.24
Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting Jinja2>=2.10.1
Downloading Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
Collecting s3transfer<0 .4.0="">=0.3.0
Downloading s3transfer-0.3.3-py2.py3-none-any.whl (69 kB)
Collecting jmespath<1 .0.0="">=0.7.1
Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting botocore<1 .17.0="">=1.16.6
Downloading botocore-1.16.19-py2.py3-none-any.whl (6.2 MB)
Collecting pytz
Downloading pytz-2020.1-py2.py3-none-any.whl (510 kB)
Collecting MarkupSafe>=0.23
Downloading MarkupSafe-1.1.1-cp38-cp38-manylinux1_x86_64.whl (32 kB)
Collecting python-dateutil<3 .0.0="">=2.1
Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting docutils<0 .16="">=0.10
Downloading docutils-0.15.2-py3-none-any.whl (547 kB)
Collecting urllib3<1 .26="">=1.20; python_version != "3.4"
Downloading urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
Collecting six>=1.5
Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: werkzeug, click, itsdangerous, MarkupSafe, Jinja2, Flask, six, python-dateutil, docutils, jmespath, urllib3, botocore, s3transfer, boto3, pytz, tzlocal
Successfully installed Flask-1.1.2 Jinja2-2.11.2 MarkupSafe-1.1.1 boto3-1.13.6 botocore-1.16.19 click-7.1.2 docutils-0.15.2 itsdangerous-1.1.0 jmespath-0.10.0 python-dateutil-2.8.1 pytz-2020.1 s3transfer-0.3.3 six-1.15.0 tzlocal-2.1 urllib3-1.25.9 werkzeug-0.16.1
WARNING: You are using pip version 20.1; however, version 20.1.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
Removing intermediate container 45827297fb12
---> ac86d48f81e6
Step 5/10 : RUN mkdir bootstrap fonts download upload
---> Running in 6fc0e64800a9
Removing intermediate container 6fc0e64800a9
---> e5b6c2505c07
Step 6/10 : COPY . .
---> 62002b4eefe0
Step 7/10 : COPY ./bootstrap ./bootstrap
---> 8909f349d50f
Step 8/10 : COPY ./fonts ./fonts
---> 1be3dd0c5262
Step 9/10 : ENTRYPOINT ["python3"]
---> Running in bf9e16f104bb
Removing intermediate container bf9e16f104bb
---> 4afaaef6b481
Step 10/10 : CMD ["main.py"]
---> Running in a5b3ecd0e25d
Removing intermediate container a5b3ecd0e25d
---> b63b9f6788b2
Successfully built b63b9f6788b2
Successfully tagged s3browser:latestRun the docker image we have created
[ec2-user@ip-10-0-1-38 app]$ sudo docker run -d -p 5000:5000 s3browser
4a78858638e5c05e5c5352752ba04e16dbb7914f63ea38f77092060e117ff5b2
[ec2-user@ip-10-0-1-38 app]$ sudo docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4a78858638e5 s3browser "python3 main.py" 18 seconds ago Up 17 seconds 0.0.0.0:5000->5000/tcp nostalgic_kalamOpen the app in your browser
Part 2 - Host this service in your Kubernetes Cluster
Steps to host service:
- Create your free docker account - hub.docker.com - user gavihs
- Create your repository (make it a public repository) - s3browser
- Tag your image as <Account>/<Repository> - gavihs/s3browser
[ec2-user@ip-10-0-1-38 app]$ sudo docker images REPOSITORY TAG IMAGE ID CREATED SIZE s3browser latest 6a56635d563d 20 minutes ago 1.01GB python 3.8.2 4f7cd4269fa9 4 weeks ago 934MB [ec2-user@ip-10-0-1-38 app]$ sudo docker tag 6a56635d563d gavihs/s3browser [ec2-user@ip-10-0-1-38 app]$ sudo docker images REPOSITORY TAG IMAGE ID CREATED SIZE gavihs/s3browser latest 6a56635d563d 21 minutes ago 1.01GB s3browser latest 6a56635d563d 21 minutes ago 1.01GB python 3.8.2 4f7cd4269fa9 4 weeks ago 934MB - Push docker image to docker repository
[ec2-user@ip-10-0-1-38 app]$ sudo docker login -u gavihs Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded [ec2-user@ip-10-0-1-38 app]$ sudo docker push gavihs/s3browser The push refers to repository [docker.io/gavihs/s3browser] 36e19271ecd9: Pushed a925ec3c1db0: Pushed 15cd094801a0: Pushed 9307954483b2: Pushed 20ae7d89eead: Pushed d77ded6c9229: Pushed dd64728994ac: Pushed 508c3f3b7a64: Pushed 7e453511681f: Pushed b544d7bb9107: Pushed baf481fca4b7: Pushed 3d3e92e98337: Pushed 8967306e673e: Pushed 9794a3b3ed45: Pushed 5f77a51ade6a: Pushed e40d297cf5f8: Pushed latest: digest: sha256:9c579687302b0ba1683b263f9b1fab07080231d598325019c1f0afbf7a2031fb size: 3678 - Create the deployment file - s3browser-deployment.yaml
apiVersion: apps/v1 kind: Deployment metadata: name: s3browser spec: selector: matchLabels: run: s3browser replicas: 1 template: metadata: labels: run: s3browser spec: containers: - name: s3browser image: gavihs/s3browser:latest ports: - containerPort: 5000 - Create the deployment
ubuntu@ip-10-0-1-79:~/flask-app$ kubectl create -f s3browser-deployment.yaml deployment.apps/s3browser created - Create the service
ubuntu@ip-10-0-1-79:~/flask-app$ kubectl expose deployment.apps/s3browser service/s3browser exposed ubuntu@ip-10-0-1-79:~/flask-app$ kubectl get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 100.64.0.1443/TCP 9h s3browser ClusterIP 100.67.84.177 5000/TCP 4s - Access the service from kubernetes
https://api-k8-shivag-io-covt8s-1234567890.eu-west-1.elb.amazonaws.com//api/v1/namespaces/default/services/https:s3browser:/proxy/browse?folder=
