We have created our clusters and we have a few services running. How can we control the nodes that are created using kops
What are the AWS Resources Created
- VPC
- Subnet
- 3 for the cluster master and nodes
- 3 for bastions
- Routes
- 1 route for the Bastions
- 3 routes for each region
- Routes
- Bastion route table have routes 0.0.0.0/0 going to the internet gateway
- Cluster route table have routes 0.0.0.0/0 going to the NAT gateway
- Internet Gateway
- Elastic IP
- 3 public ip's attached to each of the 3 NAT gateways
- NAT Gateways
- 3 NAT Gateways for each region
- Security Groups
- 3 security groups for each of Bastion, Master & Nodes
- Load Balancers
- 1 for Master
- 1 for Bastion
- Launch Configurations
- 1 each for Master, Node & Bastion
- Autoscaling Groups
- 1 each corresponding to Launch configurations
- Instances
- As requested for Master, Nodes and Bastion
Advanced Configuration
kops has the following parameters which enables you to control your cluster
- master-count -> Allows you to specify the number of masters
- master-size -> Allows you to specify the size of the master machine
- node-count -> Allows you to specify the number of nodes
- node-size -> Allows you to specify the size of the node machines
Use spot instances to reduce costs
It is no secret that Spot instances are much cheaper than on-demand instances. You can have the master and/or nodes as spot instances with this hack
Master ==> kops edit ig master-eu-west-1a --name k8.shivag.io --state s3://k8-kops-cluster-state-s3
Nodes ==> kops edit ig nodes --name k8.shivag.io --state s3://k8-kops-cluster-state-s3
Bastion ==> kops edit ig bastions --name k8.shivag.io --state s3://k8-kops-cluster-state-s3
apiVersion: kops.k8s.io/v1alpha2
kind: InstanceGroup
metadata:
creationTimestamp: "2020-05-18T19:28:46Z"
labels:
kops.k8s.io/cluster: k8.shivag.io
name: master-eu-west-1a
spec:
image: kope.io/k8s-1.17-debian-stretch-amd64-hvm-ebs-2020-01-17
machineType: t2.medium
maxPrice: "0.05" <=== Max cost for spot instance
maxSize: 1
minSize: 1
nodeLabels:
kops.k8s.io/instancegroup: master-eu-west-1a
role: Master
subnets:
- eu-west-1aEnsure you have the price corresponding to the instance you are using
Additional Security Group Assignment
The following flags on kops ensure you can have additional security groups assigned to the instance groups
--master-security-groups
--node-security-groups
Note:
- Security Groups are attached to VPC; we need to have the VPC Pre-created and specify the VPC during the create phase or we need to make this change later and propagate it; in which case we need to remove the security group and delete the security group before deleting the cluster
Additional Security Group for Bastions
ubuntu@ip-10-0-1-79:~$ kops edit ig bastions --name k8.shivag.io --state s3://shivag.kube-kops-state piVersion: kops.k8s.io/v1alpha2 kind: InstanceGroup metadata: creationTimestamp: "2020-05-21T21:40:33Z" generation: 1 labels: kops.k8s.io/cluster: k8.shivag.io name: bastions spec: additionalSecurityGroups: - sg-095b938fcbad614bc image: kope.io/k8s-1.17-debian-stretch-amd64-hvm-ebs-2020-01-17 machineType: t2.micro maxSize: 1 minSize: 1 nodeLabels: kops.k8s.io/instancegroup: bastions role: Bastion subnets: - utility-eu-west-1a - utility-eu-west-1b - utility-eu-west-1c
