In this blog post, I will walk you through what Kubernetes is, how and why it is used, what it can do, and who uses it today. Afterward, I’ll discuss how to first set up your Linux instance on AWS, install some dependencies then deploy Kubernetes on AWS using Terraform.
What is Kubernetes?
Kubernetes is an open-source container orchestration platform that automates many of the manual processes involved in deploying, managing, and scaling containerized applications. It was originally developed and designed by engineers at Google but it is now maintained by the Cloud Native Computing Foundation, a Linux organization created to advance container technology.
Kubernetes gives you the ability to cluster together groups of hosts running Linux containers and helps you easily and efficiently manage those clusters. Kubernetes clusters can span hosts across on-premise, public, private, or hybrid clouds. For this reason, Kubernetes is an ideal platform for hosting cloud-native applications that require rapid scaling; like real-time data streaming through Apache Kafka.
Containers support separation but with far less overhead and far greater flexibility. As a result, containers have reshaped the way people think about developing, deploying, and maintaining software. In a containerized architecture, the different services that constitute an application are packaged into separate containers and deployed across a cluster of physical or virtual machines. But this gives rise to the need for container orchestration—a tool that automates the deployment, management, scaling, networking, and availability of container-based applications.
Why use Kubernetes?
I have mentioned “containers” and “container orchestration” so some of you may be wondering how Kubernetes differs from other container technologies such as Docker, Docker Swarm, and Mesos.
In regards to Docker, Kubernetes actually makes docker better by allowing you to automate container provisioning, networking, load-balancing, security, and scaling across all these nodes from a single command line or dashboard. But Kubernetes does replace Docker Swarm.
It’s still possible to use Docker Swarm instead of Kubernetes, but Docker has chosen to make Kubernetes part of the Docker Community and Docker Enterprise editions going forward. Realistically speaking, Kubernetes is significantly more complex than Docker Swarm and requires more work to deploy. But, the work is intended to provide a big payoff in the long run by ensuring a more manageable, resilient application infrastructure. However, for smaller projects, Docker Swarm is the smarter choice.
As for Mesos, that technology does in fact offer container orchestration services, but it takes it to a whole new level. Mesos is a cloud operating system that can coordinate both containerized and non-containerized components. To that end, a lot of different platforms can run within Mesos—including Kubernetes itself.
So what can Kubernetes do?
Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to start. Wouldn’t it be easier if this behavior was handled autonomously?
This is where Kubernetes found its setting. It takes care of scaling and failover for your application, provides deployment patterns, and more. Its main features include:
Service Discovery and Load Balancing
Kubernetes can expose a container using the DNS name or using its own IP address. If traffic to a container is high, Kubernetes is able to load balance and distribute the network traffic so that the deployment is stable.
Storage Orchestration
Kubernetes allows you to automatically mount a storage system of your choice, such as local storage, public cloud providers, and more
Automated Roll Outs and Rollbacks
You can describe the desired state for your deployed containers using Kubernetes, and it can change the actual state to the desired state at a controlled rate. For example, you can automate Kubernetes to create new containers for your deployment, remove existing containers, and adopt all their resources to the new container.
Automatic Bin Packing
You provide Kubernetes with a cluster of nodes that it can use to run containerized tasks. You tell Kubernetes how much CPU and memory each container needs. Kubernetes can fit containers onto your nodes to make the best use of your resources.
Self-healing
Kubernetes restarts containers that fail, replaces containers, kills containers that don’t respond to your user-defined health check, and doesn’t advertise them to clients until they are ready to serve.
Secret and Configuration Management
Kubernetes lets you store and manage sensitive information, such as passwords, OAuth tokens, and SSH keys. You can deploy and update secrets and application configuration without rebuilding your container images, and without exposing secrets in your stack configuration.
Companies that use Kubernetes
Demo Walkthrough
To-Do:
- Install Java
- Install GIT
- Install Jenkins – which is an automation server for deployments
- Install Terraform
- Allow SUDO permissions for Jenkins User
- Make PasswordAuthentication to yes
- Install Kubectl (Kubernetes CLI)
- Install aws-iam-authenticator
- Credentials: AWS
1) Launch Linux instance on AWS (EC2)
- download key (you will need this to SSH into your Linux instance)
- rename instance
2) SSH to AWS instance from your SSH client
- copy IPv4
- username: ec2-user
- use private key
- root user: sudo -i
4) Install java & git
- yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel git -y
- check java: java -version
- check git: git --version
5) Install Jenkins
- sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins.io/redhat/jenkins.repo
- sudo rpm --import http://pkg.jenkins.io/redhat/jenkins.io.key
- sudo yum install jenkins
- check status: service jenkins status
- start jenkins: service jenkins status
6) Install terraform
- wget https://releases.hashicorp.com/terraform/0.13.0/terraform_0.13.0_linux_amd64.zip
- unzip terraform_0.13.0_linux_amd64.zip
- mv terraform /usr/bin
- terraform --version
7) Install Kubernetes CLI
- curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.17.9/2020-08-04/bin/linux/amd64/kubectl
8) Install aws-iam-authenticator
- curl -o aws-iam-authenticator https://amazon-eks.s3.us-west-2.amazonaws.com/1.17.9/2020-08-04/bin/linux/amd64/aws-iam-authenticator
- configure aws credentials (won't do it on the demo because of my secret keys)
9) Allow SUDO permissions for Jenkins User
- command: visudo
- Add line: jenkins ALL=(ALL) NOPASSWD: ALL
- :wq!
10) Change PasswordAuthentication to yes
- command: vi /etc/ssh/sshd_config
- Change PasswordAuthentication value from no to yes
- :wq!
11) Allow 8080 port
- go to security groups
- open security group id
- edit inbound rules
- add rule: 8080 port
- source: 0.0.0.0/0
12) Navigate to Jenkins in browser
- copy IPv4 + 8080
- copy password from SSH: cat + path from browser
- mention create job + pipeline
For now, that concludes the end of the demo. When I am able to resolve the issues I was experiencing with the Kubernetes provider for Terraform then I’ll update this post to reflect the final stage of getting Kubernetes up and running.
Cassandra.Link
Cassandra.Link is a knowledge base that we created for all things Apache Cassandra. Our goal with Cassandra.Link was to not only fill the gap of Planet Cassandra but to bring the Cassandra community together. Feel free to reach out if you wish to collaborate with us on this project in any capacity.
We are a technology company that specializes in building business platforms. If you have any questions about the tools discussed in this post or about any of our services, feel free to send us an email!
