Deploying a Flask API and MongoDB on Kubernetes

Deploying a Flask API and MongoDB on Kubernetes

Kubernetes is a powerful container orchestration platform that allows you to deploy and manage complex applications at scale. In this tutorial, we'll walk through the process of setting up a Kubernetes cluster on AWS EC2, deploying a Flask API and a MongoDB instance on the cluster, and troubleshooting some common issues that you may encounter along the way.

Prerequisites

Before we get started, make sure you have the following tools and services installed and configured:

  • Two AWS EC2 instances with Ubuntu 20.04 LTS installed of instance type t2.medium

  • Basic knowledge of Kubernetes concepts and YAML syntax

Step 1: Setting up the Kubernetes cluster

The first step in deploying our Flask API and MongoDB on Kubernetes is to set up a Kubernetes cluster. I'll use the kubeadm tool to set up a single-node cluster with one master and one worker node.

Step 1.1: Installing Kubernetes dependencies

First, we need to install some dependencies on our EC2 instance (both master and worker):

sudo apt update -y
sudo apt install docker.io -y

sudo systemctl start docker
sudo systemctl enable docker

sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg

echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt update -y
sudo apt install kubeadm=1.20.0-00 kubectl=1.20.0-00 kubelet=1.20.0-00 -y

Step 1.2: Initializing the cluster

Next, we need to initialize the Kubernetes cluster on the master node:

sudo su
kubeadm init

This command will create a new Kubernetes cluster with the default configuration and networking settings. The --pod-network-cidr option specifies the IP address range that will be used for pod networking in the cluster.

Step 1.3: Configuring kubectl

After initializing the cluster, we need to configure kubectl to connect to it:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

This will copy the Kubernetes configuration file to your local machine and configure kubectl to use it.

Step 1.4: Installing Weave Net CNI plugin

Then we will deploy the Weave Net CNI plugin as a DaemonSet in your Kubernetes cluster. This will allow the Weave Net plugin to be run on every node in the cluster.

kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml

Step 1.5: Generating token to join nodes

And then will generate a new bootstrap token for joining new nodes to our Kubernetes cluster

kubeadm token create --print-join-command

Step 1.6: Joining the worker node

Finally, we need to join the worker node to the cluster. On the worker node, run the command that we goy in output from command in step 1.5:

sudo su
kubeadm reset pre-flight checks
Paste the Join command on worker node and append `--v=5` at end

Here, I have used kubeadm reset pre-flight checks because it performs a series of pre-flight checks to ensure that the node is in a clean state and can be used as a worker node.

Congratulations! You now have a functioning Kubernetes cluster with one master and one worker node.

Step 2: Deploying the Flask API

Now that we have our Kubernetes cluster set up, we can deploy the Flask API on it. I'll use a Docker image that I've already created and pushed to Docker Hub.

Note: If you want to see the code and follow along with the steps, you can find everything I am doing in my GitHub repository: github.com/ambuzrnjn33/taskmaster-k8s.git

Step 2.1: Creating the deployment

To create a deployment for the Flask API, we need to define a YAML file that describes the deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: taskmaster
  labels:
    app: taskmaster
spec:
  replicas: 1
  selector:
    matchLabels:
      app: taskmaster
  template:
    metadata:
      labels:
        app: taskmaster
    spec:
      containers:
        - name: taskmaster
          image: ambuzrnjn33/taskmaster:latest
          ports:
            - containerPort: 5000
          imagePullPolicy: Always

Save this file as taskmaster.yml and deploy it to the Kubernetes cluster with the following command:

kubectl apply -f taskmaster.yml

This will create a new deployment for the Flask API with one replica.

Step 2.2: Scaling the deployment

To scale the deployment to three replicas, run the following command:

kubectl scale --replicas=3 deployment/taskmaster

This will increase the number of replicas of the deployment to three.

Step 2.3: Creating the service

Next, we need to create a Kubernetes Service to expose the Flask API to the outside world:

apiVersion: v1
kind: Service
metadata:
  name: taskmaster-svc
spec:
  selector:
    app: taskmaster
  ports:
    - port: 80
      targetPort: 5000
      nodePort: 30007
  type: NodePort

Save this file as taskmaster-svc.yml and deploy it to the Kubernetes cluster with the following command:

kubectl apply -f taskmaster-svc.yml

This will create a new Kubernetes Service with a NodePort that maps port 80 to port 5000 on the Flask API container.

Step 2.4: Accessing the API

Now that we have deployed the Flask API and created a Kubernetes Service for it, we can access it from outside the cluster using the IP address of the worker node and the NodePort we specified in the taskmaster-svc.yml file.

You can access the API at http://<ip-of-worker-node>:30007/.

Step 3: Deploying MongoDB

Now that we have the Flask API up and running, we need to deploy a MongoDB instance to store the data for the API. I'll use PersistentVolumeClaim to ensure that the data is persisted even if the MongoDB container is restarted.

Step 3.1: Creating the PersistentVolume

First, we need to create a PersistentVolume to store the data for MongoDB:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: mongo-pv
spec:
  capacity:
    storage: 256Mi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: /tmp/db

Save this file as mongo-pv.yml and deploy it to the Kubernetes cluster with the following command:

kubectl apply -f mongo-pv.yml

This will create a new PersistentVolume with 256MB of storage.

Step 3.2: Creating the PersistentVolumeClaim

Next, we need to create a PersistentVolumeClaim to request storage from the PersistentVolume we just created:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mongo-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 256Mi

Save this file as mongo-pvc.yml and deploy it to the Kubernetes cluster with the following command:

kubectl apply -f mongo-pvc.yml

This will create a new PersistentVolumeClaim that requests 256MB of storage.

Step 3.3: Creating the Mongo Deployment:

Finally, we can create the Deployment for MongoDB:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongo
  labels:
      app: mongo
spec:
  selector:
    matchLabels:
      app: mongo
  template:
    metadata:
      labels:
        app: mongo
    spec:
      containers:
        - name: mongo
          image: mongo
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: storage
              mountPath: /data/db
      volumes:
        - name: storage
          persistentVolumeClaim:
            claimName: mongo-pvc

Save this file as mongo.yml and deploy it to the Kubernetes cluster with the following command:

kubectl apply -f mongo.yml

This will create a new Deployment with one replica that runs the official MongoDB container and mounts the persistent volume claim we created earlier.

Step 3.4: Creating the MongoDB Service

Finally, we need to create a Service to expose the MongoDB instance to the Flask API:

apiVersion: v1
kind: Service
metadata:
  labels:
    app: mongo
  name: mongo
spec:
  ports:
    - port: 27017
      targetPort: 27017
  selector:
    app: mongo

Save this file as mongo-svc.yml and deploy it to the Kubernetes cluster with the following command:

kubectl apply -f mongo-svc.yml

This will create a new Service that maps port 27017 to the MongoDB container.

Step 4: Test the Flask API

With the MongoDB database and Flask API deployed, we can now test the API to ensure that everything is working as expected. To do this, I'll use the curl command to make HTTP requests to the API.

Create a task

To create a task in the API, run the following command:

curl -H "Content-Type: application/json" -X POST -d '{"title": "Do laundry", "description": "Wash and fold clothes"}' http://<server-ip>:30007/tasks

Replace <server-ip> with the IP address of the node that the Flask API Service is running on. If the task is created successfully, you should see a JSON response containing the task details:

{"_id": {"$oid": "60924af78432c1d826ef23ec"}, "title": "Do laundry", "description": "Wash and fold clothes"}

Get all tasks

To get all tasks from the API, run the following command:

curl http://<server-ip>:30007/tasks

Replace <server-ip> with the IP address of the node that the Flask API Service is running on. If everything is working correctly, you should see a JSON response containing the task you just created:

[{"_id": {"$oid": "60924af78432c1d826ef23ec"}, "title": "Do laundry", "description": "Wash and fold clothes"}]

Get a specific task

To get a specific task by ID, run the following command:

curl http://<server-ip>:30007/tasks/<task-id>

Replace <server-ip> with the IP address of the node that the Flask API Service is running on, and <task-id> with the ID of the task you want to retrieve. If everything is working correctly, you should see a JSON response containing the details of the specified task:

{"_id": {"$oid": "60924af78432c1d826ef23ec"}, "title": "Do laundry", "description": "Wash and fold clothes"}

Update a task

To update a task by ID, run the following command:

curl -H "Content-Type: application/json" -X PUT -d '{"title": "Do laundry", "description": "Wash and fold clothes and towels"}' http://<server-ip>:30007/tasks/<task-id>

Replace <server-ip> with the IP address of the node that the Flask API Service is running on, and <task-id> with the ID of the task you want to update. If the task is updated successfully, you should see a JSON response containing the updated task details:

{"_id": {"$oid": "60924af78432c1d826ef23ec"}, "title": "Do laundry", "description": "Wash and fold clothes and towels"}

Delete a task

To delete a task by ID, run the following command:

curl -X DELETE http://<server-ip>:30007/tasks/<task-id>

Replace <server-ip> with the IP address of the node that the Flask API Service is running on, and <task-id> with the ID of the task you want to delete. If the task is deleted successfully, you should see a JSON response containing the ID of the deleted task:

{"_id": {"$oid": "60924af78432c1d826ef23ec"}}

Troubleshooting

During the deployment process, you may encounter some errors. Here are a few common errors and how to troubleshoot them:

Error: "The connection to the server localhost:8080 was refused"

This error occurs when the Kubernetes API server is not running or is not accessible from the machine you are deploying from. The API server is responsible for managing the Kubernetes cluster and exposing the Kubernetes API to client applications. To troubleshoot this error, make sure that the Kubernetes API server is running and that you have the correct URL for accessing it. You can try running the following command to check if the API server is running:

kubectl cluster-info

If the API server is not running, you may need to start it by running the following command:

sudo systemctl start kube-apiserver

Error: "pod has unbound immediate PersistentVolumeClaims"

This error occurs when a Pod is unable to find a PersistentVolume to use for storage. Kubernetes uses PersistentVolumes to provide persistent storage for containers in a cluster. When a Pod requests storage, it creates a PersistentVolumeClaim that specifies the size and access mode of the storage it needs. To troubleshoot this error, make sure that you have created a PersistentVolume and a PersistentVolumeClaim that match the storage requirements of the Pod. You can check the status of your PersistentVolumeClaims by running the following command:

kubectl get pvc

If a PersistentVolumeClaim is not bound to a PersistentVolume, you may need to create a new PersistentVolume that matches the requirements of the PersistentVolumeClaim.

Error: "ServerSelectionTimeoutError: mongo:27017"

One common DNS issue that can occur in Kubernetes is when Pods are unable to resolve the IP addresses of other Pods or services in the cluster. This can lead to errors like pymongo.errors.ServerSelectionTimeoutError when trying to connect to a MongoDB instance.

This error occurs when the Flask API is unable to connect to the MongoDB instance. MongoDB is a document-oriented database that stores data in flexible JSON-like documents. To troubleshoot this error:

  1. Check that the affected Pods are running and are healthy. You can use the kubectl get pods command to see if the Pods are running. You can also check the logs for the affected containers using kubectl logs to see if there are any issues.

  2. Check that the affected Pods are able to communicate with each other. You can use the kubectl exec command to shell into the affected containers and try to ping the IP addresses of other Pods or services in the cluster. If the ping is successful, it means that the Pods are able to communicate with each other.

  3. Check the DNS service in your Kubernetes cluster. Kubernetes uses a DNS service to resolve domain names to IP addresses within the cluster. If there is an issue with the DNS service, it can cause Pods and containers to be unable to resolve the IP addresses of other Pods and services in the cluster. You can use the kubectl logs command to view the logs for the DNS Pods and see if there are any errors or issues.

  4. Restart the DNS service. If you've identified an issue with the DNS service, you can try restarting it by deleting the kube-dns Pods using the kubectl delete pods -n kube-system -l k8s-app=kube-dns command. This will trigger a restart of the DNS service, which may resolve the issue.

Conclusion

In this blog post, I have demonstrated how to deploy a Flask API and a MongoDB instance to a Kubernetes cluster. I have shown how to create a Deployment, scale it, and expose it to the outside world using a Service. I have also shown how to deploy a MongoDB instance using a PersistentVolumeClaim, and how to connect it to the Flask API using a Service. Finally, I have discussed some common errors and how to troubleshoot them.

Kubernetes provides a powerful platform for deploying and managing containerized applications at scale. By following the steps outlined in this blog post, you can easily deploy a Flask API and a MongoDB instance to a Kubernetes cluster and start building your own microservices architecture.


Thank you for reading this Blog. Hope you learned something new today! If you found this blog helpful, please like, share, and follow me for more blog posts like this in the future.

If you have some suggestions I am happy to learn with you.

I would love to connect with you on LinkedIn

Meet you in the next blog....till then Stay Safe ➕ Stay Healthy

#HappyLearning #Kubernetes #Flask #MongoDB #Microservices #Docker #DevOps #Containerization #CloudNative #Deployment #PersistentVolume #Service #API #ErrorHandling #Troubleshooting #devops #Kubeweek #KubeWeekChallenge #TrainWithShubham #KubeWeek_Challenge #kubeweekTroubleshooting