Going back to an article that I wrote a while ago, Container security in the Cloud. A state of uncertainty. - Florin Loghiade, I was writing a
Recently I started working a lot more with Kubernetes and I started migrating more and more workloads towards it.
My latest challenge was to migrate an Azure bot that was running in an Azure App Service to Kubernetes and the problem was that simply creating a container with the bot and starting it in Kubernetes was not enough.
Azure App Services do a lot of magic behind the scenes to make a web application work. For example, it exposes Python applications with Gunicorn whether they are Flask apps or aio http.
So you would think that slapping an NGINX ingress route towards the container would solve the problem without any code changes but the reality is that it's not that simple.
Let's take the Python Bot Framework bot example
This example works perfectly locally with the Bot Emulator and in an App Service Plan but if we build a container with the same sample using the following dockerfile
We will find out very fast that the container will fail with a boatload of http 404 or http 500 errors or just plainly not work at all. The amount red herrings its going to throw at you will make your head hurt.
So what's the solution?
The solution is quite simple after you figure it out but nerve wrecking while debugging it.
First of all, we need to convert the app.py file from aiohttp to flask and run Gunicorn on top.
The above shows roughly what you need to change to make it a flask app. For Bot Framework you will have to create an async loop for it to keep running hence the import of asyncio and the Loop = asyncio.get_event_loop()
Once we converted the app.py to flask, we need to adjust the dockerfile to run Gunicorn. So first add in your requirements.txt file Gunicorn and flask with what ever versions you want to run then create the following dockerfile
FROM python:3.8.3 RUN apt-get clean \ && apt-get -y update COPY . /app/bot RUN mkdir logs WORKDIR /app/bot COPY requirements.txt /app/bot RUN pip install --upgrade pip RUN pip install wheel RUN pip install -r requirements.txt EXPOSE 3978 CMD ["gunicorn", "app:APP" ,"--bind", "0.0.0.0", "--access-logfile=gunicorn-access.log" ,"--error-logfile" ,"gunicorn-error.log"]
As you can see instead of running python app.py we're running Gunicorn and calling the flask app which in our example is called APP.
Now we can send it to Kubernetes and expose it with your favorite ingress system or just attach an IP to it.
Gunicorn by default runs on port 8000 so we're going to create the pod and service to expose the containerport 8000
The yaml above shows how to create everything and attach an Azure IP to it so you can take Bot Emulator and test it.
One key takeaway from this example is that bot emulator needs to run ngrok otherwise the bot will want to send responses to localhost which will break it.
Once you managed to get the bot running in a Kube cluster, all that remains is to optimize it a bit for large scale. The example above will not scale well, it just showcases some hoops you have to jump to get it working.
That being said, have a good one.