Google Cloud offers different services to train and deploy machine learning algorithms on cloud. In this tutorial, we will deploy an Object Detection model using flask as a web service on Google Cloud Run using Docker. We will also work with continuous deployment using github to easily deploy models with just git push. In this tutorial, we will also work to store models on gcloud storage and load models from there. So, here are list of steps we will be performing to make all this work and create a live web service.

  1. Create a Basic Flask Web App 
  2. Perform Inference using Models
  3. Setup Gcloud bucket and Load models
  4. Deploy on GCloud Run
  5. Setup Continuous Deployment with Github

So, let start creating a flask application.

1. Flask Application

Flask is a very popular framework for development of web applications and apis for Deep Learning models. So, first we create a basic flask server with single page to upload files to server. For installation, use pip to install flask on your system.

pip install flask

Now we create a flask web server. Create a python file in a directory and create app. For this, we are creating two initial routes.

  1. Index page to show html page
  2. Upload route to upload images for inference
# import required modules
from flask import Flask, render_template, request

# create flask app
app = Flask(__name__)

# main route (show html page)
def index():
    return render_template('index.html')

# api endpoint for image upload
@app.route('/api/upload', methods=['POST'])
def upload():
    # receive the file from the client
    file = request.files['file']
    filepath = f'static/temp/{file.filename}' # save to directory
    # return server url to client
    return f"{request.url_root}{filepath}"

# Run flask server
if __name__ == '__main__': # set debug true to load reload server auto on changes

Now, we need to define an HTML page for templates defined in index route. We can create a directory named templates inside current folder and create index.html inside this directory. We are using bootstrap for designing and some custom css code is inside a css file. For requests and basic image handling, we are using javascript with jquery.

<!DOCTYPE html>
<html lang="en">

    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <!-- Latest compiled and minified CSS -->
    <link rel="stylesheet" href="" crossorigin="anonymous">
    <link rel="stylesheet" href="{{ url_for('static', filename='css/styles.css') }}" />

<div class="container main-container">
    <div class="row">
        <div class="col-md-10 form-group">
            <label for="fileInput">File input</label>
            <input type="file" class="form-control" onchange="loadImg()" id="fileInput" aria-describedby="fileHelp">
        <div class="col-md-2 btn-submit-container">
        <button type="button" class="btn btn-primary" id="upload">Process Image</button>
        <img src="" id="imagePreview" class="image-preview">
<script src="" integrity="sha256-/xUj+3OJU5yExlq6GSYGSHk7tPXikynS7ogEvDej/m4=" crossorigin="anonymous"></script>
<script src="{{ url_for('static', filename='js/utils.js') }}">

For static files created above, we can create another folder named static to server our static files, where we create static/css/styles.css and static/js/utils.js files.

Fore styles.css

    padding-top: 50px;
    padding-top: 25px
    max-width: 85%;

And for file preview and upload, we use ajax using jquery to upload files to api.

For utils.js

function loadImg(){
    // Show preview image for select file before upload
    $('#imagePreview').attr('src', URL.createObjectURL([0]));

// Upload image using ajax
    // Create form data
    var formData = new FormData();
    // add file to form data
    formData.append('file', $('#fileInput')[0].files[0]);
        url: '/api/upload', // API Endpoint
        type: 'POST', // Request type
        data: formData, // Request data
        contentType: false,
        processData: false,
        success: function(data){
            // On request succss, we show image from server
            $('#imagePreview').attr('src', data);

After creating all these files, we can run server and test current workflow of api. To run flask server open command prompt and run this command.


It will start server on localhost with port 5000 by default. Go tohttp://localhost:5000 in any browser and go to this url to view application.

Inference using Tensorflow

Next, we create script to load Deep Learning models and perform inference. For this tutorial, we are using Tensorflow and will use pretrained models from tensorflow model zoo which is trained on 90 classes and different models are avialable based on speed and accuracy. For more information on this, view this post.

First we need to write script to fetch models, so for this tutorial we are using Tensorflow model zoo and fetch models from their repository and perform inference. Create a file named utils to fetch any model, unzip it and provide full qualified path to load model using tensorflow. For fetching model from repository and unzip it, we are using python package gdown.

Filepath: src/

import os, gdown

base_url = ""

def download_model(model_name):
    """ Download model from if not exists extract and return path to model """
    url = f"{base_url}{model_name}" # Model URL
    model_path = f"models/{model_name}" # Model zip path

    model_dir = model_path.replace(".tar.gz", "") # Model directory path

    if not os.path.exists(model_dir):
        # If not exists download model and extract
        gdown.cached_download(url, model_path, postprocess=gdown.extractall)

        # remove the zip file

    return os.path.join(model_dir, "saved_model")

Now, we create a script inside a directory which process an image from path and write to file.

Filepath: src/

import numpy as np
import tensorflow as tf
import cv2
from src.utils import download_model

# Model to Use
model_name = "ssd_mobilenet_v2_fpnlite_640x640_coco17_tpu-8.tar.gz"

# load model from path (it will download from url if not exists locally)
model= tf.saved_model.load(download_model(model_name))

def process_image(image_path):
    # read image and preprocess
    img = cv2.imread(image_path)
    h, w, _ = img.shape
    img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
    input_tensor = np.expand_dims(img, 0)

    # predict from model
    resp = model(input_tensor)

    # get the output of the prediction
    # iterate over boxes, class_index and score list
    for boxes, classes, scores in zip(resp['detection_boxes'].numpy(), resp['detection_classes'], resp['detection_scores'].numpy()):
        for box, cls, score in zip(boxes, classes, scores): # iterate over sub values in list
            if score > 0.6: # we are using only detection with confidence of over 0.8
                ymin = int(box[0] * h)
                xmin = int(box[1] * w)
                ymax = int(box[2] * h)
                xmax = int(box[3] * w)
                # draw on image
                cv2.rectangle(img, (xmin, ymin), (xmax, ymax), (128, 0, 128), 4)

    # convert back to bgr and save image
    cv2.imwrite(image_path, cv2.cvtColor(img, cv2.COLOR_RGB2BGR))

This code can be utilized in app upload route and we can process image after upload.

@app.route('/api/upload', methods=['POST'])
def upload():
    # process image after upload

    # return server url to client
    return f"{request.url_root}{filepath}"

After this, reload page and upload image to process and response should be image updated with detections drawn on image. Once application is working fine, we can move to next step.

Dockerize Flask App

GCloud offers different options to setup and deploy application on cloud. We will be using GCloud Run and for that we need to dockerize our application. For docker, we need to define all required packages in a requirements file and then we create dockerfile to specify workflow.

In requirements.txt


Now we define dockerfile and we will be using base tensorflow image from tensorflow repository on docker hub.

FROM tensorflow/tensorflow

# Install production dependencies.
ADD requirements.txt .
RUN pip install -r requirements.txt

# Copy local code to the container image.
COPY . .

# Service must listen to $PORT environment variable.
# This default value facilitates local development.

# Run the web service on container startup. Here we use the gunicorn
# webserver, with one worker process and 8 threads.
# For environments with multiple CPU cores, increase the number of workers
# to be equal to the cores available.
CMD exec gunicorn --bind$PORT --workers 1 --threads 8 --timeout 0 main:app

Now we can build docker image using docker. Open command prompt in project root directory and write following command. 

Change tensorflow-api to any other name, if you want.

docker build -t tensorflow-api .

Once docker image is ready, we can test locally to verify is application is working fine inside docker by create a docker container using this image.

docker run -p 8080:8080 tensorflow-api

Once container is running, navigate to http://localhost:8080 in your browser and it should work. Here is a sample image tested in application.

My alt text

Deploy on Google Cloud

After testing application, this can be deployed on any cloud service you want easily using Docker. For this example, we will deploy this application on google cloud. Google cloud has different options to deploy and we will use GCloud Run service for deployment. There are also options to deploy using either gcloud command line tool, github or other services.

Deploy using GCloud CLI

First, we deploy using Google Cloud CLI. First setup a project on google cloud console and install google cloud cli tools using this url on your OS.

Ater setting up google cloud sdk, you can login to gcloud sdk and select your project. For example.

# login to google account
gcloud auth login

# select project
gcloud config set project PROJECT_ID

Once you have setup project, then you can build docker image using gcloud and store on google container registery.

# replace PROJECT_ID with your project id and image name if your want

gcloud builds submit --tag

After completion, we can create a cloud run service by visiting to this url.

Now we can select a docker container from docker image in google container registery or we can setup build from github. For this step we select docker image we created above. Click on select in Container Image URL button and select your required image.

My alt text
Select a image from given builds

After select, there are other configurations that you can modify.

  • Service Name - Name of cloud run service
  • Region - Select any suitable region
  • CPU Allocation - Whether to always allocate CPU or only on request and you are charged accordingly.
  • Autoscalling - Max and Min number of instance you want to manage traffic

And then there are other configurations related to request managment as only to cloud services or all and authentication. In advanced settings, you can setup CPU, RAM, ports and other details. For this example, we setup 2 CPU and 4GB of RAM. Once done with configurations, we can create service. Once deployment is complete, navigate to given url and you can test service.

For github, view continuous deployment section.

Setup Continuous Deployment

Continuous deployment is when we use some trigger to autmatically deploy changes to application. We will use github to setup continuous deployment for application so once we make changes to app and push to github, it will be automatically deployed.

First create a repository on github and push your code to that repository so that can be usedMy alt text for continuous deployment.

Open service on gcloud run and cloud on Setup Continuous Deployment on top and it will open sidebar to setup continuous deployment. Here you can select different options like Github, Bitbucket and Cloud resources. For github, you will need to login using github and then it will display all repositories.

My alt text

Select your repository and click on Next button. In next tab, select branch name that on change will trigger build and build type which in our case is dockerfile. Select dockerfile location which in this case is default.

Now you can click build to create a build and it will deploy latest code on github to gcloud service. Now it will trigger every time you push changes to main branch. All configurations related to build can be changed in build section.

For more information related to gcloud run, visit gcloud documentation.