Webifying Machine Learning: Unleashing its Potential

A few months ago, I gave a talk on Tensorflow.js at a javascript meetup and I’ve been procrastinating writing an article about it for ages! Well, quarantine has left me with a lot of free time lately and I decided to finally pen it down.

Let’s first talk about Tensorflow.

TensorFlow is a library developed by the Google Brain Team to accelerate machine learning and deep neural network research.

It is called Tensorflow because it takes input as a multi-dimensional array, also known as tensors. You can construct a sort of flowchart of operations (called a Graph) that you want to perform on that input. The input goes in at one end, and then it flows through this system of multiple operations and comes out the other end as output.

Tensorflow.js.

Tensorflow can be a little complex to use for web developers as it requires many installations and prerequisites. Tensorflow.js bridges the gap between web development and machine learning. Tensorflow.js allows you to run complex Machine learning models on your web.

With tensorflow.js you can

1. keep user data local

2. no prerequisites or installs

3. Interactive and easy to use

The main components of tensorflow.js are:

1. WebGL-is a standard in-browser that allows you to render 3D graphics, tensorflow.js utilizes it to do linear algebra for complex matrix calculations.

2. With eager execution enabled, TensorFlow functions execute operations immediately

3. Layers API that allows you to use best practices and high-level building blocks to build models (Keras)

Working principle

ML running in the browser means that from a user’s perspective, there’s no need to install any libraries or drivers, just open a webpage, and your program is ready to run. Also, it’s ready to run with GPU acceleration. TensorFlow.js automatically supports WebGL and will accelerate your code behind the scenes when a GPU is available.

What can we do with Tensorflow.js

1. Write model directly in browser

2. import pre-trained models for interface

3. RE-train imported models

Object Detection using tensorflow.js

I developed a simple web app to detect various objects using a pre-trained javascript model. Go ahead and check it out for a better understanding of this article.

The code is pretty straight forward so I am not going to explain every line of it in detail. It mainly consists of three parts:

1. Loading tensorflow.js in a script tag.

Go to https://github.com/tensorflow/tfjs-models/tree/master/mobilenet and copy the script tag and add it to your code.

!-- Load TensorFlow.js. This is required to use MobileNet. -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@1.0.1"> </script>
<!-- Load the MobileNet model. -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/mobilenet@1.0.0"> </script>

2. Writing the Javascript code.

async function predict(){              context.drawImage(video,0,0,500,500)             
const predictions=await model.classify(canvas)             if(predictions[0].probability>=0.5)
{               
status.innerHTML=`Prediction: ${predictions[0].className} / ${predictions[0].probability}`               
status.style.fontSize = "25px";            
}              
requestAnimationFrame(predict)           
}             
})()

The video stream is converted to a canvas and the images are sent to the pre-trained model. The result is displayed only if the model detects it with an accuracy of more than 50 percent.

3. Adding HTML and CSS code

<center>                        
<video id="video" height="80%" width="80%" autoplay muted playsinline></video>                        
<canvas id ="canvas" width="500" height="500" style="display:none;"></canvas>                        
<div  id="status">Object Not Detected</div>                     </center>

You can beautify your application as you want, I just used a simple background and a box with the camera frame in the middle. You can find the full code here.

Thanks for the read!