About TensorflowJS

This post describes the motivations and reasoning behind yet another machine learning library called TensorflowJS and how it's unique. If you're interested in more technical aspects of Machine learning check out my machine learning projects series here.✨✨Since last December I started interning at Outreachy for the Tensorflow community on the TensorFlow.js project and it has been a wonderful month since that. In this post, I want to introduce you to Tensorflow JS and my internship work.

Traditional ML pipelines heavily rely on high-performance hardware eg TPU, GPU as the standard environment for doing ML tasks. Other than that, we can also scale ML solutions to large-scale using platforms like GCP, AWS, and Microsoft Azure. Due to these heavy prerequisites, it poses a very high barrier to entering into ML. As a result, people from different backgrounds find it quite difficult to use and apply ML-based solutions to solve problems. If you have been infected by the Machine Learning wave you would probably know that the Tensorflow python package created by Google is the cool guy of deep learning in this industry these days but it's limited to users with experience in the python ecosystem.

One of the main goals of Tensorflow JS is to reduce this barrier to entry in ML for people of different backgrounds. TensorflowJS has no advanced software or hardware requirements to get started with. At the time of writing this post, TensorflowJS supports 13 pre-trained models that you can add water and use, check them out here. To add it to your project all you need to do is grab some cdns via script tags of your favorite model and start using it. This makes it unusually easy and fast to get up and running. The other advantage of using pre-made models is that you don't need to worry about training and validating models this is already done for you! Even if you do need something like that the models are highly customizable to fit your needs as well.

Demos to get you pumped up

In order to get you more excited here are some really cool demos by the community to get more inspiration

Object detection

A simple demo showcasing effective and accurate multiple object detection entirely on the browser.

Figma Plugin

My particular favorites are Charlie Gerard's projects. They are really cool and fun experiments with TensorflowJS.

She made this Figma plugin to manipulate UI in Figma using hand gestures utilizing handpose model of TensorFlow js.

Check out the project here

A squid game app with TensorflowJS

I have not watched Squid game but this game is inspired by the popular tv show. It detects body movements utilizing Posenet model of Tensorflow Js.

Check out the project here

You can find a lot more on Twitter here #MadeWithTFJS

Cool libraries built on top of TensorflowJS

Since the launch of TensorflowJS, the amazing open source community has utilized it to build libraries to cater to your specific needs here are some that I found interesting in no particular order.

Ml5 JS - it's a friendly high-level interface to TensorFlowJs that you can use, to get started with ML on the web. I loved their easy-to-follow docs which is well organized and are oriented towards people with no prior experience with ML.

Handtrack Js - Handtrack.js is a library for prototyping real-time hand detection (bounding box), directly in the browser. This project specifically solves your needs to solve hand tracking-based problems in the browser, lots of useful demos are already built using this: game controls, detect face touching to minimize covid spread, air guitar are some of the ones mentioned on their repo.

Magenta Js - JavaScript API for doing inference with Magenta models, you can work with music, image, and sketch using the variations of this package.

Danfo Js- this library is not built on top of TensorFlow Js perse but is a great companion library for data science-related tasks. If you love using pandas for data manipulation, then you'll be a fan of the Js version heavily inspired by pandas. It provides high-performance, intuitive, and easy-to-use data structures for manipulating and processing structured data on the browser.

And many more!

I get to play a Developer relations Engineer for TensorflowJs with Outreachy

For my internship project, I need to build reusable technical demo/(s) of TensorflowJS which needs to be

• easy to understand
• easily reproducible by anyone
• shows some interesting use cases of a particular TensorflowJs model

and most importantly

• solves a real-world problem

The nature of this project is such that it has an interesting mixture of research and implementation. Through this project, I'm getting a bird's eye view of a day in the life of a developer relations engineer at Google and that is something I look forward to experiencing every day after I wake up in the morning. The idea of using ML in the browser to solve problems is very new to the world which is why developer relations engineers play a big role to introduce and teach people about TensorflowJS in an easy and accessible way. To me, it feels like going on an adventure to discover some kind of treasure every day when I sit for work. Also since I already had some experience using Tensorflow with python, a lot of the library-specific concepts seemed familiar to me but I was amazed at the ease of getting started with the Javascript version.

In the past month alone, I learned about validating an idea, scoping a project from scratch, and drafting a timeline to complete the project. I had a lot of fun in the initial research phase in which I had to find out an industry to explore a practical use-case of Machine Learning that can be demonstrated using TensorflowJS. The process was both insightful and humbling and just to give you a bit of a spoiler we will be demonstrating an application in the human-computer interaction field. I plan to write a detailed blog post about my project once I'm halfway done, so stay tuned 😁. To sum up, I'm very grateful to play a small part to teach the world and myself included about Machine learning on the web and making ML solutions accessible to everyone.