INTRODUCTION TO TENSORFLOW

10) INTRODUCTION TO TENSORFLOW

Tensorflow is software application popular for implementing machine learning algorithms particularly neural networks, it was developed by Google and it was released as an open source platform in 2015. It’s called as tensorflow because it takes input as multi-dimensional array.

Multidimensional arrays are also known as tensors. You can construct a sort of flowchart of operations that you want to perform on that input. So if input goes in at one end and then it flows through this system of operations and comes out the other end as output and that’s why it’s called tensor flow because the tensor goes in it, flows through list of operations and then it comes out the other side.

Why tensorflow?

Tensorflow is popular because it’s extremely versatile.

– It can be run on many different platforms; you can run it on desktop or on a cloud or you can run it on a mobile device.

– You can train it on multiple machines and then you can run it on different machine once you have the trained model.

– It can also run on GPUs as well as CPUs.

– Now although GPUs were originally designed for video games, they are also very good at matrix operations and algebra, so that makes them very fast for doing these kind of calculations, which rely on lot of the newer algebra.

– Tensorflow is very fast because it’s written in C++ but it can be accessed and controlled by other languages.

– Another great feature of tensorflow is something called tensor board.

Tensorflow is mathematical computation library for training and building your machine learning and deep learning model with a simple to use high level APIs.

If you have a mathematical formula to forecast sales for every zip code in the world or forecast weather for every zip code in the world and you want to basically take the formula and accerlate it so that the final execution is super fast.

– So what you can do is, you can take the formula and you can convert that into a Tensorflow computational graph and then use Tensorflow to execute the formula in specialized hardware accelerators like either GPUs or TPUs. TPUs are basically Google cloud tensor processing units, specialized hardware designed by Google.

Now let’s take an example, as you developed a machine learning model in either a spark or, scikit-learn, or R library, basically you want to take a model and do inference for millions and billions of data points. So according to all this the output of a logistic regression looks as-:

Basically it’s just a mathematical formula like-:

Y = b+x₁.w₁ +x₁.w₂….+ x_n.w_n

Y = B + X*W

Sigmoid = 1/ (1+e-y)

So here weights are multiplied by the variables, do it for all the variables and corresponding coefficient, and add the intercepts to get an output and further use a sigmoid function to basically convert that into probability.

So here a computation graph is created using tensorflow. So, there is an X variable which is input variable matrix and then a weight matrix which are just coefficients for each of the variable. Do a matrix multiplication, then adding the intercepts, and apply a sigmoid function.

Now this computational graph is generated rather than inference on a single data point, make a particular computational graph, schedule it on GPU or schedule it on CPU.

– But is it different using numpy or sigmoid function in numpy. So what to do is take your data and put it in a GPU memory and you can vectorize it, so that you can use all the GPU cores or TPU.

– Not only that, you can broken down your graph into multiple sub-formulas.

If you have 100 variables in the logistic regression, so you can take 50 variables and execute it as one flow and execute it as one flow and take the other 50 variables are executed a second flow each on its own GPUs.

And finally you can add the sum and apply a sigmoid function.

So, basically here formula is breaking down and then executing it on multiple hardware accelerators, so that you can speed up the computation.

1. Think about complex formula where you have lot of equations.
2. So can take a breakable unit of formula and then schedule it on multiple GPUs or TPUs accelerators over here.

So basically you are creating a computational graph and breaking it into chunks and executing each chunk in specialized hardware accelerator and you are vectorizing it to use the cores of that particular hardware accelerator.

Apart from breaking the computational graph, tensorflow also provides an API that can use to build machine learning and deep learning models.

Tensorflow has tensorflow estimators and also Keras is a high level API for building deep learning models. It has interface to develop models and develop a mathematical functions in C++, it has python API and also a java API, that is available. So you can even deploy your trained model in a Java application easily.

It also comes with the auto differentiation function or auto diff.

So auto diff helps to calculate the gradient of a function. When you take a neural network, you have a forward pass and a backward pass and you use your weights in the forward pass and calculate error and then try to update the weights in backward pass and for that you need to calculate derivatives across each and every layer in deep neural network and the auto diff function.

The part calculating the partial derivative gradients is taken care automatically by Tensorflow framework. You can use to calculate, gradient of function for any custom function as well.

It also provides nice visualization tool called tensor board where you can go and debug and analyze your model.

More you can sum up as-: