How To Learn The Tensor Calculus Visually?

A Gentle Introduction to Tensors for Machine Learning with NumPy

Terminal Updated on December 6, 2019

In deep learning information technology is mutual to encounter a lot of discussion effectually tensors as the cornerstone data construction.

Tensor fifty-fifty appears in name of Google's flagship car learning library: "TensorFlow".

Tensors are a blazon of data structure used in linear algebra, and like vectors and matrices, you can calculate arithmetics operations with tensors.

In this tutorial, you will detect what tensors are and how to manipulate them in Python with NumPy

After completing this tutorial, y'all volition know:

• That tensors are a generalization of matrices and are represented using due north-dimensional arrays.
• How to implement element-wise operations with tensors.
• How to perform the tensor product.

Kick-starting time your projection with my new volume Linear Algebra for Machine Learning, including step-by-step tutorials and the Python source code files for all examples.

Let's go started.

• Update Oct/2019: Stock-still typo in the names of array indexes (thanks Henry Chan).

A Gentle Introduction to Tensors for Machine Learning with NumPy
Photograph by Daniel Lombraña González, some rights reserved.

Tutorial Overview

This tutorial is divided into 3 parts; they are:

1. What are Tensors?
2. Tensors in Python
3. Element-Wise Tensor Operations
4. Tensor Product

Demand assist with Linear Algebra for Automobile Learning?

Take my gratis 7-day email crash form at present (with sample code).

Click to sign-upwardly and also get a gratis PDF Ebook version of the course.

What are Tensors?

A tensor is a generalization of vectors and matrices and is easily understood as a multidimensional array.

In the general case, an assortment of numbers arranged on a regular grid with a variable number of axes is known as a tensor.

— Page 33, Deep Learning, 2016.

A vector is a i-dimensional or first order tensor and a matrix is a two-dimensional or second order tensor.

Tensor annotation is much like matrix notation with a capital letter representing a tensor and lowercase letters with subscript integers representing scalar values within the tensor.

t111, t121, t131     t112, t122, t132     t113, t123, t133 T = (t211, t221, t231),  (t212, t222, t232),  (t213, t223, t233)      t311, t321, t331     t312, t322, t332     t313, t323, t333

Many of the operations that can be performed with scalars, vectors, and matrices tin be reformulated to be performed with tensors.

Every bit a tool, tensors and tensor algebra is widely used in the fields of physics and engineering science. Information technology is a term and set of techniques known in machine learning in the training and operation of deep learning models can exist described in terms of tensors.

Tensors in Python

Like vectors and matrices, tensors tin can exist represented in Python using the N-dimensional assortment (ndarray).

A tensor can be divers in-line to the constructor of array() equally a list of lists.

The example beneath defines a 3x3x3 tensor as a NumPy ndarray. Three dimensions is easier to wrap your head effectually. Hither, nosotros showtime define rows, so a list of rows stacked as columns, and so a listing of columns stacked every bit levels in a cube.

# create tensor from numpy import assortment T = assortment ( [ [ [ 1 , two , 3 ] , [ 4 , v , 6 ] , [ 7 , viii , 9 ] ] , [ [ xi , 12 , 13 ] , [ xiv , 15 , 16 ] , [ 17 , 18 , nineteen ] ] , [ [ 21 , 22 , 23 ] , [ 24 , 25 , 26 ] , [ 27 , 28 , 29 ] ] , ] ) print ( T . shape ) impress ( T )

Running the instance first prints the shape of the tensor, then the values of the tensor itself.

You can see that, at least in iii-dimensions, the tensor is printed as a series of matrices, one for each layer. For this 3D tensor, centrality 0 specifies the level, axis 1 specifies the row, and axis 2 specifies the column.

(3, 3, 3) [[[ 1  ii  3]   [ 4  5  half-dozen]   [ 7  eight  9]] [[11 12 13]   [fourteen 15 16]   [17 18 19]] [[21 22 23]   [24 25 26]   [27 28 29]]]

Element-Wise Tensor Operations

Equally with matrices, we tin perform chemical element-wise arithmetics between tensors.

In this section, we will work through the four main arithmetic operations.

Tensor Improver

The element-wise improver of two tensors with the same dimensions results in a new tensor with the same dimensions where each scalar value is the element-wise addition of the scalars in the parent tensors.

a111, a121, a131     a112, a122, a132 A = (a211, a221, a231),  (a112, a122, a132)      b111, b121, b131     b112, b122, b132 B = (b211, b221, b231),  (b112, b122, b132) C = A + B      a111 + b111, a121 + b121, a131 + b131     a112 + b112, a122 + b122, a132 + b132 C = (a211 + b211, a221 + b221, a231 + b231),  (a112 + b112, a122 + b122, a132 + b132)

In NumPy, we can add tensors directly by adding arrays.

# tensor improver from numpy import array A = array ( [ [ [ ane , 2 , 3 ] , [ iv , v , half-dozen ] , [ vii , 8 , 9 ] ] , [ [ 11 , 12 , 13 ] , [ 14 , xv , 16 ] , [ 17 , 18 , xix ] ] , [ [ 21 , 22 , 23 ] , [ 24 , 25 , 26 ] , [ 27 , 28 , 29 ] ] , ] ) B = array ( [ [ [ 1 , 2 , 3 ] , [ 4 , 5 , vi ] , [ 7 , viii , 9 ] ] , [ [ xi , 12 , xiii ] , [ xiv , fifteen , 16 ] , [ 17 , 18 , 19 ] ] , [ [ 21 , 22 , 23 ] , [ 24 , 25 , 26 ] , [ 27 , 28 , 29 ] ] , ] ) C = A + B print ( C )

Running the example prints the addition of the two parent tensors.

[[[ ii  4  6]   [ viii 10 12]   [14 16 eighteen]] [[22 24 26]   [28 30 32]   [34 36 38]] [[42 44 46]   [48 50 52]   [54 56 58]]]

Tensor Subtraction

The element-wise subtraction of 1 tensor from another tensor with the same dimensions results in a new tensor with the same dimensions where each scalar value is the element-wise subtraction of the scalars in the parent tensors.

a111, a121, a131     a112, a122, a132 A = (a211, a221, a231),  (a112, a122, a132)      b111, b121, b131     b112, b122, b132 B = (b211, b221, b231),  (b112, b122, b132) C = A - B      a111 - b111, a121 - b121, a131 - b131     a112 - b112, a122 - b122, a132 - b132 C = (a211 - b211, a221 - b221, a231 - b231),  (a112 - b112, a122 - b122, a132 - b132)

In NumPy, we can decrease tensors direct past subtracting arrays.

# tensor subtraction from numpy import array A = array ( [ [ [ 1 , 2 , 3 ] , [ 4 , 5 , 6 ] , [ 7 , 8 , 9 ] ] , [ [ 11 , 12 , xiii ] , [ fourteen , xv , 16 ] , [ 17 , 18 , 19 ] ] , [ [ 21 , 22 , 23 ] , [ 24 , 25 , 26 ] , [ 27 , 28 , 29 ] ] , ] ) B = array ( [ [ [ 1 , two , 3 ] , [ 4 , 5 , 6 ] , [ 7 , eight , nine ] ] , [ [ 11 , 12 , 13 ] , [ 14 , 15 , 16 ] , [ 17 , 18 , 19 ] ] , [ [ 21 , 22 , 23 ] , [ 24 , 25 , 26 ] , [ 27 , 28 , 29 ] ] , ] ) C = A - B impress ( C )

Running the instance prints the effect of subtracting the starting time tensor from the 2nd.

[[[0 0 0]   [0 0 0]   [0 0 0]] [[0 0 0]   [0 0 0]   [0 0 0]] [[0 0 0]   [0 0 0]   [0 0 0]]]

Tensor Hadamard Product

The element-wise multiplication of one tensor from another tensor with the same dimensions results in a new tensor with the same dimensions where each scalar value is the element-wise multiplication of the scalars in the parent tensors.

Equally with matrices, the operation is referred to equally the Hadamard Production to differentiate it from tensor multiplication. Here, nosotros will use the "o" operator to indicate the Hadamard product operation between tensors.

a111, a121, a131     a112, a122, a132 A = (a211, a221, a231),  (a112, a122, a132)      b111, b121, b131     b112, b122, b132 B = (b211, b221, b231),  (b112, b122, b132) C = A o B      a111 * b111, a121 * b121, a131 * b131     a112 * b112, a122 * b122, a132 * b132 C = (a211 * b211, a221 * b221, a231 * b231),  (a112 * b112, a122 * b122, a132 * b132)

In NumPy, we can multiply tensors directly by multiplying arrays.

# tensor Hadamard production from numpy import array A = array ( [ [ [ one , 2 , 3 ] , [ 4 , five , vi ] , [ 7 , 8 , nine ] ] , [ [ 11 , 12 , thirteen ] , [ 14 , xv , 16 ] , [ 17 , eighteen , 19 ] ] , [ [ 21 , 22 , 23 ] , [ 24 , 25 , 26 ] , [ 27 , 28 , 29 ] ] , ] ) B = array ( [ [ [ i , 2 , 3 ] , [ iv , v , 6 ] , [ 7 , viii , nine ] ] , [ [ 11 , 12 , 13 ] , [ 14 , 15 , xvi ] , [ 17 , xviii , 19 ] ] , [ [ 21 , 22 , 23 ] , [ 24 , 25 , 26 ] , [ 27 , 28 , 29 ] ] , ] ) C = A * B print ( C )

Running the example prints the result of multiplying the tensors.

[[[  1   4   9]   [ 16  25  36]   [ 49  64  81]] [[121 144 169]   [196 225 256]   [289 324 361]] [[441 484 529]   [576 625 676]   [729 784 841]]]

Tensor Partition

The element-wise division of 1 tensor from some other tensor with the aforementioned dimensions results in a new tensor with the same dimensions where each scalar value is the element-wise division of the scalars in the parent tensors.

a111, a121, a131     a112, a122, a132 A = (a211, a221, a231),  (a112, a122, a132)      b111, b121, b131     b112, b122, b132 B = (b211, b221, b231),  (b112, b122, b132) C = A / B      a111 / b111, a121 / b121, a131 / b131     a112 / b112, a122 / b122, a132 / b132 C = (a211 / b211, a221 / b221, a231 / b231),  (a112 / b112, a122 / b122, a132 / b132)

In NumPy, we can divide tensors directly by dividing arrays.

# tensor partition from numpy import assortment A = array ( [ [ [ i , 2 , iii ] , [ four , 5 , 6 ] , [ seven , 8 , ix ] ] , [ [ 11 , 12 , 13 ] , [ 14 , 15 , 16 ] , [ 17 , 18 , 19 ] ] , [ [ 21 , 22 , 23 ] , [ 24 , 25 , 26 ] , [ 27 , 28 , 29 ] ] , ] ) B = array ( [ [ [ 1 , ii , iii ] , [ 4 , v , 6 ] , [ 7 , eight , 9 ] ] , [ [ 11 , 12 , thirteen ] , [ 14 , 15 , 16 ] , [ 17 , 18 , 19 ] ] , [ [ 21 , 22 , 23 ] , [ 24 , 25 , 26 ] , [ 27 , 28 , 29 ] ] , ] ) C = A / B print ( C )

Running the instance prints the event of dividing the tensors.

[[[ 1.  1.  ane.]   [ one.  1.  i.]   [ ane.  one.  ane.]] [[ 1.  1.  1.]   [ ane.  one.  ane.]   [ 1.  1.  1.]] [[ one.  1.  1.]   [ one.  1.  1.]   [ 1.  1.  one.]]]

Tensor Production

The tensor product operator is oft denoted every bit a circle with a small x in the middle. We will denote information technology here as "(10)".

Given a tensor A with q dimensions and tensor B with r dimensions, the product of these tensors will exist a new tensor with the order of q + r or, said another mode, q + r dimensions.

The tensor product is non limited to tensors, but can also be performed on matrices and vectors, which can be a proficient identify to practice in lodge to develop the intuition for higher dimensions.

Allow's take a look at the tensor product for vectors.

a = (a1, a2) b = (b1, b2) c = a (ten) b      a1 * [b1, b2] c = (a2 * [b1, b2])

Or, unrolled:

a1 * b1, a1 * b2 c = (a2 * b1, a2 * b2)

Allow's take a expect at the tensor product for matrices.

a11, a12 A = (a21, a22)      b11, b12 B = (b21, b22) C = A (ten) B             b11, b12          b11, b12      a11 * (b21, b22), a12 * (b21, b22) C = [       b11, b12          b11, b12 ]      a21 * (b21, b22), a22 * (b21, b22)

Or, unrolled:

a11 * b11, a11 * b12, a12 * b11, a12 * b12      a11 * b21, a11 * b22, a12 * b21, a12 * b22 C = (a21 * b11, a21 * b12, a22 * b11, a22 * b12)      a21 * b21, a21 * b22, a22 * b21, a22 * b22

The tensor product can be implemented in NumPy using the tensordot() function.

The part takes as arguments the two tensors to be multiplied and the axis on which to sum the products over, called the sum reduction. To calculate the tensor product, likewise called the tensor dot product in NumPy, the axis must be set to 0.

In the instance beneath, we define two order-1 tensors (vectors) with and calculate the tensor production.

# tensor product from numpy import array from numpy import tensordot A = array ( [ 1 , 2 ] ) B = assortment ( [ three , four ] ) C = tensordot ( A , B , axes = 0 ) print ( C )

Running the instance prints the result of the tensor product.

The event is an society-2 tensor (matrix) with the lengths 2×2.

The tensor product is the most common form of tensor multiplication that yous may encounter, merely in that location are many other types of tensor multiplications that exist, such as the tensor dot product and the tensor contraction.

Extensions

This section lists some ideas for extending the tutorial that y'all may wish to explore.

• Update each example using your ain small contrived tensor data.
• Implement 3 other types of tensor multiplication not covered in this tutorial with pocket-size vector or matrix data.
• Write your own functions to implement each tensor performance.

If you explore any of these extensions, I'd love to know.

Farther Reading

This department provides more resources on the topic if you are looking to go deeper.

Books

• A Student'south Guide to Vectors and Tensors, 2011.
• Chapter 12, Special Topics, Matrix Computations, 2012.
• Tensor Algebra and Tensor Analysis for Engineers, 2015.

API

• The Northward-dimensional assortment
• numpy.tensordot() API

Articles

• Tensor algebra on Wikipedia
• Tensor on Wikipedia
• Tensor product on Wikipedia
• Outer production on Wikipedia

Other

• Fundamental Tensor Operations for Big-Scale Data Analysis in Tensor Train Formats, 2016.
• Tensor product, direct sum, Quantum Mechanics I, 2006.
• Tensorphobia and the Outer Production, 2016.
• The Tensor Product, 2011

Summary

In this tutorial, you discovered what tensors are and how to manipulate them in Python with NumPy.

Specifically, you learned:

• That tensors are a generalization of matrices and are represented using n-dimensional arrays.
• How to implement element-wise operations with tensors.
• How to perform the tensor production.

Practice you have whatsoever questions?
Ask your questions in the comments below and I will do my best to answer.

Go a Handle on Linear Algebra for Car Learning!

Develop a working understand of linear algebra

...by writing lines of lawmaking in python

Discover how in my new Ebook:
Linear Algebra for Motorcar Learning

Information technology provides self-study tutorials on topics like:
Vector Norms, Matrix Multiplication, Tensors, Eigendecomposition, SVD, PCA and much more...

Finally Sympathise the Mathematics of Data

Skip the Academics. Just Results.

See What's Inside

Source: https://machinelearningmastery.com/introduction-to-tensors-for-machine-learning/

Posted by: davisfreples.blogspot.com

Share this post