Einstein Summation Operation

Easy

Imagine you have a big box of Lego blocks. Each Lego block has a number on it, and you want to add up all the numbers on the blocks. But there’s a twist! You can only add two blocks together if they are next to each other. This is like the Einstein Summation Operation, but instead of Lego blocks, we’re talking about numbers in a grid, and instead of adding, we’re multiplying them together.

Now, let’s say you have a grid of numbers like this:

```

1 2 3

4 5 6

7 8 9

```

And you want to multiply the numbers in a special way. You look at the grid and see that the number 1 is next to the number 2, and the number 2 is next to the number 3. So, you multiply them together: 1 2 3.

But wait, there’s more! The number 4 is next to the number 5, and the number 5 is next to the number 6. So, you multiply them together too: 4 5 6.

And finally, the number 7 is next to the number 8, and the number 8 is next to the number 9. So, you multiply them together: 7 8 9.

So, you’ve added up all the numbers in the grid by multiplying them together in a special way, just like the Einstein Summation Operation. It’s like saying, “I’m going to multiply all the numbers that are next to each other in a certain way to get a total.”

And that’s how the Einstein Summation Operation works! It’s a way to do math with numbers in a grid by multiplying them together in a special way.

Another easy example

Alright, imagine you have two lists of numbers, and you want to do something special with them. Einstein Summation Operation is like a way to quickly add up these numbers, but it’s a bit magical because it’s simpler and cooler.

So, let’s say you have two lists:

List 1: [2, 3, 4]

List 2: [5, 6, 7]

Normally, if you want to add them together, you’d do it like this:

(2 + 5) + (3 + 6) + (4 + 7) = 26

But with the Einstein Summation Operation, it’s like you have a special rule. You just write down the lists next to each other, and wherever you see the same letter on top and bottom, you multiply the numbers and then add them all up.

So, for our example:

List 1: [2, 3, 4]

List 2: [5, 6, 7]

We write them like this:

(2 5) + (3 6) + (4 * 7)

Then we just do the multiplication and addition:

(2 * 5) = 10

(3 * 6) = 18

(4 * 7) = 28

Then add them all up:

10 + 18 + 28 = 56

And that’s it! It’s like magic math where you don’t have to write down all the extra plus signs and it makes adding things together really easy and fun!

Moderate

Imagine you have a bunch of legos with different colors. You can build all sorts of things with them, right? Einstein summation is like a shortcut for writing instructions to build things with numbers.

Normally, if you have to add a bunch of legos together, you might write it out like 1 + 2 + 3 + 4. But with Einstein summation, you can just write 1_i, where the little subscript “i” means you secretly add up all the different colored legos (numbers) you have.

Here are the key ideas:

1. Sum over Squiggly Letters: If a letter appears twice in a thingy (like 1_i), it means you secretly add up a bunch of stuff. That letter is like your secret “squiggly” instruction.

2. Regular Letters Stay Unique: Letters that only appear once (like i in 1_i) are like normal instructions, telling you to use a specific number of legos (or a specific number in your problem).

For example, imagine you have legos in 3 colors (i can be 1, 2, or 3). You can write 2_i * 3_i to secretly add up the product (like building a tower) of all the different colored lego pairs you have (2 red with 3 red, 2 red with 3 blue, and so on).

Einstein summation might seem strange at first, but it’s like a secret code that lets mathematicians write instructions for calculations with numbers in a shorter and cleaner way!

Hard

Imagine you have a bunch of boxes, and each box has a number written on it. You want to add up all the numbers on the boxes, but you don’t want to do it the boring way by adding them one by one. So, you come up with a special code to help you add them faster.

The code is called the “Einstein summation operation.” Here’s how it works:

You give each box a special name, like “a” or “b.” Then, you write down the numbers inside the boxes like this: a1, a2, a3, and so on.

Now, here’s the cool part. Instead of adding the numbers one by one, you use a special symbol, a big E, like this: E. This symbol means you want to add up all the numbers inside the boxes.

So, if you have three boxes with numbers 2, 5, and 8, you write:

E(a1 + a2 + a3)

And the answer is the same as if you added them normally: 2 + 5 + 8 = 15.

But the Einstein summation operation is even more powerful. Let’s say you have two sets of boxes, “a” and “b,” and you want to add the numbers inside them together. You can write:

E(a1 b1 + a2 b2 + a3 * b3)

This means you’re adding the numbers inside box a1 and box b1, then box a2 and box b2, and so on.

So, if the numbers inside the boxes are:

a1 = 3, a2 = 6, a3 = 9

b1 = 4, b2 = 2, b3 = 7

Then:

E(a1 b1 + a2 b2 + a3 b3) = 3 4 + 6 2 + 9 7 = 99

Pretty neat, right? The Einstein summation operation is a clever way to add and multiply numbers quickly, especially when you have lots of boxes and numbers to deal with. It’s like a secret code that helps mathematicians and scientists do their work faster!

A few books on deep learning that I am reading:

Book 1

Book 2

Book 3