Vectors and Tensors on NEURAL//RUN

02briefing

What it is

A vector is a list of numbers. That is the whole definition. (3, 4) is a vector. So is a list of two hundred numbers describing what you have watched this month. The reason the word exists is that lists of numbers have a geometry: draw (3, 4) as an arrow from the origin and suddenly you can talk about its length and its direction.

Tensor is the general word: a vector is a one-dimensional tensor, a matrix is a two-dimensional one, a colour image is three (height, width, colour channel). The playground stays in three dimensions because you can see it there. Every rule carries.

The dot product

Multiply matching entries, add them up:

a · b = a[0]*b[0] + a[1]*b[1] + a[2]*b[2]

Same direction gives a big positive number. Opposite gives a big negative. At right angles you get exactly zero. Set a to (2, 1, 0) and b to (-1, 2, 0) in the demo and read it: 2 times -1, plus 1 times 2, plus 0. Zero.

Length and scale

Length is the square root of the sum of squares. (3, 4, 0) has length 5. Multiply every entry by k and the length multiplies by k while the direction stays put. Divide a vector by its own length and you get a unit vector, length exactly 1. The Tune mission is that: scale a until it is a unit vector.

In the demo, scaling is done the honest way, as a matrix times a vector with k on the diagonal. That is the same operation a layer of a network performs.

The cross product

Only in three dimensions. Two arrows in, a third out, at right angles to both, with a length equal to the area of the parallelogram they span. Set a along x and b along y and watch a × b stand straight up. It shows up more in graphics and physics than in learning, but it is the reason the playground is 3-D.

Matrices and shapes

A matrix is a grid of numbers. Matrix times vector means: dot each row with the vector, collect the results, one per row. A layer of a neural network is exactly this. The input is the vector, the weights are the matrix, the output goes to the next layer.

Shapes must match. A 2 by 3 matrix needs a vector of length 3 and gives back length 2. The Debug mission has a matVec with its indices swapped. It passes on symmetric square matrices, where the transpose is the same matrix, and breaks on anything else. That is what a bug hiding behind friendly test data looks like.

03live demo

04missions

Predict

Which way does the arrow point?

Read the state, call the outcome before the engine does.

unranked

up to 120 xprun

Tune

Scale it until it fits

Turn the knobs until the loss behaves.

unranked

up to 160 xpclear Which way does the arrow point? first

Debug

The shapes do not match

Something is wrong on purpose. Find it.

unranked

up to 200 xpclear Scale it until it fits first

Code

Write dot()

Write the function. The tests are the judge.

unranked

up to 240 xpclear The shapes do not match first

05in production

Your feed is a vector

A recommendation system turns you into a list of numbers, a few hundred of them, and turns every post into another list of the same length. "Would you like this post" is one dot product: multiply matching entries, add them up, sort by the result. That is the whole ranking step, run billions of times a day. The dot product in this module is that line of code.

Matrices are the layers

A layer of a neural network is a matrix times a vector. The vector is what came in; the matrix is the layer's weights; the product is what goes out to the next layer. Graphics cards exist to do exactly that multiplication very fast, which is why the same chips that render games train models. matVec in the Debug mission is a tiny version of the operation those chips are built for.

06code samples

engine/cpp/vectors.hc++

// Vector and small-matrix arithmetic for the Vectors and Tensors module. Plain loops over
// caller-owned float arrays; internals accumulate in double.
#pragma once

namespace ne {

double vec_dot(const float* a, const float* b, int n);
double vec_norm(const float* a, int n);
/** a x b for 3-vectors, into out3. */
void vec_cross(const float* a, const float* b, float* out3);
/** Angle between a and b in radians, in [0, pi]. NaN when either has zero length. */
double vec_angle(const float* a, const float* b, int n);
/** out = M v, M row-major rows x cols, v of cols, out of rows. */
void mat_vec(const float* m, int rows, int cols, const float* v, float* out);
/** out = A B, A row-major ar x ac, B row-major ac x bc, out ar x bc. */
void mat_mul(const float* a, int ar, int ac, const float* b, int bc, float* out);

}  // namespace ne

07debrief

module tier

unranked

0 xp earned here

Predict	Which way does the arrow point?	unranked
Tune	Scale it until it fits	unranked
Debug	The shapes do not match	unranked
Code	Write dot()	unranked

Run the demo for Bronze. Pass the quiz for Silver.

08share kit

Your feed ranks posts with one operation. It is called a dot product, and you can do it by hand.

Slide 1. A vector is a list of numbers. Draw it and it is an arrow. Your taste, a song, a photo: to a model, all arrows.
Slide 2. Dot product: multiply matching entries, add them up. Same direction gives a big positive number. That number is a score.
Slide 3. Length is the square root of the sum of squares. Scale a vector by k and its length scales by k. Direction stays.
Slide 4. A matrix times a vector is one layer of a neural network. Rows dotted with the input, one output per row.
Slide 5. Shapes have to match. 2 by 3 needs a 3 in and gives a 2 out. Get that wrong and every framework yells at you. Link in bio.

story

Frame 1. Two neon arrows on a grid. Text: same direction or opposite? the dot product knows.
Frame 2. The cross product standing straight up off the plane. Text: the one that only works in 3-D.
Frame 3. The skill tree with Vectors lit. Text: first node. everything else is built on this. NEURAL//RUN.

#machinelearning #linearalgebra #vectors #matrices #learntocode #ai #datascience #deeplearning #cplusplus #webassembly #threejs #developer #mlengineer #neuralnetworks #learnml #techeducation #cyberpunk #interactivelearning

completion card

run the demo to earn a card

+50 credits, once