Tensor: the shape is part of the meaning

A tensor in machine learning is a multidimensional array. Its shape identifies axes such as batch, sequence position and feature width. The same numbers arranged along different axes can describe a different computation.

For example, a language-model activation can have shape [batch, tokens, width]. A shared linear layer transforms the final axis at every batch and token position.

One projection, every position

This NumPy example applies one weight matrix and bias vector to all six token vectors.

import numpy as np

rng = np.random.default_rng(7)
hidden = rng.normal(size=(2, 3, 4))
weights = rng.normal(size=(4, 5))
bias = rng.normal(size=5)
projected = hidden @ weights + bias

The output shape is [2, 3, 5]: batch and token counts stay fixed, while feature width changes from four to five. Broadcasting repeats the bias logically without requiring six separately authored copies.

Broadcasting can also silently permit the wrong operation when unrelated axes happen to have matching sizes. Name the axes before reshaping, transposing or reducing a tensor. A successful shape check establishes compatibility, not that the intended computation happened.