Representations of Finite Groups

These notes are being transcribed, condensed, and illustrated. They may lag or omit details, please refer to the handwritten notes for the full lecture.

This note covers the properties of group representations for finite groups. The central results are Schur's Lemma and the Wonderful Orthogonality Theorem, which together tell us that groups divide vector spaces into non-interacting subspaces called irreducible representations (irreps), and that we can derive these irreps using orthogonality relations.

1) Recap: Groups vs. Group Representations

Let's quickly distinguish between a group and its representations (from Lectures 1 & 2):

• Symmetry operations/transformations \longrightarrow elements of a group (g \in G)
• How we "write down" symmetry operations and the objects they act on \longrightarrow representations of a group (e.g. matrices & vector spaces they act on)

For the groups we cover in this class, we can always find a representation of the group that consists of matrices acting on a vector space:

The group representation D(g) acts on vector space V. Confusingly, we often refer to both the matrix representation and the vector space as "the group representation."

Representations depend on the vector space they act on. For example, P(3) can act on different spaces:

• P(3) acting on scalars: e.g. f(x_1 + x_2 + x_3) — D(g) = 1 for all g \in P(3).
• P(3) acting on ordered sets (x_1, x_2, x_3) — D(g) = permutation matrices.
• P(3) acting on 2D coordinates (x, y) of a triangle — D(g) = representation in Dresselhaus Table 1.1.

2) Direct Sum of Representations

Suppose we have two representations D(g) acting on V and D'(g) acting on V'. We can combine their vector spaces via the direct sum (D(g), V) \oplus (D'(g), V') to get a new representation acting on a larger space. In this combined representation, the matrices are block diagonal:

We could also apply a similarity transform to "mix up" these blocks, so that the block diagonal structure is hidden. The result is still a valid representation, but it no longer looks block diagonal — this is a reducible representation.

3) Reducible and Irreducible Representations

3.1) Reducible Representations

Suppose W \subseteq V is a subspace such that \forall w \in W and \forall g \in G, D(g) w \in W — that is, the group action keeps W within itself. Then the representation is reducible.

Alternatively, a representation is reducible if there exists a unitary matrix U such that U \, D(g) \, U^{-1} is block diagonal for all g \in G.

3.2) Irreducible Representations (Irreps)

A representation is irreducible (an irrep) if there is no such U — the representation cannot be further decomposed into block diagonal form. Irreps are the fundamental building blocks of representation theory.

3.3) Irreps of P(3)

P(3) has three inequivalent irreps, labeled \Gamma_1, \Gamma_1', and \Gamma_2:

open graphic ↗

These can be stacked to form a 4-dimensional space: \Gamma_R = \Gamma_1 \oplus \Gamma_1' \oplus \Gamma_2.

Note that the 1D irreps are unique — there is only one 1 \times 1 matrix that satisfies the group multiplication rules for each. But the 2D irrep \Gamma_2 is not unique: there are infinitely many equivalent choices related by similarity transforms U \, D^{(\Gamma_2)}(g) \, U^{-1}. The specific matrix entries you see above depend on the choice of basis. Any invertible 2 \times 2 matrix U gives an equally valid \Gamma_2.

In the exercises, you will calculate these irreps using the regular representation (more about this below).

You can also find all irreps computationally and verify their properties in the companion notebook:

# Find all irreps of P(3) from the multiplication table
irreps = rep.infer_irreps(table)
# → 3 irreps with dimensions 1, 1, 2

# Verify each is a valid irreducible representation
rep.is_an_irrep(table, irreps[0])  # → True

# Apply a similarity transform — still the same irrep
gamma_2_transformed = rep.similarity_transform(gamma_2, U)
rep.are_isomorphic(gamma_2, gamma_2_transformed)  # → True

4) Why Do These Results Matter?

Before we dive into the most important results in group representation theory — Schur's Lemma and the Wonderful Orthogonality Theorem — it's worth understanding why these theorems are so useful.

When we get to figuring out how to map common neural network operators to their symmetry-equivalent counterparts, these theorems will make it very obvious what we can and can't do:

• Linear layers have to obey Schur's Lemma! This tells us exactly what structure a "symmetry-respecting" linear map must have.
• These theorems show us how to enumerate all possible representations (learned or otherwise) our group can act on.
• No matter what representation your neural network layers use, if it's done via a symmetry-equivariant method, we know a lot about its properties.

5) Unitarity of Representations

Theorem (Dresselhaus 2.4): Every representation with matrices having non-vanishing determinant can be brought into unitary form by a similarity transform.

For unitary U: U^\dagger U = U U^\dagger = I.

We will use this fact freely — from now on, we'll assume our representations have been brought into unitary form. This is a very useful simplification, because unitary matrices have nice properties (e.g. D(g)^{-1} = D(g)^\dagger).

6) Schur's Lemma (Part 1)

Schur's Lemma is one of the most important results in representation theory. It has big consequences for what a "linear" layer looks like when it must respect a group symmetry.

6.1) Statement

Lemma (Dresselhaus 2.5): Let M be a matrix that commutes with all matrices of a representation — that is, M A_x = A_x M for all group elements x. Then:

1. If the representation is irreducible (an irrep), then M must be a constant matrix: M = c \, \mathbb{I} (a scalar multiple of the identity). This is the only solution — the space of commuting matrices is 1-dimensional.
2. If the representation is reducible, then constant matrices still commute, but there are additional solutions. In a block-diagonal basis where the representation decomposes as \bigoplus_j D_j, M can be a different scalar on each irrep block: M = \bigoplus_j c_j \, \mathbb{I}_{d_j}. If the same irrep appears n times, there is further freedom: those n copies can mix via an n \times n matrix of scalars. The solution space grows as \sum_j n_j^2 where n_j is the multiplicity of each irrep.

In terms of infer_change_of_basis: calling infer_change_of_basis(irrep, irrep) (same irrep with itself) returns exactly 1 solution (the identity matrix, up to scaling). Calling it on a reducible representation returns multiple solutions — one for each independent scalar degree of freedom.

6.2) Proof Sketch

1. Decompose M into Hermitian parts: Write M = H_1 - i H_2 where H_1 = M + M^\dagger and H_2 = i(M - M^\dagger). Both H_1 and H_2 are Hermitian, and they also commute with all A_x.

2. Hermitian matrices have real eigenvalues and are diagonalizable: Since H_j A_x = A_x H_j, we can diagonalize H_j as d = U^\dagger H_j U. Then \hat{A}_x = U^\dagger A_x U satisfies d \hat{A}_x = \hat{A}_x d.

3. Apply the diagonal constraint: Looking at the (i,j) entry: d_{ii} (\hat{A}_x)_{ij} = (\hat{A}_x)_{ij} d_{jj}, which gives (\hat{A}_x)_{ij} (d_{ii} - d_{jj}) = 0.

4. Two cases at each (i,j):

   • Either d_{ii} = d_{jj} (diagonal entries are equal — M is a constant on this block), or
   • (\hat{A}_x)_{ij} = 0 for all A_x (meaning the representation is block diagonal, i.e. reducible).

Example: Consider the \Gamma_1' \oplus \Gamma_2 representation of P(3) acting on \mathbb{R}^3. For element D (rotation by 120°):

The diagonal matrix M = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 2 & 0 \\ 0 & 0 & 2 \end{pmatrix} commutes with all D^{(\Gamma_1' \oplus \Gamma_2)}(g). Since d_{11} = 1 \neq 2 = d_{22} = d_{33}, step 4 forces (D(g))_{12} = (D(g))_{21} = (D(g))_{13} = (D(g))_{31} = 0 for all g, revealing the block structure: a 1 \times 1 block (\Gamma_1') and a 2 \times 2 block (\Gamma_2). Within each block, M is a scalar times the identity (1 \cdot \mathbb{I}_1 and 2 \cdot \mathbb{I}_2), consistent with each block being an irrep.

7) Schur's Lemma (Part 2)

Part 2 extends Schur's Lemma to relate two different irreps. This tells us that distinct irreps cannot be connected by a linear map.

7.1) Statement

Lemma (Dresselhaus 2.6): If \{D^{(1)}(A_x), \forall g \in G\} and \{D^{(2)}(A_x), \forall g \in G\} are two irreps of a group G with dimensionality \ell_1 and \ell_2 respectively, and if there exists a matrix M such that

then:

• If \ell_1 \neq \ell_2: M = \emptyset (the null matrix).
• If \ell_1 = \ell_2: either M = \emptyset, or D^{(1)}(A_x) and D^{(2)}(A_x) differ by an equivalence (similarity) transform — they are the same irrep in different bases.

7.2) Proof Sketch

Assume the representations have been made unitary, and assume \ell_1 \leq \ell_2.

Starting from M \, D^{(1)}(A_x) = D^{(2)}(A_x) \, M, take the adjoint:

By unitarity, [D(A_x)]^\dagger = [D(A_x)]^{-1} = D(A_x^{-1}), so:

Now multiply the adjoint equation (with inverses) by M on the left: M \, D^{(1)}(A_x^{-1}) \, M^\dagger = M \, M^\dagger \, D^{(2)}(A_x^{-1}). Using the original equation to replace M \, D^{(1)}(A_x^{-1}) = D^{(2)}(A_x^{-1}) \, M, we get D^{(2)}(A_x^{-1}) \, M M^\dagger = M M^\dagger \, D^{(2)}(A_x^{-1}) — so M M^\dagger commutes with D^{(2)}.

Similarly, multiply the original equation by M^\dagger on the right: M \, D^{(1)}(A_x) \, M^\dagger = D^{(2)}(A_x) \, M M^\dagger. Using the adjoint equation to replace D^{(1)}(A_x) \, M^\dagger = M^\dagger \, D^{(2)}(A_x) shows that M^\dagger M commutes with D^{(1)}.

By Schur's Lemma Part 1 (since these are irreps), both must be constant matrices: M M^\dagger = c \, \mathbb{I}.

Case \ell_1 = \ell_2: M is square.

• If c \neq 0: M^{-1} = M^\dagger / c, so D^{(1)}(A_x) = M^{-1} D^{(2)}(A_x) M — the two representations differ by a similarity transform (same irrep).
• If c = 0: then M M^\dagger = 0, which means \sum_k |M_{ik}|^2 = 0 for all i, so M is the null matrix.

Case \ell_1 \neq \ell_2 (take \ell_1 < \ell_2): Pad M with zeros to make it square (\ell_2 \times \ell_2) \to N. The same argument gives N N^\dagger = c \, \mathbb{I}, but since N has columns of zeros, the constant c must be zero. Therefore N = \emptyset and M = \emptyset.

See both parts of Schur's Lemma in action in this Change-of-Basis Demo Colab, which uses infer_change_of_basis to test irreducibility and identify which irreps live inside the 3D permutation representation of P(3).

8) Application: Equivariant Linear Maps

The simplest type of neural network is a dense network (or multilayer perceptron), which is built from linear layers composed with activation functions. For such a network to be equivariant with respect to a group action — meaning the group action commutes with the network — two things must hold:

1. The group action must commute with the linear layers.
2. The group action must commute with the activation functions.

Schur's Lemma tells us exactly how linear layers must be constructed to commute with the group action. How to have group action commute with activation functions is a separate topic that we leave for future discussion.

8.1) Constraining W with Schur's Lemma

Suppose we have a linear layer \mathbf{y} = W \mathbf{x}, where the input space X transforms under D_X(g) and the output space Y transforms under D_Y(g). For this layer to be equivariant, we need:

If both D_X(g) and D_Y(g) are decomposed into direct sums of irreps, Schur's Lemma tells us exactly what W can look like. By Part 2, blocks connecting different irrep types must be zero. By Part 1, blocks connecting the same irrep type must be a scalar times the identity.

Use the interactive graphic below to explore this. Choose the number of copies of each P(3) irrep in the input and output, and see how Schur's Lemma constrains W:

open graphic ↗

For example, with input \Gamma_1 \oplus \Gamma_1' \oplus \Gamma_2 (dim 4) and output \Gamma_1 \oplus \Gamma_1 \oplus \Gamma_1' \oplus \Gamma_2 (dim 5), out of the 20 entries in the 5 × 4 weight matrix, only 4 free parameters remain: a, b (two paths from \Gamma_1 to the two output \Gamma_1 copies), c (for \Gamma_1'), and d (a single scalar times I_2 for \Gamma_2).

9) The Wonderful Orthogonality Theorem

The Wonderful Orthogonality Theorem (WOT) is the culmination of the results above. It establishes that the matrix entries of inequivalent irreps satisfy a precise orthogonality relation.

9.1) Statement

Theorem (Dresselhaus 2.7): The following orthonormality relation is obeyed for all inequivalent irreps of a group:

where R sums over all group elements, \Gamma_j and \Gamma_{j'} are irreps, \ell_j and \ell_{j'} are their dimensions, and h = |G| is the number of group elements.

If the representations are unitary (which we can always arrange), this simplifies to:

9.2) Example: \Gamma_1 and \Gamma_1' of P(3)

For the two 1D irreps \Gamma_1 (all ones) and \Gamma_1' (mirrors get -1, rotations get +1):

As expected — different irreps are orthogonal!

For \Gamma_1 with itself: the sum gives 1 + 1 + 1 + 1 + 1 + 1 = 6 = h/\ell_j = 6/1.

9.3) Proof Sketch

Consider the \ell_{j'} \times \ell_j matrix:

where X is an arbitrary \ell_{j'} \times \ell_j matrix.

Multiply by D^{(\Gamma_{j'})}(S) on the left:

By the rearrangement theorem (SR runs over all group elements as R does), this equals M \, D^{(\Gamma_j)}(S). This looks like Schur's Lemma!

Case 1 (\Gamma_j \neq \Gamma_{j'} or \ell_j \neq \ell_{j'}, irreps are inequivalent): By Schur's Lemma Part 2, M = 0 for any choice of X. Choosing X to be 1 at position (\nu', \nu) and 0 elsewhere gives 0 = \sum_R D^{(\Gamma_{j'})}_{\mu'\nu'}(R) \, D^{(\Gamma_j)}_{\nu\mu}(R^{-1}).

Case 2 (\Gamma_j = \Gamma_{j'}, same irrep): Schur's Lemma Part 1 tells us M = c \, \mathbb{I}. Again choosing X to have a single nonzero entry and evaluating the trace gives c = \frac{h}{\ell_j} \delta_{\nu'\nu}, leading to:

Combining both cases yields the full theorem.

10) Representations as Vectors

There is a beautiful way to think about the WOT geometrically (Dresselhaus 2.8). We can think of the representation matrix entries D^{(\Gamma_j)}_{\mu\nu}(R) as components of vectors in an h-dimensional space (one dimension per group element):

Then the Wonderful Orthogonality Theorem is simply saying that the inner product of these vectors is:

The vectors indexed by \Gamma_j, \mu, \nu are orthogonal! But the maximum number of orthogonal vectors in an h-dimensional space is h. How many such vectors V^{(\Gamma_j)}_{\mu,\nu} can we make? For a rep of dimension \ell_j, there are \ell_j choices for \mu and \nu, giving \ell_j^2 entries. Summing over all irreps:

(It can't exceed h because that would violate orthogonality.) We will later see (Dresselhaus 3.7) that in fact \sum_j \ell_j^2 = h — these vectors form a complete basis for the h-dimensional space.

Use the interactive graphic below to explore this for P(3). Select two vectors and verify the WOT: the inner product is h/\ell_j when both vectors come from the same irrep with the same indices, and 0 otherwise.

open graphic ↗

11) The Regular Representation

The regular representation is a particular representation that is useful for finding all irreps of a (finite) group. We build it directly from the multiplication table.

11.1) From Multiplication Table to Matrices

Recall the multiplication table for P(3) from the introduction. The entry at row g and column h gives the product gh (apply h first, then g).

In the regular representation, each group element becomes a basis vector: \{|E\rangle, |A\rangle, |B\rangle, |C\rangle, |D\rangle, |F\rangle\}. The matrix D^{\text{reg}}(g) acts by left multiplication:

So each row of the multiplication table directly encodes a regular representation matrix. Row g tells us where each basis vector |h\rangle maps to under the action of g. The matrix D^{\text{reg}}(g) is the permutation matrix corresponding to the permutation in row g. Each regular representation matrix has shape |G| \times |G| (here 6 \times 6).

Select an element below to see how its row in the multiplication table becomes a permutation matrix. Hover over cells in the highlighted row to see the correspondence.

The regular representation contains all irreps of the group. In fact, as we will prove in the next chapter, each irrep \Gamma_j appears exactly \ell_j times in the decomposition. This is why \sum_j \ell_j^2 = h — the regular representation has dimension h, and it decomposes as \bigoplus_j \ell_j \cdot \Gamma_j, so the total dimension is \sum_j \ell_j \cdot \ell_j = h.

You can verify this in the companion notebook:

# Build and decompose the regular representation
reg = rep.regular_representation(table)  # shape [6, 6, 6]
sub_irreps = rep.decompose_rep_into_irreps(reg)
# → each Γ_j appears ℓ_j times

Related exercises: Representations Exercises