Quantum Relative Entropy: Why Comparing States Has a Direction

Quantum Relative Entropy: Why Comparing States Has a Direction

Compare two noncommuting qubit states with matrix logarithms. A NumPy example reveals asymmetric relative entropy and shows what happens when both states lose their Z-basis coherence.

Suppose two quantum programs produce density matrices rho and sigma. How different are those states?

There is more than one useful answer. Quantum relative entropy compares them in a way that treats their roles differently. Swapping the arguments can change the result, even for two ordinary mixed qubits.

That directionality is not a numerical bug. It is built into the definition. Let's calculate a small example, then see what happens when the same quantum channel processes both states.

The first state supplies the weighting

For density matrices rho and sigma, define

D(rho || sigma) = Tr[rho * (log2(rho) - log2(sigma))].

Here the logarithms are matrix logarithms, and the multiplication is matrix multiplication. Using base 2 expresses the result in bits.

The first argument appears outside the logarithm, so the expression is not symmetric. For normalized density matrices, D is nonnegative and equals zero exactly when the states are identical. But it is not a mathematical distance metric: symmetry and the triangle inequality do not hold in general.

A helpful starting point is the classical case. If both matrices are diagonal in the same basis, with diagonal probabilities p_j and q_j, the formula reduces to

sum_j p_j * log2(p_j / q_j).

This is the classical Kullback-Leibler divergence. The p distribution weights the discrepancy between the two log probabilities. Reversing p and q changes both the weights and the comparison.

The quantum formula extends this idea to states that need not share an eigenbasis. It does not mean that D itself is the probability of guessing a state correctly.

Two mixed qubits that do not commute

Use these matrices in the computational basis:

rho   = [[0.8, 0.0],
         [0.0, 0.2]]

sigma = [[0.5, 0.1],
         [0.1, 0.5]]

Both are valid full-rank density matrices. Rho has eigenvalues 0.8 and 0.2, with its Bloch vector along positive Z. Sigma has eigenvalues 0.6 and 0.4, with its Bloch vector along positive X.

They do not commute, so there is no common basis in which both are diagonal. Comparing their separately sorted eigenvalue lists with the classical formula would throw away their relative eigenvector orientation. That is not the quantum relative entropy of these matrices.

Take logarithms of eigenvalues, not entries

For a positive-definite Hermitian matrix

A = V * diag(lambda_j) * V†,

its logarithm is

log2(A) = V * diag(log2(lambda_j)) * V†.

NumPy's eigh supplies the eigenvalues and orthonormal eigenvectors. Calling np.log2 directly on the matrix would instead take entrywise logarithms. Our rho even has zero off-diagonal entries, which makes that mistake especially obvious.

The helper below intentionally requires strictly positive eigenvalues. It is a demonstration for full-rank inputs, not a general-purpose implementation that handles every support condition.

Calculate both directions and apply one channel

The channel Phi removes the off-diagonal entries in the Z basis. It describes Z measurement when the outcome is discarded, or complete Z dephasing.

It leaves rho unchanged and maps sigma to I/2. We apply it to both arguments before comparing the results.

import numpy as np
rho = np.diag([0.8, 0.2]).astype(complex)
sigma = np.array([[0.5, 0.1], [0.1, 0.5]], dtype=complex)

def log2pd(a):
    w, v = np.linalg.eigh(a)
    if np.any(w <= 0):
        raise ValueError("log2pd requires strictly positive eigenvalues")
    return (v * np.log2(w)) @ v.conj().T

def D(a, b):
    return float(np.trace(a @ (log2pd(a) - log2pd(b))).real)

def dephase_z(a):
    return np.diag(np.diag(a))

for a in (rho, sigma):
    assert np.allclose(a, a.conj().T, atol=1e-12)
    assert np.isclose(np.trace(a), 1, atol=1e-12)
    assert np.linalg.eigvalsh(a).min() > 0
assert np.linalg.norm(rho @ sigma - sigma @ rho) > 1e-12

r, s = dephase_z(rho), dephase_z(sigma)
assert np.allclose(r, rho, atol=1e-12)
assert np.allclose(s, 0.5 * np.eye(2), atol=1e-12)
for label, a, b in [("D(rho||sigma)", rho, sigma),
                    ("D(sigma||rho)", sigma, rho),
                    ("D(Phi(rho)||Phi(sigma))", r, s),
                    ("D(Phi(sigma)||Phi(rho))", s, r)]:
    print(f"{label} = {D(a, b):.6f} bits")
assert D(r, s) < D(rho, sigma) and D(s, r) < D(sigma, rho)
assert not np.isclose(D(rho, sigma), D(sigma, rho), atol=1e-12)

Expected output:

D(rho||sigma) = 0.307519 bits
D(sigma||rho) = 0.350978 bits
D(Phi(rho)||Phi(sigma)) = 0.278072 bits
D(Phi(sigma)||Phi(rho)) = 0.321928 bits

Both directional comparisons decrease after dephasing. They remain nonzero because the output states are still different: one has populations 0.8 and 0.2, while the other has populations 0.5 and 0.5.

Why processing cannot increase it

The general data-processing inequality says that for any completely positive, trace-preserving quantum channel Phi,

D(Phi(rho) || Phi(sigma)) <= D(rho || sigma).

The same channel must act on both states. Applying different operations to the two inputs is a different question. The inequality also concerns the full channel output, not an arbitrarily selected successful measurement branch.

Our two strict decreases are examples, not a proof of the theorem. In other cases equality holds. In particular, applying the same unitary to both states preserves relative entropy, since it changes their basis without discarding information.

Zero eigenvalues need more care

In finite dimensions, D(rho || sigma) is finite only when the support of rho is contained in the support of sigma. The support is the span of eigenvectors with nonzero eigenvalues. If that containment fails, the relative entropy is positive infinity.

For example,

D(|0><0| || I/2) = 1 bit
D(I/2 || |0><0|) = infinity

The reference state |0><0| assigns zero probability to |1>, while I/2 assigns it nonzero probability. That is a much sharper directional difference than the one in our full-rank example. Our helper rejects these rank-deficient inputs rather than trying to evaluate them incorrectly.

Do not silently replace every zero eigenvalue by a small number and call the result exact. That regularizes the inputs and changes the problem. Relative entropy is not bounded by one bit just because the system is a qubit.

The practical lesson is to keep the argument order, eigenvector basis, and support condition visible in your code. Once those are explicit, the asymmetry stops looking surprising and starts being useful information about the comparison you chose.

Further reading

Mark Wilde's lecture on quantum relative entropy gives the support condition, nonnegativity, and monotonicity under a shared quantum channel.

The quantum-relative-entropy overview connects the definition to classical relative entropy and discusses its mathematical properties.