Representation decomposition

A vision model looking at a scene encodes two different kinds of knowledge in the same embedding: what each object is, and how the objects relate to each other in space. In DINOv3, these are entangled — you can read both out of the features, but you can't easily inspect or control either one alone.

My first-author work decomposes DINOv3 representations into these two components. Using PCA, LDA, and learned linear projections, I separate an object-identity subspace from a spatial-relationship subspace — then verify the split causally: intervening on one component steers the model's behavior along that axis while leaving the other intact.

The field on the home page is a toy version of the idea. Mixed, the cloud tells you nothing. Projected, identity lives in the clusters and relational structure lives in the edges between them.

Method & evidence

• Decomposition PCA, LDA, and learned linear projections split patch embeddings into an identity subspace and a relation subspace.
• Scale Controlled experiments across 2,000+ synthetic images and 400 real-world objects spanning eight spatial relationships.
• Transfer 97.9% cross-domain transfer — relational structure learned on synthetic scenes holds on real objects.
• Causal check Interventions manipulate internal representations to test whether relational knowledge steers predictably, while ruling out low-level visual confounds.

First-author manuscript in preparation · targeting ICLR 2027