Beyond Flat Embeddings: Graph and Geometric Representation Learning

The default representation in modern AI is a flat vector, a point in a few hundred or few thousand dimensions of Euclidean space. It's the workhorse behind semantic search, retrieval, and most of what we call "embeddings." For a large class of real problems, it's also the wrong shape.

Most data that matters inside an organization isn't a flat cloud of independent points. It's relational and hierarchical: org charts, codebases, knowledge graphs, supply chains, document trees, dependency networks. Flatten that structure into a vector and you throw away the very thing that made it informative.

The trouble with flat space

Euclidean embeddings have two quiet limitations. The first is that they lose relationships. Two API endpoints might sit near each other in vector space because their text is similar, while the actual dependency connecting them, the thing you care about, stays invisible. The second is that they can't fit hierarchies cleanly, and this is a geometric fact rather than a tuning problem. The number of items in a tree grows exponentially with depth; the volume of Euclidean space grows only polynomially. You simply run out of room to place a hierarchy without distortion.

If your data is a graph, embedding it as a bag of points is lossy compression of the most useful signal you have.

Two directions that fit the data better

Graph neural networks embed nothing in isolation. They learn representations by passing information along the edges that connect items, so a node's representation becomes a function of its neighborhood. A service "knows" what it depends on, a function "knows" its callers, a document "knows" what cites it. For retrieval and reasoning over connected systems, this is a different class of capability than nearest-neighbor search over flat vectors.

Hyperbolic geometry attacks the hierarchy problem head on. Hyperbolic space is, informally, "bigger near the edges." Its volume grows exponentially, which happens to be exactly the growth rate of a tree. Embedding hierarchies in hyperbolic space preserves parent-child structure with far fewer dimensions and far less distortion than Euclidean space allows. For taxonomies, ontologies, and any deeply nested structure, the geometry does work that no amount of extra Euclidean dimensions can.

We measured it

The exponential-versus-polynomial argument sounds abstract, so we put numbers on it. We built a balanced tree (branching 3, depth 5, 364 nodes, 66,066 pairs) and embedded it two ways. The Euclidean side got its best case: classical MDS straight from the exact tree distances, at 2, 10, and 50 dimensions, with no training noise. The hyperbolic side got two dimensions of the Poincare disk and Sarkar's construction, a deterministic placement with no optimization at all. Its core fits in a few lines:

r = np.tanh(TAU / 2)                # hyperbolic circle of radius TAU
for idx, child in enumerate(children[v]):
    ang = incoming + 2 * np.pi * slot(idx) / n
    pos[child] = mobius(pos[v], r * np.exp(1j * ang))

Each embedding was scored on relative distortion of all pairwise distances against the true tree distances:

Two hyperbolic dimensions do what fifty Euclidean dimensions do, and the hyperbolic worst case (0.32) is three times better than Euclidean's at any tested dimension. No learning was involved on the hyperbolic side. The win is purely geometric.

The structural damage of flat space shows up even more sharply in a simple check: siblings sit at tree distance 2 and cousins at tree distance 4, so their embedded distances should differ by a factor of 2. Hyperbolic space gives 1.86. Euclidean 2D gives 4.47, pushing cousins more than twice as far apart as the hierarchy says they are. Retrieval over that geometry doesn't just blur the taxonomy; it systematically exaggerates distinctions near the leaves. Full code and verbatim results are in our repo's experiments directory.

One honest caveat: on a tree this small, Euclidean space does eventually catch up on mean distortion if you throw 25 times the dimensions at it. But trees grow exponentially, and the dimension bill for flat space grows with them. The geometry mismatch never goes away; you just pay for it in coordinates.

Why this is a representation problem, not a model problem

It's tempting to think a big enough model makes representation choices moot, that scale will "figure out" the structure on its own. In practice the opposite holds. Give a model a representation that matches the shape of the data and it gets smaller, faster, more sample-efficient, and easier to interpret. You aren't handicapping the model by encoding structure. You're handing it a head start.

There is an explainability angle too. A graph representation can point to the path of evidence ("this answer depends on these three connected facts") in a way a flat similarity score never can. For enterprise systems that have to justify a recommendation, the representation is the audit trail.

How we use it

Representation and geometric learning is one of ArthaVortex's core research directions, and it isn't so much a separate track from the rest of our work as the substrate under it. World models learn predictive representations of dynamics. Retrieval gets sharper when it respects the graph structure of the knowledge it searches. Agents reason better over a knowledge graph than over a pile of vectors.

The lesson we keep relearning: before reaching for a bigger model, ask whether the data is represented in a space that fits its shape. Often that's the whole game.

Related: World Models on predictive representations, and Beyond the Language Model on why systems are eating scale.