Learning Manifold-Faithful Representations

Autoencoders learn a code from which data can be reconstructed, but offer no guarantee that the learned latent space is faithful to the data manifold. A small step away from an embedded code leaves the region the decoder was trained on, and the output degrades. We formalize manifold-faithful traversal, the property that a latent code can be perturbed repeatedly, with every intermediate decode remaining a plausible sample, without drift accumulating across steps. This property is distinct from disentanglement, which identifiability results rule out without supervision, and is not implied by reconstruction quality, since the decoder is unconstrained off the measure-zero set of embedded codes. We propose a two-phase training framework whose second phase adds a nearest-neighbor latent projection loss, where a code is perturbed, decoded, re-encoded, and the result is pulled toward the nearest code rather than its own origin. This trains the decode-encode composition to act as a retraction onto the latent manifold. We evaluate on a task we call unsupervised image editing, starting from a given image and traversing the latent space via repeated perturbation and projection with no labels or target attributes. On a synthetic dataset, trajectories stay on-manifold while uncorrected baselines compound drift, and on Colored MNIST, both targeted and undirected walks produce smooth, recognizable transitions throughout.