GLOSS:Geometric Local Self-Similarity Learning for Faithful Reference-guided Texture Fill

• Geometry self-similarity

  Shapes repeat locally, so a region already seen constrains one that has not.

• Local geometry–texture prior

  Conditioned on the surface underneath, so a reference lands aligned to it.

• Shape-specific, small data

  One shape and an off-the-shelf image model — no textured 3D corpus.

A prototype Blender add-on brings reference-guided fill into a real texturing workflow. Five professional 3D artists piloted it; all five said they would integrate interactive reference and geometry-guided fill into their work.

2.1 Diverse reference-based authorship on a single creature

2.2 Artist user study clips

No textured 3D dataset is involved. Off-the-shelf image models generate training data from the shape itself, and the network learns local texturing from that shape's own self-similarity.

3.1 Data and Training

We generate local patch level data pairs (geometry and texture) from ControlNet rendered single view renderings from a single asset. We pass them through batch multi-attention layer for the model to learn faithful texture inpainting.

3.2 Model

Our model supports interactive and automatic texturing. The artist can select any reference and create a “brush”. The model inferences faithful patches on the asset.

3.2.1 Batch multi-attention visualization

Given a target patch, the trained attention layers look at reference patches and learn to synthesize based on geometric similarity.

Rows: albedo, normals, and where attention lands for the target patch.

3.2.2 Geometry conditioned local inpainting

Each fill is conditioned on geometry as well as colour. The reference contributes its albedo and its surface normals; the target contributes the region left to fill and its own normals. The model reads the two together, so the synthesised patch follows the shape it lands on rather than being pasted flat.

Once trained, our model supports versatile applications: blend several references across one surface, carry beyond base colour into full PBR channels, texture a shape it was never trained on, and fill a whole mesh from a single view.

4.1 Blending with two or more references

4.2 Zero-shot PBR

A model trained on texture can be used for PBR generation, padding the metallic and roughness channels with ones as pseudo RGB. The model retains the material–geometry correlation from the source. The following meshes are generated using the automatic texture pipeline.

4.3 Zero-shot transfer to unseen mesh

A model trained on a single shape transfers the learned geometry–texture correlation to unseen shapes. The following meshes are generated using the automatic texture pipeline.

4.4 Automatic texturing

References in, textured out via globally synced progressive inpainting

Comparison against baselines

All seven panels are one video, so every method shown is the same frame of the same turntable.

Per-mesh setting, averaged over 10 shapes. Best in orange.

@inproceedings{cai2026gloss,
  title     = {GLOSS: Geometric Local Self-Similarity Learning for
               Faithful Reference-guided Texture Fill},
  author    = {Cai, Chenyue and Hu, Anita and Lucas, James and
               Rusinkiewicz, Szymon},
  booktitle = {SIGGRAPH Asia 2026 Conference Papers},
  year      = {2026},
  doi       = {10.1145/3829340.3842197}
}