SGSST: Scaling Gaussian Splatting Style Transfer

Bruno Galerne1,2, Jianling Wang1, Lara Raad3, Jean-Michel Morel4,
1Institut Denis Poisson, Université d'Orléans, Université de Tours, CNRS
2Institut Universitaire de France (IUF) 3Instituto de Ingeniería Eléctrica, Facultad de Ingeniería, Universidad de la República 4City University of Hong Kong
Project Paper arXiv Code Gallery of our results Comparison evaluations

Teaser of style transfering videos from our method

Abstract

Applying style transfer to a full 3D environment is a chal- lenging task that has seen many developments since the advent of neural rendering. 3D Gaussian splatting (3DGS) has recently pushed further many limits of neural render- ing in terms of training speed and reconstruction quality. This work introduces SGSST: Scaling Gaussian Splatting Style Transfer, an optimization-based method to apply style transfer to pretrained 3DGS scenes. We demonstrate that a new multiscale loss based on global neural statistics, that we name SOS for Simultaneously Optimized Scales, enables style transfer to ultra-high resolution 3D scenes. Not only SGSST pioneers 3D scene style transfer at such high image resolutions, it also produces superior visual quality as assessed by thorough qualitative, quantitative and perceptual comparisons.

Interpolate start reference image.

BibTeX

@article{sgsst_arXiv2024,
  author    = {Bruno Galerne, Jianling Wang, Lara Raad, Jean-Michel Morel},
  title     = {SGSST: Scaling Gaussian Splatting Style Transfer},
  journal   = {arXiv},
  year      = {2024},
}