3D Generation

3D Gaussian Splatting No Longer Needs an Nvidia Card: This Trainer Finishes the Whole Job on a Phone

November 20, 20260 views阅读中文原文
3D Gaussian Splatting No Longer Needs an Nvidia Card: This Trainer Finishes the Whole Job on a Phone

TL;DR

Here is the short version: 3D Gaussian Splatting (3DGS) training has long assumed you shoot a scene, ship the frames back to a CUDA workstation, and let an Nvidia GPU do the work. Cross3DGS, posted to arXiv on 17 November 2026, removes that assumption. One Vulkan compute core now completes standard-quality training — 30,000 updates, one million Gaussians — on eight different GPU configurations spanning six vendors, including two Android phones and an iPhone. Final image quality lands within 0.019 dB (bicycle) and 0.063 dB (garden) across all of them. The authors also shipped complete Android and iOS apps, so capture, sparse reconstruction, training and free-view browsing all happen on the handset.

Put plainly: the workflow that used to require an RTX card now runs end to end on a phone. It is just slower.

  • Paper: Cross3DGS: Unified Vulkan-Based 3D Gaussian Splatting Training from Workstations to Mobile Devices
  • Authors: Hiroyuki Sato (MinD in a Device), Yusuke Takimoto
  • Venue: WACV 2027 Workshop on Eyes of the Future (EOF)
  • Code: https://github.com/unclearness/cross3dgs (AGPL-3.0-or-later, with a commercial license option)

What it actually is

Three pieces.

One: a trainer with no CUDA in it. C++20 on Vulkan 1.1 / SPIR-V 1.3, building on Windows, Linux, macOS (through MoltenVK), Android and iOS. GPU coverage runs across Nvidia, AMD, Intel, Arm Mali, Qualcomm Adreno and Apple silicon — six vendors, eight configurations: RTX PRO 6000, Iris Xe, UHD730, RX 9070 XT, Mali G1-Ultra, Adreno 740, M5 Pro and A14.

Two: a mechanism that picks an execution plan per device. The authors break training into 256 schedules, then apply 13 feasibility rules that read device limits, workload size and known driver failures to eliminate schedules that will not run correctly. A fixed preference order picks one of what remains. The critical constraint: every schedule has to compute a numerically identical optimizer update. The paper checks this against same-GPU references and a CPU double-precision oracle — maximum gradient difference 3.4e-8, maximum Adam update difference 1.5e-6, with nothing outside tolerance.

Three: an app you can install on a phone. The Android build records with ARCore, feeds the metric camera poses and intrinsics to an Android port of CPU COLMAP as priors, then trains and opens a viewer. iOS uses ARKit and shares the rest. Nothing about your living room has to leave the device.

Why it matters

3DGS has become the most practical route in 3D reconstruction: walk around a space, shoot photos, get a scene you can explore in real time. But the training code almost universally lives on CUDA. gsplat, LichtFeld Studio and Postshot all require Nvidia hardware. That produces two concrete problems:

  • A hardware gate. Anything captured on a phone, tablet or headset has to be transferred in bulk to a workstation.
  • Privacy and offline operation. Captures of interiors and private spaces are forced off the device.

Cross3DGS does not claim a phone beats a workstation. It removes the requirement that the data leave the device at all. The paper frames the target workload explicitly as batch work after a capture — the hours while the phone is charging.

The numbers

| Item | Result |

|---|---|

| Standard-quality training time span | 115 s on a PRO 6000; 2.36–3.23 h on the two Android devices and the iPhone; 3.44 h on the UHD730 integrated GPU |

| Cross-device quality gap | PSNR span of 0.019 dB on bicycle, 0.063 dB on garden |

| Against VkSplat | Higher PSNR on 9 of 11 scenes, +0.074 dB on average; time difference of 4.4 s with an interval that includes zero |

| Against Brush | 0.041 dB higher PSNR (interval includes zero), while Brush reports roughly 3x the training time |

| Phone energy (25.5k of 30k updates, screen on) | Adreno 740: 16.9 Wh (6.0 W); G1-Ultra: 7.2 Wh (3.0 W); A14: about 9.2 Wh |

| Phone app, capture to viewable model | 28–46 min at the demonstration preset; SfM takes 19–30 min of that, 3DGS only 7–18 min |

Two comparisons show why "portable" is not just a recompile. VkSplat needs subgroups of at least 32 lanes and has no Android or iOS host. Brush exceeds the 128 MiB storage-buffer limit of the Adreno 740 at one million Gaussians. And on the UHD730, VkSplat hangs the GPU within its first 200 updates.

How to use it

Desktop (Windows, Linux, macOS):

  1. Install the Vulkan SDK, a C++20 compiler, CMake 3.28 or newer, and Ninja.
  2. Clone the repo and build per the README. Running configs/standard_quality.json reproduces the paper's standard-quality setting.
  3. Every run records the chosen schedule in bench.json — that is your first debugging stop.

Android:

  1. Download the Mip-NeRF 360 dataset (the app bundles seven scenes).
  2. Build the APK with gradlew :app:assembleRelease and install via adb install.
  3. Reproduce the paper's figures with scripts/benchmark_android.py --config configs/standard_quality.json --scene bicycle --evaluate.

iOS follows the same shape through Xcode, recording with ARKit. If you would rather skip the toolchain, just use the app to capture.

Who it is for

  • Interior and private-space capture, where keeping data on the device is the whole point.
  • Teams without Nvidia hardware: AMD cards, Macs, Intel integrated graphics and Android devices run the same code.
  • Mobile product prototypes: the app is a working reference for AR capture plus on-device SfM plus on-device training plus free-view rendering.
  • Offline field work: sites, factories and surveys without reliable uplink bandwidth.

When it is not a good fit

  • You want speed. Standard quality on a phone is measured in hours (3.7–4.4 h to fill the one-million-Gaussian budget). The 115-second figure is a PRO 6000 result; do not benchmark a phone against it.
  • You need closed-source commercial integration. The code is AGPL-3.0-or-later with a separate commercial license. Read the terms before embedding it.
  • Headsets and smart glasses. The paper explicitly defers less capable devices to future work; none were validated.
  • Cross-device hardware comparisons. The phone scenes in the paper were each captured on their own device, and the authors state those columns are not device comparisons.

Pitfalls

Pitfall one: the fixed preference order is not free. The second selection step never times schedules on the device; the order was set offline. On the held-out M5 Pro and A14 it still picks the fastest feasible method. But on the development machines RX 9070 XT and UHD730, its choice was 2.94x and 4.17x slower than the fastest feasible option — which is why AMD and Intel each get their own order. After a device or driver change, do not assume it will pick the optimum on its own.

Pitfall two: on a phone, the real time sink is SfM, not 3DGS. At the demonstration preset, SfM takes 19–30 minutes while 3DGS takes 7–18. If you want a faster "capture to viewable" loop, optimize the COLMAP stage first.

Pitfall three: heat and charging change the results. The G1-Ultra's GPU drops from 0.68 to 0.60 GHz at 55–56 °C, raising mean step time 8%. The A14 enters the "serious" thermal state from the fifth minute, and on the charger its median step time rose 44%. Measure endurance on battery, screen on.

Pitfall four: portable does not mean performance-matched. Forcing certain conservative options on individually — exact pair gather, the portable sort, streamed training images — more than doubles step time. Streaming does cut tracked memory from 995 MB to 312 MB, but you pay for it in speed.

Cost and time

  • Hardware: any desktop GPU with Vulkan 1.1 compute support; no CUDA required. On mobile, the phone you already own.
  • Time: desktop standard quality ranges from 115 seconds to 3.44 hours depending on the card; phones run 2.36–3.23 hours.
  • Energy: measured phone runs consumed roughly 7–17 Wh for a training pass — an overnight-charge order of magnitude.
  • Setup effort: desktop is a standard CMake build; Android and iOS each need their own toolchain, and that is the main barrier.

Next steps

  1. Get the desktop path working first. Run standard_quality.json on a non-Nvidia machine (AMD or Mac) and confirm the Vulkan path produces results for you.
  2. Then try the phone app. Reproduce one bundled Mip-NeRF 360 scene and measure your own real capture-to-view time.
  3. Read the license. AGPL-3.0-or-later reaches network services too; confirm whether you need the commercial license before shipping.
  4. Watch the schedule log. The choice recorded in bench.json is the best entry point when performance looks wrong.

This article is compiled from the paper (arXiv 2611.22240v1) and the official code repository. No independent reproduction was performed; every performance figure is quoted from the authors' own measurements.

Sources

  • arXiv abstract: https://arxiv.org/abs/2611.22240
  • Full paper (HTML): https://arxiv.org/html/2611.22240v1
  • Official code repository: https://github.com/unclearness/cross3dgs
  • Project page: https://unclearness.github.io/cross3dgs/

Comments

0 published
Log inorsign up to leave a comment.

Loading comments...