Sympo: Render - Points

(Feature image from: A Comprehensive Overview of Gaussian Splatting - Medium - Kate Yurkova)

Surveys

A Survey on 3D Gaussian Splatting

Arxiv

(2024-01-10)
- Review papers until Jan 2024
- Not very detailed.

3D Gaussian as a New Vision Era: A Survey

(2024-02-13)
- Papaers in 2023.

Render Quality

Sorting

StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering SIG'24 | Lukas Radl, Bernhard Kerbl, TU Graz

(Mentioned by will)

Code
1. Task: Rending with splatting
2. Why matter?: Real-time rendering
3. Problem: 3D Gaussians are not sorted carefully for varing view directions.
4. Solution:
  - Progressively find the first intersection with a 3D Gaussian on the ray
  - Optical flow
5. Conclusion
6. Rethink
7. Sidenotes

MLP

Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering CVPR'24 | Tao Lu, Bo Dai

Code | Live: 【CVPR大佬讲paper第二期】上海AI实验室鲁涛大佬讲Scaffold-GS, GSDF, OctreeGS录制内容

(2024-05-04)
- 显隐结合

Quantum Physics

(2024-09-25)

Jason wechat group (24/09/02):

清华大学的2DGH，从量子物理中汲取灵感，提出使用高斯-埃尔米特核作为高斯分层中的新基元，在几何重建和新视图合成任务中的非凡性能。

Reduction

EAGLES: Efficient Accelerated 3D Gaussians with Lightweight EncodingS

Code | Arxiv

(2023-12-08)
1. Vector-Quantized to encode each Gaussian’s color and rotation attributes to a discrete vector to reduce memory usage.

LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS

Code

(2023-12-09)

Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis

Emergent

(2024-01-08)
1. Sensitivity-aware clustering, quantization-aware fine-tuning, and entropy encoding.
2. Learned codebooks, compress 31x

Compact 3D Gaussian Representation for Radiance Field

Code | Emergent | ProjPage | Joo Chan Lee

(2024-01-15)
1. Reduce the number of Gaussians by learnable masking.
2. use MLP to enhance the Gaussians’ color.

GES emergent

(2024-02-18)
1. Replace Gaussian distribution with Generalized Exponential Function

PBR

Relightable 3D Gaussian: Real-time Point Cloud Relighting with BRDF Decomposition and Ray Tracing

ProjPage

(2025-04-17T23:40:22)

Will’s collection:

3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes Moenne-Loccoz等 Radiant Foam: Real-Time Differentiable Ray Tracing Govindarajan 等 RaySplats: Ray Tracing based Gaussian Splatting Byrski 等 REdiSplats:Ray Tracing for Editable Gaussian Splatting Byrski 等 Reflective Gaussian Splatting Yao 等

Geometry

Mesh Recon

Trim 3D Gaussian Splatting for Accurate Geometry Representation Arxiv | Lue Fan, Zhaoxiang Zhang, CASIA

No Code | src:QChatGrp

(2024-06-12)
- Task: Geometry reconstruction
- Problem: redundant and inaccurate Gaussians
- Solution: Contribution-based trimming and small Gaussians’ scale
- Conclusion: Remove false Gaussian and preserve correct structures.

SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering

Code | Arxiv | Coolpaper

(2023-12-17)
1. Align Gaussians with surface

NeuSG: Neural Implicit Surface Reconstruction with 3D Gaussian Splatting Guidance

Arxiv

(2024-02-20)
1. Joint optimization (3DGS can’t perform gradient descent simultaneously with a neural network.)

QQ chat 2024-02-20: 除了 Sugar，现在还有什么新的 3DGS 提 mesh 的方法嘛

will：GaMeS，GSIR，Mesh-based Gaussian splatting for real-time large-scale deformation，都得把 gs 退化成椭圆，目前应该只有 sugar 开了源。还有一个 NeuSG 是 nerf+GS 联合出 mesh 的。 gs 本身的拓扑关系不强，提 mesh 还是不如 nerf，更别说比过传统方法了。

2D Gaussian Splatting for Geometrically Accurate Radiance Fields

Arxiv

(2024-04-16)
- 2D Gaussians compose of the surface.
(2024-04-24)
- Math derivation (Chinese): will-zzy/2dgs-non-official
- python imple - Colab
GOF

Single-View

Splatter Image: Ultra-Fast Single-View 3D Reconstruction

Code | arXiv | brief

(2023-12-30)
1. One image is input into a UNet to reconstruct “images” that is interpreted as parameters (opacity, RGB, Covariance, positon) of all Guassians.
2. Each pixel corresponds to a Guassian.
3. Cross-view transformer
4. Compared with PixelNeRF

AGG: Amortized Generative 3D Gaussians for Single Image to 3D

No Code | CoolPaper | Emergent | Dejia Xu, UofTexas

(2024-01-10)
1. Comparing with optimization-based single image to 3D method, such as leveraging diffusion model, AGG obtains Gaussian-wise color and opacity from augmented image features.
2. Coarse representation: DINOv2 image features followed by 2 transformers that encode features to Gaussian locations and a tri-plane texture field.
3. The Gaussian locations + texture feature are mapped to color and opacity for each Gaussian by an MLP
  - While in NeRF, MLP is used to output each point’s rgb and density.
4. In fine stage, Gaussian’s coarse color and opacity are augmented with the DINOv2 image feature, and perform super-resolution on the low-resolution feature map, which will be decoded by MLP.
(2024-04-12)
- Triplane is a kind of implicit representation. While Gaussians is an explicit representation. 显隐结合
- Upsampling Gaussians is a densification? (Heard from 【3D AIGC论文串讲】单视图3D重建-从单视图中预测3D模型】)

Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers

Gradio Demo | Code | ProjPage | VAST

(2024-01-20)
- This model (on huggingface) is trained on Objaverse-LVIS (~45K synthetic objects) only.
  
  Is their multi-view consistency derivated from geometry prior contained in the huge dataset?
- Image -> pre-trained ViT –> Upsample (SnowflakeNet) –> point cloud -> Tri-plane -> Gaussians’ parameters –> Splatting

Gamba: Marry Gaussian Splatting with Mamba for single view 3D reconstruction

Arxiv

(2024-04-16)
- Single view -> DINO features -> Mamba-based sequential network -> 3D Gaussians

Touch-GS IROS'24

Code

(2024-07-09)
- 斯坦福开源Touch-GS！视觉-触觉有监督的3D高斯泼溅-计算机视觉life
- Bayesian. How to introduce probability?

Sparse views

Mask

GaussianObject

GaussianObject: Just Taking Four Images to Get A High-Quality 3D Object with Gaussian Splatting

Code | Arxiv | CoolPapers | Chen Yang, Wei Shen, SJTU

(2024-02-24)
- Mask for an object. Not reconstructing a scene.
- Train a diffusion model for repairing.
(2024-04-09)

Two problems and two solutions:
1. Problem 1: Overfitting to sparse views resulting in fragmented structure.
  - Visual hull of objects for structure prior to restrict Gaussian kernel within object outline.
2. Problem 2: Information missing.
  - Train a 2D diffusion model, with leave-one-out training and noise-added gaussians, to predict what a corrupt rendered image should have looked like. (for arbitrary viewpoint?)
  - The predicted images are used to refine 3DGS.
(2024-07-26)
- The project: SAM has been archived. I’m confused as it’s so popular.
- Download the “Dataset Pt. 1” of Mip-NeRF 360 (11.7G): wget http://storage.googleapis.com/gresearch/refraw360/360_v2.zip
  
  Check: unzip -l 360_v2.zip

Depth Regularized

DNGaussian

DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization ~ CVPR 2024

Code | Arxiv | Jiahe Li, Beihang

Supports

(2024-04-14)

Less training time and less memory footprint.

Geometry degradation can be mitigate by dpeth constraint.

Loss: hard- and soft-depth regularization; Global- and local-depth normalization.

Freeze Gaussians’ shape (covariance), and apply depth regularization to position and opacity of Gaussians.
- Hard depth: Gaussians on the surface are the outest hull of the point cloud, they can be identified by setting a large opacity, and when rendering depth map, only their depths are revealed.
- Small depth variation matters for 3DGS -> normalization
- A depth map is split into patches. Normalize each patch with local patch-level depth variance, and global image-level depth variance separately.
- Colors are predicted by a MLP.

Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Images

Arxiv | Jaeyoung Chung

(2023-11-30)
1. Few shot but without overfitting
2. Pre-trained monocular depth estimation model
Representing a 3D scene by combining numerous Gaussian splats has yielded outstanding visual quality.

NexusGS

References:
1. NexusGS: Sparse View Synthesis with Epipolar Depth Priors in 3D Gaussian Splatting
2. Code

Supports:
1. Problem solving
2. Implementation

Introduction:
1. Problem Analyzing
  
  flowchart LR a("Densification") a --> b1("The number of points") a --> b2("Their position accuracy") b1 --> c1("FSGS") b2 --> c2("Depth")
2. Competition
  
  “NexusGS for Novel View Synthesis”
  1. Continuous camera pose rendering
  2. Metrics: PSNR, SSIM, LPIPS
  3. Sparse input views
3. Approach
  
  flowchart LR a("Depth map") -- constrainted --> b("Optical Flow,
  Epipolar Line") b --> c("Point insert")

Image Features

CoherentGS

CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians

No Code | Arxiv | Avinash Paliwal, Texas A&M

(2024-04-06)
(2024-04-16)
- Regularizer: single&multiview convolution decoder + total variance loss + flow-based loss.
- Initilization: Monocular depth estimation model

pixelSplat

pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction

Code | CoolPapers | David Charatan, Vincent Sitzmann

(2024-04-15)
- 2 views
- Sample from a distribution with reparameterization trick.
- Feed-forward: reference-image colors are fused into novel-views colors.
  
  Feed-forward (Generalizable) methods require training on large datasets, e.g., RealEstate10K, which is also used by: IBRNet, GNT, MuRF, to accquire a general 3D scene prior.

MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images

Code | Arxiv | Yuedong Chen, Monash

(2024-04-13)
- MVSNet + 3DGS
- I guess this paper has been rejected like his last project Match-NeRF, because as Jiayuan said, “just changed a dataset”.

MVSGaussian: Fast Generalizable Gaussian Splatting Reconstruction from Multi-View Stereo

NoCode | Arxiv | Tianqi Liu, HUST

(2024-05-27)
- “MVSNet” -> depth map -> pixel-aligned feature -> Gaussian parameters.
- Chinese post

Frequency

RAIN-GS: Relaxing Accurate Initialization Constraint for 3D Gaussian Splatting

Code | Jaewoo Jung, KoreaU

(2024-04-18)

Accurate Densification

Revising Densification in Gaussian Splatting

(2024-04-10)

X - Zhenjun Bao

Gradient Direction

Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splatting

Code | Arxiv | Zheng Zhang, UoHK

(2024-05-09)
- Improve the clone and split with focus on gradients.
(2024-05-27)
- The Gaussian growth is based on the gradient. Thus, controling the growth requires to control the gradients.
- Use the number of visible views as the weights of gradients
  - This reminds me the NeuRay which uses “visibility” to facilitate consistent geometry.
- Scale the gradient according to the distance to the camera to avoid the tendency that folaters exist near the camera.

AbsGS: Recovering Fine Details for 3D Gaussian Splatting

No Code | Arxiv

(2024-04-17)
- The gradients of various Gaussians have different directions, so the gradients from covered pixels for a big Gaussian get canceled each other out. Thus, the big Gaussians won’t split

Regularization

GaussianPro: 3D Gaussian Splatting with Progressive Propagation

Code | Arxiv | CoolPapers

(2024-04-17)
- Supplement Gaussians with high uncertainty from the rendered normal maps.
- Regularization term: planer normal map loss
- Propagate: fuse neighboring pixels’ normals
- Accurate geometry: evaluate new Gaussians with photometric consistancy.

Interpolate

FSGS: Real-Time Few-Shot View Synthesis using Gaussian Splatting

Code | ProjPage | Zhiwen Fan, UoTexas

(2023-12-04)
- 3 views
1. Unpooling the existing Gaussians to densify the sparse point cloud resulting from sparse views
2. Depth regularization from a pre-trained monocular depth estimation model.

Stochastic Process

References:
1. 3D Gaussian Splatting as Markov Chain Monte Carlo Arxiv (Was trying to find the img of densifying. Search: “3D Gaussian Splatting ar5iv” in DDG)

(2024-07-19)

Task:
Rethink:
- 感觉 densification 就是点云上采样的问题，可以单独研究上采样的问题，和 3DGS 结合就是蹭热点。 3DGS 的“本质”是：渲染快。

Attack

Poison

Problem Investigating
1. Make a computation-intense attack on the service vendor server
::: aside
- References:
  1. Talk | 新加坡国立大学博士生卢嘉昊：Poison-Splat：针对3D高斯溅射的计算成本攻击 - TechBeat人工智能社区 :::

Supports
1. f

Extend
1. How to densify wisely?
  
  flowchart LR a("Wise densification") a -- reverse --> b("Suppress bad densification") b --> c("Defence computational-intense attack")
2. Solutions
  
  flowchart LR a("抵制下毒：计算成本的增加") a --> b("分类问题：判别一个 “高斯基元是否对场景重要？”") a --> c("去噪问题：消除输入图片中的（高频）噪声")
3. Attack in another way?
  
  flowchart LR a("攻击") --> b("加噪") --> c("DDPM")

Deblur

Deblurring 3D Gaussian Splatting

Emergent

(2024-01-05)
1. MLP “re-fuses” quaternion (covariance matrix), scaling matrix and position

SLAM

Generic 3DGS optimize a pre-generated point cloud.

Gaussian Splatting SLAM

Arxiv | ProjPage

(2023-12-14)
- Incrementally construct the point cloud
1. Optimize camera pose alongside point cloud via gradient descent according to the Jacobian of camera pose w.r.t. the 3D Gaussian map (screen function).
2. Confine geometry consistency via
3. Point cloud growing and pruning
哈工大博士分享：基于Gaussian Splatting的SLAM新发展与新论文（上）- bilibili - 计算机视觉life
How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey

(2024-02-29)

Anti-alias

TRIPS: Trilinear Point Splatting for Real-Time Radiance Field Rendering

Code | Emergent | ProjPage | Linus Franke

(2024-01-14)
- screen-space image pyramid
- Light-weight MLP

Edit

GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting

Code | arxiv | Yiwen Chen, Guosheng Lin

(2024-01-21)
1. Integrated various papers.
2. Gaussian semantic tracing ? Hierarchical Gaussian Splatting ?

Diffusion Prior

InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior

Code | Arxiv

(2024-04-18)

Splatting

GS++: Error Analyzing and Optimal Gaussian Splatting

Arxiv | Letian Huang, NJU

(2024-02-04)
- Minimize the Taylor approximation error in the projective transformation
- Research aspect: point cloud storage, performance , and robustness in sparse viewpoints

360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming

Arxiv | Jiayang Bai, NJU

(2024-02-04)
- Spherical surface, analogy to NeRF++

Simulation

Dynamic 3DGS

GMix.ai Dynamic Gaussian Splatting post

Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle

ProjPage |Youtian Lin, Yao Yao

(2024-02-21)
- Movement in a timestep

GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting

| Surfaced by Jason

(2024-02-29)

网格面的顶点对每个高斯分量进行参数化

SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes ~ CVPR 2024

ProjPage | Yi-hua Huang, Xiaojuan Qi

(2024-03-06)
- Sparse control points
- Decompose motion and appearance
- MLP predict 6DoF movement. MLP can’t reach accurate result, so it works for large-scale movement. And can it represent the multi-object interaction? like collision?

Material Points

References:

PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics
NeuMA: Neural Material Adaptor for Visual Grounding of Intrinsic Dynamics - NeuIPS‘24
- Mentioned in Jason’s WeChat group.

(2024-03-06)

PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics
- Related:
  1. Code
  2. Pdf: Xuan Li, Chenfanfu Jiang
- Reasons:
  1. MPM solver!
  2. This method may produce reasonable effects as the underling physics simulation.

(2024-12-02)

NeuMA ^r2-Nips

Depth Regularized

EndoGS: Deformable Endoscopic Tissues Reconstruction with Gaussian Splatting Emergent | Code

(2024-02-10)
- Depth-guided supervision for handling occulusion

Text23D

GSGEN: Text-to-3D using Gaussian Splatting

Code | Zilong Chen

(2024-02-29)

No Pose

InstantSplat: Unbounded Sparse-view Pose-free Gaussian Splatting in 40 Seconds

ProjPage | Zhiwen Fan, UTAustin

(2024-04-01)

3DGS + DUSt3R

COLMAP-Free 3D Gaussian Splatting ~ CVPR 2024

No Code | Arxiv | Yang Fu, Xiaolong Wang, UCSD

(2024-04-09)
- Given a video, each frame produces a depth map and a local Gaussian set, which will be merged into a global Gaussian set.
- Optimize the camera pose affine transformation between 2 adjacent frams: the current frame and its previous frame.

Speed

NeRF Prior

RadSplat: Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS

No Code | CoolPapers | Michael Niemeyer ;

(2024-04-16)
- Training 3DGS with NeRF as supervision.

Texture

Textured-GS: Gaussian Splatting with Spatially Defined Color and Opacity Arxiv | Zhentao Huang, Minglun Gong, UofGuelph

(2024-07-20)
1. Task: The color attribute of each 3D Gaussian
2. Why matter: Rendering surface
3. Problem: SH coeffs
4. Solution:
  - SH coeffs -> RGB,opacity

Init Point Cloud

VGGT Replace Colmap

Motivation
1. Colmap functionality: Generate sparse point cloud
2. VGGT functionality:
3. Implementation: VGGT + 3DGS^r1-快速
  - Use Brush
::: aside
- References:
  1. VGGT＋3DGS快速生成高质量场景 - Yuinix
  2. ArthurBrussee/brush
:::
Results
1. Speed up
2. Process more input images

VGGT Bad Pose

Supports:
1. It is said the pose are not usable for gs ^r1-Dscd
::: aside
- References: {{{
  1. Discord - MrNeRF & Brush | camera-pose-estimation }}} :::

VGGT + BA

Problem:

(2025-09-22T12:55)
1. Bundle Adjustment can improve the poses generated by VGGT
  - Although the poses are not as accurate as those from COLMAP, it could be an alternative when Colmap fails. ^r1-Dscd
::: aside
- References: {{{
  1. Discord - MrNeRF & Brush | camera-pose-estimation }}} :::

Number of Points

Problems
1. 空间中的高斯基元数量是否指定？
  - 有的 3dgs 实现需要指定高斯基元的数量：LichtFeld (MCMC)
  - 而如果不指定数量，即允许随场景规模扩张增加的话，这会是一个潜在的安全漏洞: 计算成本攻击
::: aside
- References:
  1. MrNeRF/LichtFeld-Studio :::

Table of contents

Surveys

Render Quality

Sorting

MLP

Quantum Physics

Reduction

PBR

Geometry

Mesh Recon

Single-View

Sparse views

Mask

GaussianObject

Depth Regularized

DNGaussian

NexusGS

Image Features

CoherentGS

pixelSplat

Frequency

Accurate Densification

Gradient Direction

Regularization

Interpolate

Stochastic Process

Attack

Poison

Deblur

SLAM

Anti-alias

Edit

Diffusion Prior

Splatting

Simulation

Dynamic 3DGS

Material Points

Depth Regularized

Text23D

No Pose

Speed

NeRF Prior

Texture

Init Point Cloud

VGGT Replace Colmap

VGGT Bad Pose

VGGT + BA

Number of Points