Rendering rays API

The following API allows to represent rendering Rays for the Lightplane renderer and splatter, and also provides several related utilities.

class lightplane.Rays(directions: Tensor, origins: Tensor, grid_idx: Tensor, near: Tensor, far: Tensor, encoding: Tensor | None = None)

Dataclass for representing rendering or splatting rays.

The 3D point x along a ray emitted from a 3D coordinate origin in along a 3D vector direction is given by: ` x = origin + t * direction ` where t is a scalar in range [near, far]. Note that direction does not have to be l2-normalized.

In order to render multiple scenes given a single batch of rays, each ray is associated with an integer index grid_idx which specifies the index of its coresponding scene.

Optionally, the object can store an encoding of the rays: encoding. This can be useful to define a user-specific encoding of the rays, e.g. a custom version of the harmonic embedding originally proposed by NeRF. Note that the dimensionality of the embedding has to match the number of channels accepted by the corresponding MLP of Lightplane Renderer or Splatter.

Parameters:

• directions – Tensor of shape (B, 3) storing the directions of B rays.

• origins – Tensor of shape (B, 3) storing the origins of B rays.

• grid_idx – 1D Tensor of shape (B,) storing an integer index of each ray into its corresponding feature grid.

• near – Tensor of shape (B,) storing the ray-length at which raymarching starts.

• far – Tensor of shape (B,) storing the ray-length at which raymarching ends.

• encoding – Optional Tensor of shape (B, C) storing the encoding of each ray.

lightplane.calc_harmonic_embedding(directions: Tensor, n_harmonic_functions: int)

Calculates harmonic embedding for the given directions.

Note that the function is strongly inspired by PyTorch3D’s implementation:

https://github.com/facebookresearch/pytorch3d/blob/c292c71c1adb0712c12cf4fa67a7a84ad9b44e5c/pytorch3d/renderer/implicit/harmonic_embedding.py#L12

Parameters:

• directions – Ray directions. … x 3.

• n_harmonic_functions – Number of harmonic functions. If set to 0, the function will only return the input directions, otherwise returns the input directions concatenated with the harmonic embeddings.

Returns:

encoding – Harmonic embedding. … x n_harmonic_functions.

lightplane.calc_harmonic_embedding_dim(n_harmonic_functions: int) → int

Calculates the dimension of the harmonic embedding.

lightplane.jitter_near_far(near: Tensor, far: Tensor, num_samples: int)

Jitters the near and far planes by a random offset in range [-delta, delta], where delta = (far - near) / num_samples.