Reconstruction from Spatio-Spectrally Coded Multispectral Light Fields

PhD Thesis


Maximilian Schambach
Karlsruhe Institute of Technology

Thesis Publications Jupyter Code Data

Introduction


This is a digital supplementary to my thesis 'Reconstruction from Spatio-Spectrally Coded Multispectral Light Fields' supervised and reviewed by Michael Heizmann and co-reviewed by Bastian Goldlücke.
The thesis concludes my work as a research associate with the Institute of Industrial Information Technology at Karlsruhe Institue of Technology from 2017 to 2022.

Supplementing the written thesis, this website contains links to an additional, interactive evaluation of the proposed reconstruction approaches using a Jupyter notebook. Here, all investigated methods can be assesed in terms of their test dataset performance but also to visualize their prediction using different synthetic and real-world datasets. In addition to the results presented in this thesis, further scenes and evaluation metrics can also be evaluated.
Furthermore, you'll find links to my public repositories for the implementaion of all considered methods and conducted experiments.

Notebooks


Jupyter notebooks are provided via Binder for further interactive evaluation. In particular, the 'evaluation' notebook provides full access to the reconstruction results. Here, additional scenes and metrics can be evaluated on the test dataset as well as the dataset challenges and real-world examples.

Evaluation Neural fractals

Datasets


Spectral light field data

Two spectral light field datasets were created: a synthetic dataset with ground truth disparity and a real-world dataset using a custom-build spectral light field camera.

Synthetic dataset Real-world dataset

Lytro Illum whiteimages

To evaluate the pre-calibration of the Lytro Illum camera, a large dataset of synthetic whiteimages with ground truth microlens centers was created.

Whiteimage dataset

Code


Here, find the links to the created Python frameworks. This includes the framwork for light field deep learning (lfcnn). There, in particular, you find the used multi-task training strategies, the developed auxiliary loss training strategies, and the used convolutional networks. Furthermore, you'll find a general light field framework (plenpy), and the source files of the conducted experiments.

lfcnn plenpy Experiments