'Light field' - what

Central to all image-based rendering is the idea of a "light-field": a function that quantifies the amount of light traveling in every direction through every point in space. It is a dense amount of data, which traditional cameras 'flatten' and therefore lose.

Trivia - the idea of light-as-a-field was first proposed by Michael Faraday (in 1846!), who compared light fields to EM fields.

'Light field' - why

What can be done, after capturing a light field?

• refocusing ["synthetic aperture photography"]
• perspective shifting ["light field rendering"]
• 2D <-> 3D switching (mono <-> stereo)
• ...

The area that encompasses the above (and more) is called 'computational imaging', or 'computational photography'.

The plenoptic function

To back up (prior to talking about light fields), we need to consider the 'plenoptic illumination function', which is used to 'express the image of a scene from any possible viewing position at any viewing angle at any point in time' (wikipedia). This is multi-dimensional function - eg. here is a 5D version:

If we additionally consider time, wavelength and polarization angle, this becomes an 8D function. That's a large 'D'! Need to simplify..

The light field, revisited

The 5D plenoptic function reduces to a 4D light field, if there are no 'blockers' to obstruct rays. Eg. this happens in free space:

Scene with blockers (need full 5D)

Mark Levoy

With the basic defns. out of the way, let's look at slides from a talk by Mark Levoy (Stanford)..

And this is another talk from Stanford, with overlapping content.

PaulDebevec@ICT@USC

No discussion of light field tech/research is complete, w/o mentioning Paul's pioneering work in this area.

IBL, IBR, IBM, IB(x)

There is Image Based {Lighting, Rendering}, and there's also Image Based Modeling. Eg. here is a set of pages on the topic.

This is a Microsoft paper on the topic, and a large-scale implementation.

IBR tech/products

This is Stanford's light field microscope movie.

Here is Raytrix's plenoptic cam..

Adobe has created a lightfield cam prototype, using a microlens array, similar to a compound eye..

Lytro's light field cam is the newest kid on the block..