PhotoRealistic RenderMan
Application Note #24

Contents:

• Introduction
• Changes to RiDisplay
• Using Shaders to Output Arbitrary Variable Values
• Quantization and Dithering
• What is it good for?
• Limitations and Bugs

Introduction

Most rendering software is designed to compute a fixed set of output variables and store them in a frame buffer. These often include the color, the opacity or alpha channel, and often the z-depth.

At times, other kinds of outputs could be useful. For instance, you might want an image which contained an object tag for each pixel, indicating which object was present. You might wish to get a dump of all the normals to the surfaces. You might wish to generate special matte frames for use in some post-processing compositing trick. In previous releases, this required the writing of special shaders that encoded the information that you wanted as colors, and rendering the frame potentially multiple times to get each of the values that you want.

With the 3.8 release of PhotoRealistic Renderman, you can create additional display channels to display either geometric information associated with the surfaces themselves, or values computed by the surface shader. The rendering pass will generate multiple output files simultaneously, each containing the variables you desire.

Changes to RiDisplay

The easiest kind of additional values to export are those which are global to the state of each geomtric primitive. These are the same as the set of external variables available inside each surface shader.

Name	Meaning
P	the XYZ coordinates of the surface point
N	the normal at the surface point
Ng	the geometric normal at the surface point
E	the position of the camera
dPdu, dPdv	the tangent vectors to the surface
s, t, u, v	the texture coordinate
du, dv	the stepsize in u and v
dPdtime	the motion vector for the point P
Cs, Os	default surface color and opacity
Ci, Oi	computed surface color and opacity

To create a file containing any of these variables, you need to add a new RiDisplay call. Suppose we wish to output the normals to the surface, we might add a Display line which looked like:

Display "+normal.tif" "tiff" "N"

The leading plus sign is an indication that this is an extra variable, and should create an additional output stream to hold the values. The specified driver (TIFF, in this case) will be sent values of the variable specified in the third argument. Up to nine additional output channels can be specified in this way. When the RIB file is rendered, an image will be created for each Display statement.

Using Shaders to Output Arbitrary Variable Values

If you wish to export a specific expression value from within a shader, you need to define it as an output parameter of the surface shader. A simple example below shows how to export the variable Nn (the normalized front facing normal to the surface) from the trivial defaultsurface shader.

surface
defaultsurface(float Kd=.8, Ka=.2; output varying normal Nn = 0 ;)
{
    float diff ;

    diff = I.N ;
    diff = (diff * diff) / (I.I * N.N) ;

    Ci = Os * Cs * (Ka + Kd * diff) ;
    Oi = Os;

    /* here we compute the value that we wish to export */
    Nn = normalize(N) ;
}

The variable may be displayed just like any external variable above. The variable being displayed should be declared prior to the call to RiDisplay. The corresponding Display call for our example would look like:

Declare "Nn" "varying normal"
RiDisplay "+nn.tif" "tiff" "Nn"

Quantization and Dithering

By default, all extra output channels are sent to the display system as unquantized floating point numbers. You can change the default quantization by using the Quantize statement, just as you did for colors. For example, to quantize the output variable Cs to eight bit values, we might issue the following RIB stream:

Display "image.tif" "tiff" "rgba"
Quantize "rgba" 255 0 255 0
Display "+cs.tif" "tiff" "Cs"
Quantize "Cs" 255 0 255 0

The original Quantize statement does have a number of problems however. Imagine that you would like to output some colors which represent the normalized surface normal as in our example above. Many tools do not really process floating point tiff files very well, so you would like to output the image as an 8-bit tiff. Unfortunately, the Quantize statement does not allow you to remap the value of zero, so you effectively cannot output any of the negative values of the normal when you choose to use Quantize.

To alleviate this problem, you can now specify arguments directly on the Display line which specify a special four argument quantization. The required RIB line would look like:

Display "filename" "driver" "variable" "quantize" [zeroval oneval minval maxval]

The process by which values are quantized is described by the following pseudocode:

	value = round(zeroval + value * (oneval - zeroval) + dithervalue ;
	value = clamp(value, minval, maxval) ;

For example, to display our computed normals as 8-bit numbers where zero maps to 128 and +1 maps to 255, we can issue the Display statement:

Display "image.tif" "tiff" "rgba"
Quantize "rgba" 255 0 255 0
Display "+nn.tif" "tiff" "Nn" "quantize" [128 255 0 255]

Display "image.tif" "tiff" "rgba"
Quantize "rgba" 255 0 255 0.5
Display "+nn.tif" "tiff" "Nn" "quantize" [128 255 0 255] "dither" [0.5] 

In this case, both the RGBA channels and the normal will be dithered by plus or minus 0.5.

What is it good for?

The primary motivation for adding this feature was to allow the creation of non-photorealistic effects, particularly by the use of post processing. Many artistic or painterly effects are easier to achieve by performing two dimensional operations on final images, rather than three dimensional operations within shaders.

For example, the image below is one of Pixar's older characters, Tinny, from the Academy Award winning short animation, Tin Toy. First, we deleted all of Tinny's original shaders, and replaced it with the following shader, written to output a number of other variables derived from his geometry:

surface
myoutput( float Ks=.5, Kd=.5, Ka=1, roughness=.1; color specularcolor=1;
	 output varying vector Nn = 0;
         output varying float Y = 0;
	 output varying float bright = 0;)
{
    normal Nf;
    vector V;

    Nf = faceforward( normalize(N), I );
    V = -normalize(I);

    /* set the values of the output variables */
    Nn = normalize(Nf) ;

    /* pretend that we have plastic */
    Oi = Os;
    Ci = Os * ( Cs * (Ka*ambient() + Kd*diffuse(Nf)) +
                specularcolor * Ks * specular(Nf,V,roughness) );

    /* calculate a bright mask, giving positive values where
     * Tinny's surface is illuminated by various lights.
     */
    bright = 0.5 ;
    illuminance(P) {
	if (N . L > 0) {
	    bright += 0.25 ;
	}
    }

    /* we'd also just like to have the gray scale component, or
     * Y value of the output color.
     */
    Y = comp(ctransform("yiq", Ci), 0) ;
}

The following additional Display statements were added to Tinny's RIB file:

Display "tinny.tif" "tiff" "rgba"
Declare "Nn" "varying vector"
Display "+normal.tif" "tiff" "Nn" "quantize" [128 255 0 255]
Display "+color.tif" "tiff" "Cs" "quantize" [0 255 0 255]
Display "+s.tif" "tiff" "s" "quantize" [0 255 0 255]
Display "+t.tif" "tiff" "t" "quantize" [0 255 0 255]
Declare "Y" "varying float"
Display "+y.tif" "tiff" "Y" "quantize" [0 255 0 255]
Declare "bright" "varying float"
Display "+bright.tif" "tiff" "bright" "quantize" [0 255 0 255]

tinny.tif	color.tif
bright.tif	y.tif
s.tif	t.tif
normal.tif

These can serve as inputs to many kinds of post processing effects that have been suggested in the computer graphics literature.

A Simple Cel-Like Rendering

Many people liked the simple, flat shaded Tinny, saying he looked kind of like a cartoon. We can enhance this type of image by adding in dark lines that surround the regions of solid color, and adding some simple two-toned shading. We do this by computing an edge map by running an edge detection algorithm over the Y image, and darkening all the edges in the solid color image. We also multiply the color image by the brightness map, to result in darker colors where the original model was in shadow.

A Simple Painted Like Rendering

Many software paint programs such as Fractal Design Painter incorporate some kind of "faux oilpaint" like process. By using the image containg normals to set brush direction, and the original color map to set the brush color, we can create an automated process that literally "paints" by picking random places on the image, rotating a brush stroke image to the angle dictated by the normal, and drawing a stroke colored by the original input image. Some randomness has been added in this case to make the colors a bit more varied.

A Simple Pen and Ink Rendering

We shouldn't feel too constrained to merely use pixel or frame buffer devices for the information we generate. The image on the right was generated by using PostScript to draw lots of random strokes, each in the direction tangent to the surface normal in screen space, until the darkness achieved was proportional to the original darkness of the image. PostScript is a convenient language to express vector strokes, and allows a look which would be difficult or impossible to achieve within the confines of RenderMan and the shading language. Example source code for the program ppmtopen which generated these images can be found here.

Limitations and Bugs

The current implementation of arbitrary output variables have a number of limitations:

• The only variable types which can be exported are point, vector, normal, float, and color. Arrays, matrices and strings are not supported.
• RiExposure has no effect whatsoever on additional output variables.
• When outputting additional variables, the PixelFilter must be set to "box" 1 1.
• Output variables can only be exported from surfaces, not from lights or atmospheres.