PhotoRealistic RenderMan 3.9
Shading Language Extensions

Pixar
September, 1999


Since the publication of the RenderMan Interface 3.1 Specification, several proprietary extensions have been made to the Shading Language in PhotoRealistic RenderMan. PRMan 3.8, in particular, has some fairly substantial changes in the Shading Language. This document summarizes all of the Shading Language extensions in PhotoRealistic RenderMan which are not covered in the RenderMan 3.1 specification.

Subsections of this document include:

  1. New Variable Types
  2. Arrays in Shading Language
  3. New and Improved Built-in Functions
  4. Smooth Derivatives
  5. Lighting Control -- nondiffuse and nonspecular lights
  6. Broad Solar Lights
  7. Message Passing Between Shaders
  8. User Functions and Scoping Changes
  9. Time Derivatives
  10. Volume Shader Functionality
In addition, dynamically loaded user-defined shadeops are documented in Adding C Functions to Shading Language with DSOs.

1. New Variable Types: vector, normal, matrix

The RenderMan Interface Specification Version 3.1 specifies that the Shading Language has exactly four data types: float, point, color, and string. This set has been extended to include two variants on point types, vector and normal, and a 4x4 matrix type.

New point-like types

Previously, there was only one point type which was used to represent three different geometric concepts: (1) spatial positions, (2) directions, and (3) surface normals. However, these three geometric entities represent different concepts and have subtly different usage properties. Now there are three separate types corresponding to these concepts: point, vector, and normal, respectively.

Variables declared as vector and normal behave similarly to variables declared as point in most situations. point, vector and normal variables can be arbitrarily assigned from one to another and can be used identically in expressions and function arguments, however mixing these types in ways which are not geometrically appropriate (for example, taking the cross product of two points) will generate a compiler warning.

Keeping these types separate will help to curtail common errors which result in buggy shaders. Note in particular that transform, vtransform, and ntransform work on points, vectors, and normals, respectively, and using them on incorrect types will result in incorrect transformations.

When a shader parameter is declared (with Declare in a RIB file or with RiDeclare in an RI program) to be a vector, any value provided for that variable in any RI call will be transformed into current space using the vtransform function described above. When a variable is declared to be a normal, any value provided for that variable in any RI call will be transformed into current space using the ntransform function. Note that the declaration of the variable in the Shading Language code should match the declaration in the RI stream. If the two declarations do not match, PRMan will issue a warning (and may not transform the data correctly) .

When a triple of floating point values is assigned to a point, vector or normal variable using the type cast operator, an optional coordinate system name can be specified, such as

These operators now transform the triples of floats from the named coordinate system to the current coordinate system using the appropriate transformation routine. Variables declared as vectors and normals do not automatically call vtransform or ntransform when used as arguments or destinations of the point transform routine. This is left to the SL programmer.

Note: Prior to PRMan 3.6, the keyword vector was reserved in the Shading Language, but it was an alias for point, and did not have the behavior described above.

The matrix type

A matrix represents the transformation matrix required to transform points and vectors between one coordinate system and another. matrix is a first-class data type in the language, so it can be used to declare shader instance parameters, local variables and function parameters. Beware if you declare matrix of type varying. That's going to be a lot of data!

matrix variables can be tested for equality and inequality with the == and != boolean operators. In addition, some functions will accept matrix variables as arguments, as described below.

A matrix is, internally, a 4x4 homogeneous premultiplication transformation matrix. However, our intent was to maintain the programmer's model that the matrix is an atomic data type, not a 2-dimensional array. Individual rows, columns or elements will not be accessible through any form of array notation. When used as a shader instance parameter, it is used in cooperation with the matching RIB declaration type matrix to pass 4x4 transformation matrices into the shader. As with the point data type, a matrix in the RIB stream represents a quantity relative to the then-current object space, while a matrix in the Shading Language represents a quantity relative to the renderer's "current" space. Thus, there will be an implicit transformation between the two representations.

The syntax of matrix specification may be subtly confusing the first time it is seen, but it is consistent with the semantics of point specification. A matrix constant can be specified with the following notation:

This generates a transformation matrix which transforms points in the "current" coordinate system into some other coordinate system which is relative to space. That is, if the optional specifier space is used, it indicates that the 4x4 array (a,...,p) itself converts points from the coordinate system space to some other coordinate system which is relative to space, and thus the final matrix produced transforms points from "current" space to that other coordinate system relative to space.

If the optional coordinate system specifier space is not used, the default coordinate system is "current", and so the matrix transforms points from "current" space to some system relative to "current".

In addition, the values 0 and 1, when promoted to type matrix, generate the obvious zero matrix (all elements zero), and identity matrix (elements on the diagonal one, the rest zero).

Gearhead detail: Because of the way that math of matrices works, the syntax matrix (a,...,p) actually transforms points from any space X to a space relative to X, and matrix "Y" (a,...,p) is just the "current" to "Y" transform concatenated onto the front of that. And for those with a scorecard, this means that the syntax matrix "object" 1 generates the matrix which simply transforms from "current" coordinates to "object" coordinates.

Warning: Version 3.7 of PRMan had a major bug in the matrix constant construction shadeop. For example, lines of the form

        M = matrix "shader" 1;
generated the inverse of the matrix that was intended. That is to say, M was documented to be and was supposed to be the current-to-shader matrix, but the operator actually returned the inverse of this.

To fix the problem without removing functionality that very old shaders may empirically depend on, the keyword matrixinv was introduced to have the functionality of the old, broken matrix, and matrix has been adjusted to return the correct forward matrix. N.B.: This change breaks old shaders that use matrix.

The following operators and functions operate on matrix variables.

 

0 and 1
matrix * matrix
matrix1 / matrix2

The following new built-in functions operate on matrices. They are more fully described in the section below on New Built-in Functions.

float determinant(matrix m)
point transform(matrix m, point p)
vector vtransform(matrix m, vector p)
normal ntransform(matrix m, normal p)
point transform(string space, matrix m, point p)
vector vtransform(string space, matrix m, vector p)
normal ntransform(string space, matrix m, normal p)
matrix translate(matrix m, point t)
matrix rotate(matrix m, float angle, vector axis)
matrix scale(matrix m, point s)

In addition, the comp and setcomp functions have been extended to access and set individual floating point components of matrices.

float comp (matrix m, float row, float column)
void setcomp (matrix m, float row, float column, float newvalue)

 


2. Arrays

 

Shading Language now supports arrays of all the basic data types, with the following limitations:

As in C, the syntax for declaring an array of any data type uses square brackets, as datatype[length]. length is a compile-time uniform float constant, which is rounded down to generate an integer length. Zero and negative-length arrays are not permitted. Also as in C, the syntax for specifying the data of a constant array uses curly braces, as {value1, value2, ...}.

As in C, individual array elements may referenced using the familiar square bracket notation. An array element index is a float expression, which is rounded down to generate and integer index. An array element reference may be used in any expression where a variable of the same base type is legal. An array element reference on the left side of an assignment statement is an lvalue, and sets that single element to the result of the right side expression.

The following functions operate on arrays:

arrayname [ index ] (in an expression)
arrayname [ index ] = expression;
datatype spline( float x, datatype-array ctlpts )

You may not assign or compare entire arrays (just like you cannot in C). However, like C, you may initialize an array with constant values. For example, the following declares a shader which takes an array as a parameter, and specifies defaults for the argument:

	surface mysurf (float Kd = 0.5;
			float vals[5] = { 0, 2, 3, 4, PI };
			float pts[10] = { 0 }; )
	{
	  ...
	}

In the above example, default values are given for the entire vals array, in the event that an array of values is not specified in the RIB stream. The array pts is also initialized, but only the value for pts[0] is specified.

There are two other minor syntactic difficulties with arrays, due to the fact that the original language syntax was developed without arrays in mind. First, the existing square bracket notation in the texture call for channel number identification is syntactically ambiguous with the use of string array elements for texture names. This will be handled by defining that if an element of a string array is used as a texture name, the channel number is not optional. Second, if an element of an array is used as an actual parameter to a function which returns a value in that parameter, the Spec seems to imply (due to it's mention of call-by-reference) that this should set that element to the return value. This behavior may be extremely difficult to implement in the current run-time system, however, so programmers should not depend on it.

NOTE: In the initial implementation, there is a significant limitation on declaring arrays. Declarations of local variables and instance variables (i.e. shader parameters) must be of fixed, compile-time-determined length, and length determination cannot be delayed until the RIB file supplies instance data. If the RIB file intends to supply a variable amount of data, the array must be declared large enough for the worst expected case, and the RIB file definition must match this length by providing dummy data in the excess slots. We recognize that this restriction is not ideal; however, it is done purely for practical implementation reasons at this time. We intend to find a way around this restriction in a later release.

 


3. New and Improved Built-in Functions

The following section describes new built-in functions which have been added to the Shading Language, as well as previously existing functions which have changed their arguments or functionality:

The shading language provides four functions that provide access to global rendering internal state (as of PRMan 3.8). In each case, the function is prototypically like the shader variable access functions surface, lightsource, etc. Each function takes a string that is the name of a piece of global data, and a variable to contain the result. The functions return non-zero if the data exists and the variable is of the right type to contain the result, in which case the variable will be filled with the data requested. The functions return zero if any error prevents returning the requested data, including unrecognized requests, unobtainable data and variable type mismatches. Note that the request names are case-sensitive, and correspond exactly to the names of data specified in the RIB Binding of RenderMan. The data that is returned is as would be expected from inspection of the appropriate RenderMan Interface call.

The following built-in Shading Language functions have been extended to add greater flexibility and to increase the overall efficiency of shaders which use them.

Smooth Derivatives

PRMan 3.8 supports smooth differentials and derivatives in Shading Language. RenderMan's derivatives functions (Du(), Dv(), Deriv()) and surface differentials (du, dv) are fundamental to writing shaders that antialias well. In earlier versions of PRMan, surface differentials have not been continuous across the surface, but instead were uniform float values on an individual grid, and that changed discontinuously from one grid to the next. This could result in subtle filtering artifacts, particularly on large objects viewed in perspective. Starting with version 3.8, PRMan computes differentials in a way that is free from grid discontinuities, by using smoothly varying estimates of the optimal size of micropolygons, rather than their exact size. This allows higher-quality antialiasing of shaders. Further, the shading language area functions (texture(), Du(), filterstep(), etc.), that implicitly use these differentials when filtering, benefit from these smooth differentials, automatically removing many grid artifacts previously ascribed to these functions.

This automatic smooth derivative behavior can be turned off, reverting to the pre-3.8 behavior, on a shader-by-shader basis using the shader compiler's -nosd flag. The complementary flag -sd turns smooth derivatives on and is the default.

Full details on smooth derivatives can be found in the associated Application Note.

Dynamically Loaded User-defined Shadeops

RenderMan Shading Language has always had a rich library of built-in functions (sometimes called "shadeops"). This built-in function library included math operations (sin, sqrt, etc.), vector and matrix operations, coordinate transformations, etc. It has also always been possible to write user-defined functions in Shading Language itself. Improvements to function syntax and semantics and compiler robustness was specifically addressed by the new Shading Language compiler, nshader. However, defining functions in the SL itself has several limitations.

PRMan 3.8 allows you to write new built-in SL functions in C or C++. Such functions overcome many of the limitations of SL-defined functions. Full documentation on the syntax, programming requirements, capabilities and limitations of this new feature are provided in the extensive on-line document Adding C Functions to Shading Language with DSOs.

 


4. Lighting Control

The specular and diffuse shadeops now respond to two special parameters of the light shader. Any light shader which contains the following parameter

float __nondiffuse = 1

will be ignored by diffuse (note that there are two underbars). Similarly, any light which contains the following parameter

float __nonspecular = 1

will be ignored by both specular and phong. Please note that if the value of the parameter is 0.0, it is not ignored, so this behavior can be controlled both from the RIB file, or procedurally from inside the light shader, if desired. These variables may be either uniform or varying. These special parameters can also be accessed within illuminance statements, as described in the section on Message Passing.

The illuminance statement, as well as the specular and diffuse shadeops, now correctly ignore all ambient lights (lights where L = (0,0,0)).

Light Categories

PRMan 3.8 introduces support for “light categories”. Previously all light source shaders attached to an object were evaluated simultaneously whenever a shader requested light values by calling diffuse, for example, or in illuminance loops. There was no mechanism for executing a single light source shader at a time. PRMan 3.8 allows lights to be tagged with a set of category identifiers, and illuminance statements to request evaluation of only those shaders that match particular categories. Other light shaders will not be evaluated until their categories are requested.

Light shaders can specify the categories to which they belong by declaring a uniform string parameter named __category (this name has two underscores), whose value is a comma-separated list of categories into which the light shader falls. For example,

        lightsource uvglow (
		float intensity = 0.5;
		output varying color uvcolor = 0;
		uniform string __category = "uv,photonic,Tony") {

                solar(vector "shader" (0,0,1), 0) {
                        Cl = intensity;
                        uv = 1.0;
                }
        }
The value of this parameter may be a compile-time default, or may be specified in the RIB file, but may not be computed by the shader -- the parameter cannot be output.

Other shaders that have illuminance loops may specifically execute, or specifically exclude from execution, lights in any particular category. This is done with an optional string parameter to the illuminance block, which is matched against the categories of each light source. Only light sources that match will be run. If the string value starts with a hyphen, then any matched light sources will be excluded, and all other lights will be run. For example,

        surface foo ( ) {
                color uv = 0;
                illuminance("uv", P) {
                        lightsource("uvcolor", uv);
                }
                illuminance("-uv", P) {
                        Ci += Cl;
                }
        }

5. Broad Solar Lights

The solar block is used by distantlight to specify a light which is emitted from infinity along a particular axis direction. In this use, the solar block takes two arguments, the axis vector A, and 0.0, which is the angular width of the emission cone. The RenderMan specification refers to the fact that this cone can have non-zero width, and specifically mentions the use of solar with no arguments to write an environment mapping light source using a 360 degree cone, but PhotoRealistic RenderMan has never supported this feature.

PhotoRealistic RenderMan now supports solar cones of any angular width, including the zero-argument omnidirectional case. The meaning of a non-zero width solar cone is extremely confusing to explain. Nonetheless, imagine a distant light source which is willing to emit light in any of a range of directions (not just down its axis vector), and merely needs the renderer to tell it which direction is pointed most favorably toward the specular highlight of surface. Another way of thinking of it is a large area light source, located at infinity, where each point on the area light is emitting a little spotlight.

The most obvious example is an environment-mapping light. The light wants to cast the environment texture map like a slide projector from infinity, and just needs to know which pixel on the map will reflect back into the camera. The pixel to choose is based entirely on the reflection direction R of the surface; the light itself is willing to send light in any direction as needed.

The second example is a diffuse skylight, where light is arriving on the surface from everywhere above it, and would like to be colored by all of it (radiosity-style). In this case, the solar light cone would be a hemisphere pointed down.

At this time, PhotoRealistic RenderMan will not point-sample the area light source, but it will attempt to find the most favorable direction by considering the size of the solar cone and the surface's reflection direction. The light direction vector L inside the solar block will be this direction. In the specific case of an omnidirectional light, this L will be exactly equal to -R. For narrower solar cones, it will be some vector between -R and A, such that it falls within both cones and is somewhat central to the intersection of the cones.

Here is a light source which casts an environment map onto a surface. Notice the use of the __nondiffuse flag to put the environment only in the specular component (since environment maps are really specular reflections of the worldsphere).

light envir ( string mapname = ""; float __nondiffuse = 1; ) {
	solar() {
		Cl = environment(mapname, vtransform("world", -L));
	}
}

Note: Some surface shaders use non-standard reflection directions for their specular highlights. For example, some anisotropic shaders (such as brushedmetal) have multiple specular highlights, none of them in the "normal" place. solar's guess about which direction is the most favorable direction will be wrong in these cases.


6. Message Passing

There are four Shading Language functions which permit shaders to peek at the parameters of the other shaders that are being executed on the same piece of geometry. This permits shaders to customize their behavior based on their knowledge of what the other shaders will do with those parameter settings.

float atmosphere( string param; {float|point|color|string} var )
float displacement( string param; {float|point|color|string} var )
float lightsource( string param; {float|point|color|string} var )
float surface( string param; {float|point|color|string} var )

For example, the new diffuse shadeop, which notices the special __nondiffuse light parameter, could be written as follows:

color diffuse( point N ) {
	color C = 0;
	point Nn, Ln;
	uniform float nondif;

	Nn = normalize( N );
	illuminance( P, Nn, PI/2 ) {
#if EITHER_WAY
		nondif = 0;
		lightsource("__nondiffuse", nondif);
#else
		if (lightsource("__nondiffuse", nondif)==0)
			nondif = 0;
#endif
		if (nondif == 0) {
			Ln = normalize( L );
			C += Cl * Ln.Nn;
		}
	}
	return C;
}

Parameters to a shader typically cannot be modified by the shader. For example, the following shader

surface foo ( float bar = 1 ) {
	bar = 2;
}

will generate the following error if compiled

"foo.sl", line 2: shader "foo" - warning: "bar" readonly but modified

This can now be eliminated by the use of the new keyword output:

surface foo ( output float bar = 1 ) {
	bar = 2;
}

Shaders can set flags and send them as "messages" to other shaders, by placing them in this globally-visible "blackboard" of output parameters, where they can be received (using the parameter-reading functions above) by any other shader that is savvy to their existence.

A more powerful variation of this theme is to use output varying parameters. This way, shaders can compute arbitrary values that vary across the surface, and pass them to other shaders. For example, consider the following pair of shaders, which shade a displaced surface based on the original shape (if that information is available).

displacement wavy (float height = 1; output varying point oldP = 0;) {
	point Po = transform("object", P);
	point No = ntransform("object", N);

	oldP = Po;
	Po += height * normalize(No) * sin(2*PI*s);
	P = transform("object", "current", Po);
}

surface prepainted (float Kd = .5) {
	point Po;

	if (displacement("oldP", Po)==0)
		Po = transform("object", P);
	Ci = Kd * texture("monasmile.tx", xcomp(Po), ycomp(Po));
}

Notice that prepainted will work correctly if there is no displacement shader, and do the best it can if the displacement does not set oldP, but will have the special effect if oldP does exist.

Unlike standard varying parameters, output varying parameters do not need to be bound to the geometry in the RIB file. However, there is currently an implementation limitation (aka bug) whose effect is to make the default initialization values and the RIB overrides of those defaults not always be correct for output varying parameters. The programmer should not rely on those values, even within the shader itself. In other words, it is best for a shader to consider its output varying parameters to be write-only.

The wily programmer may ask: At what point in the shading pipeline do output parameters get set so that I can read them? (Since there is no restriction on the use of the reading functions, one might imagine that the displacement shader might try read an output of the atmosphere shader before the atmosphere has ever been run.) The answer is complicated, but the basic rules are:


7. User Functions and Scoping

 

One of the biggest limitations of the old compiler was that it had trouble compiling functions in SL. There was a funny issue about separate compilation, there were severe restrictions on what you could do inside functions (e.g. you could not call texture()), and sometimes the compiler just compiled functions incorrectly.

Variable Declarations are Statements

In old SL, local variables could only be declared at the beginning of a function or shader. In new SL, you can declare local variables any place where a statement is valid. Variables are accessible in the scope in which they are declared (or any nested inner scopes) after the declaration appears. Variables are not visible outside their scopes, or prior to the line on which they are declared. You may generate a new scope any place that a statement is valid, using braces { }.

This may seem confusing, but basically, the rules are nearly identical to C++.

You will get a warning (but not an error) if you declare a variable whose name hides an identically named variable in an earlier scope. It is an error to use a variable outside the scope in which it is declared, or to attempt to declare two variables with the same name in the same scope.

New inlining strategy

The old compiler generated separate .slo files for each function. When your shader finally compiled, the pre-compiled slo for each function which your shader called was reread and inlined into your shader code. This was the source of many problems.

The new compiler does not support separate compilation of functions. Instead, you should declare your functions either in the same source file as your shader, or put the function definition in a header (.h) file and use #include to tell the compiler to include the source for the function. In some sense, this makes functions more like the "Pascal model", where you define your functions in the same source module before using them, whereas the old way was more like the "C model" of separate compilation of modules.

Function Scoping Rules

You may now define a function local to any scope, any place that you would be allowed to define a local variable. (Local variable declarations are now ordinary statements as well, so by extension, you may define a function anywhere you would put a statement.) The function may be called within its scope (or inside any nested scopes) after it is defined. It may not be called from outer scopes, or before (in the textual sense) it is defined. You may recognize this as being somewhat reminiscent of the way Pascal allows locally defined functions.

Here is an example of a function which is local to a shader:

	surface mysurf (float Kd = 0.5;)  /* parameter */
	{
	    color shadernoise (point p)
	    {
		return color noise (transform ("shader", p));
	    }

	    float q;   /* local variable of the surface shader */

	    /* code for the shader */
	    q = 1;
	    Ci = shadernoise(P) * diffuse(faceforward(normalize(N),I));
	}

Functions may still be declared outside of shaders, just as they always were before. Such functions may not access variables other than their parameters and locals. Functions may have their own locally defined functions, and anyplace where you create a new scope using curly braces, you may declare functions local to that scope.

New declaration syntax: output, extern, void

The following changes have been made to the declaration of formal parameters, variables, and return values for functions:

 

Relaxed restrictions on operations in user functions

It is now legal to refer to graphics state variables (i.e. P, N, Cs, etc.) inside of functions, as long as they are declared as extern and the function is defined inside a scope where the variable is defined. For example, a function inside a surface shader may reference Cs, but a function inside a light shader may not.

The old compiler did not allow use of language constructs inside functions which were not allowable inside all shader types. For example, functions could not contain illuminance loops and could not access texture maps. The new compiler allows any syntactically legal constructs to be used inside functions, though it may issue an error if you use the function from within a shader where such a construct is not allowed.

Polymorphism

Like built-in functions (such as noise) and DSO functions, users are now allowed to write SL functions which are polymorphic. In other words, you may write multiple functions with the same name, but which take different arguments, and the compiler will correctly use the right function under the right circumstances.


8. Time Derivatives

Two new global variables and a new built-in function have been added to allow shaders to take into account how the surface changes with time (motion blur).

The following two new built-in variables are available:

float dtime

 

vector dPdtime

In addition, the following new built-in function is reserved for future use, but is currently unimplemented:

float Dtime (float f)
vector Dtime (point p)
vector Dtime (vector v)
vector Dtime (normal n)

 


9. Volume Shader Functionality

 

Illuminance loops inside volume shaders

Versions of PhotoRealistic RenderMan prior to 3.7 had greatly restricted functionality of volume (Atmosphere) shaders. The most restrictive limitation was that there could be no illuminance loops (or calls to diffuse, specular, or phong) inside volume shaders, which made it difficult to implement volumes which responded to light.

Beginning with PRMan 3.7, it is now legal to sample illumination inside volume shaders. A separate applications note on writing volume shaders gives examples of volume shaders for smoke.

Clarification about the direction of I

Inside surface shaders, the built-in variable I holds the incident ray which points from the eyepoint E to the shading point P.

The original RenderMan Interface Specification 3.1 document was ambiguous about the direction of I inside volume shaders. Table 12.4, "Volume Shader Variables" (p. 112 of the spec), seemed to imply that I had its origin at P for volume shaders, in other words, opposite the direction of I in surface shaders.

At this time we would like to clarify that I faces the same direction in volume shaders as in surface shaders. In other words, I points toward P rather than away from P.

 

Pixar Animation Studios
(510) 752-3000 (voice)   (510) 752-3151 (fax)
Copyright © 1996- Pixar. All rights reserved.
RenderMan® is a registered trademark of Pixar.