Plug-in Ri Filters

May 2004


Introduction

Release 12 of Pixar's PhotoRealistic RenderMan introduces a new RIB pipeline feature. We have found that pipeline designers appreciate programmability, so we identified and implemented a new contact-point where pipeline designers and TDs can “do their dirty work”. This app-note introduces a new class of RenderMan plug-in that offers new opportunities to improve the flexibility, performance, and encapsulation of a RenderMan-based production pipeline.


Plug-in Ri Filters

Many of the RIB fixup utilities found in RenderMan production pipelines can accurately be described as RenderMan Interface filters. They parse a RIB stream, search for patterns that match some criteria, and replace these patterns with a more preferable, possibly empty, pattern. It is often convenient to construct chains of RIB filters and the result is a powerful back-end RIB processing system that can solve production pipeline problems in a unique way, due largely to its position in the pipeline at the “mouth” of the renderer. To the extent that these filtering systems process data from one on-disk representation to another, it can be far from optimal — it is easy to construct a system that spends more time filtering RIB than rendering. To eliminate the disk I/O overhead and the burdens associated with parsing RIB, we have introduced plug-in Ri filters.


The figure above depicts the contract between the RenderMan renderer and the various plug-in types currently defined. The Ri calls are at the heart of the system and allow procedural primitives and plug-in filters to fundamentally affect the RI stream driving the renderer. The Rx library is provided to allow plug-ins to query rendering state and is available to all plug-in types. For example, RxOption and RxAttribute can be used to look up values of rendering state variables. The new Rif interface is provided to allow plug-in Ri filters to query and affect the state of the active filter chain. The names in boxes represent the entry points required of the various plugin types by the renderer. Ri filter plug-ins must implement the RifPluginManufacture method and this must construct a C++ object whose class is a derivative of the virtual base-class, RifPlugin. The GetFilter method is required to return a dispatch table of Ri functions and it is through this table that an Ri Filter plug-in can do its work. By implementing a subset of the procedures in the RifFilter structure, the plug-in establishes callbacks that will be invoked as an individual Ri request works its way through the filter chain.


Plug-ins can control whether Ri procedures not implemented by the plug-in result in pass-through or terminate behavior. If a plug-in only implements the RiSphere request, the default filtering mode would determine whether the result was a RIB stream of spheres or everything but spheres.


Implementation Example

A trivial filter that converts all spheres to cones would set the default filtering mode to pass-through and would implement a procedure to be called when spheres are encountered. There it would simply invoke RiCone. Because an individual filter has access, via the Rif interface, to the state of the Ri filter chain it can choose to invoke new procedures at the beginning, current, or next point in the filter chain. Here are some snippets from a useful Ri filter plug-in that applies regexp pattern matching to a RIB stream. RIB requests that are bound to an identifier matching the user-provided regular expression are filtered.

First we define our subclass of the RifPlugin virtual base class:

	#include "ri.h"
	#include "librif/RifPlugin.h"
	#include <stdio.h>
	#include <regex.h>
	#include <stack>
	#include <string>
		
	class matcher : public RifPlugin
	{
	public:
				matcher(const char *expr, bool invert);
	virtual 		~matcher();
	
	virtual RifFilter & 	GetFilter(); // Returns my filled in dispatch table

	private:
		RifFilter 	m_rifFilter; // Here's my dispatch table
				  	     // we fill it in at construction
		void 		setName(const char *); // This will be called whenever
				 		       // Attribute "identifier" "name" changes.
		std::stack m_validityStack; // Tracks AttributeBegin/AttributeEnd
						  // contains bool representing whether the current
						  // Attribute "identifier" "name" matches the
						  // expression.
		std::string 	m_expression;     // Here's the expression provided by the user
		regex_t 	m_regexp; 	  // A compiled version of the user's expression
		bool 		m_invert; 	  // Should we invert the sense of the test?
		
		// Here are the class methods which will be plugged into the RifFilter
		// dispatch table. They will need to access the internals of the current
		// matcher object when invoked.
		
	static RtVoid myAttributeBegin();
	static RtVoid myAttributeEnd();
	static RtVoid myAttributeV(RtToken nm, RtInt n, RtToken tok[], RtPointer val[]);
	};

Next we implement the entrypoint that the RifPlugin machinery will invoke to construct an instance of our matcher class. Note that instantiation arguments are provided at the time of construction. For brevity, we omit error checking.

	RifPlugin *
	RifPluginManufacture(int argc, char **argv)
	{
		char *pattern = 0L;
		bool invert = false;

		// here we parse incoming arguments looking for -pat regexp and -inv
		for (; argc > 0 && argv[0][0] == '-'; argc--, argv++)
		{
			if( !strncmp("-pat", argv[0], 4) && argc > 1 )
			{
				argc--;
				argv++;
				pattern = argv[0];
			}
			else if( !strcmp("-inv", argv[0]) )
				invert = true;
		}
		if(!pattern) return 0L;
		// all okay here, so construct an instance of matcher
		matcher *s = new matcher(pattern, invert);
		return s;
	}

Next, our constructor initializes the members of the filter table with our overrides, and does some book-keeping.

	matcher::matcher(const char *expr, bool invert) :
		m_invert(invert),
		m_expression(expr)
	{
		// First we initialize our RifFilter structs with the (static) class methods
		m_rifFilter.AttributeBegin = myAttributeBegin;
		m_rifFilter.AttributeEnd = myAttributeEnd;
		m_rifFilter.AttributeV = myAttributeV;

		// here we initialize other state and compile the regular expression
		m_validityStack.push(!m_invert);
		int err = regcomp(&m_regexp, expr, REG_EXTENDED|REG_NOSUB);
	}

	matcher::~matcher() {
		regfree(&m_regexp);
	}

	RifFilter & matcher::GetFilter()
	{
		return m_rifFilter;
	}

Finally, we define our Ri filter over-ride procedures. Note that, since the callback functions are static class members, we recover the per-object context via a call to RifGetCurrentPlugin. The key is to look for changes to the RenderMan attribute "identifier:name". These come about when the attribute stack is popped or when the attribute is set. These procedures manage the validity stack and apply the pattern matcher, regexec, when needed. Whenever the validity state changes the filter sets its filtering mode accordingly, and since we have not implemented any other Ri requests, they are subject to this filtering mode.

	RtVoid
	matcher::myAttributeBegin()
	{
		matcher *obj = static_cast<matcher *>
		    ( RifGetCurrentPlugin() ); // find matcher object

		obj->m_validityStack.push(obj->m_validityStack.top()); // push the current validity
		RiAttributeBegin(); // pass the RiAttribute call down the chain
	}

	RtVoid
	matcher::myAttributeEnd()
	{
		matcher *obj = static_cast<matcher *> ( RifGetCurrentPlugin() );

		obj->m_validityStack.pop();
		if (obj->m_validityStack.top()) // the validity may have changed
			obj->m_rifFilter.Filtering = RifFilter::k_Continue;
		else
			obj->m_rifFilter.Filtering = RifFilter::k_Terminate;
		RiAttributeEnd(); // pass the call down the chain
	}

	RtVoid
	matcher::myAttributeV(RtToken nm, RtInt n, RtToken tokens[], RtPointer parms[])
	{
		if( !strcmp(nm, "identifier") && !strcmp(tokens[0], "name") )
		{
			matcher *obj = static_cast<matcher *> ( RifGetCurrentPlugin() );
			obj->setName( ((RtString *) parms[0])[0] );
		}
		RiAttributeV(nm, n, tokens, parms); // pass the call down the chain
	}

	void
	matcher::setName(const char *nm) // name has changed, check for a pattern match
	{
		bool match;
		if( 0 == regexec( &m_regexp, nm, 1, 0, 0) )
			match = true;
		else
			match = false;
		m_validityStack.top() = m_invert ? !match : match;
		if(m_validityStack.top())
			m_rifFilter.Filtering = RifFilter::k_Continue;
		else
			m_rifFilter.Filtering = RifFilter::k_Terminate;
	}


Using Ri Filters

Now we are ready to run our filter. Ri filter chains must be described outside of the RIB format so we have added some new command-line options to PRMan to support them.

	% prman -rif matcher.so -rifargs -pat ^her -rifend example.rib

This invocation identifies the location of the Ri filter plug-in and establishes the instantiation arguments for this run of the filter. Here we invoke a single Ri filter but we can easily chain them by supplying multiple -rif blocks. The ordering of the -rif blocks on the command line determines the placement of each filter in the filter chain. Depending on the behavior of your collection of filters, dramatically different results can be obtained simply by reordering the filter chain.

Compared to ad-hoc or home-grown solutions for RIB filtering, this new plug-in type offers a number of advantages. First, the RIB parsing problem is eliminated. The native RIB parser built into the renderer does all the heavy lifting. More importantly, because the filters are expressed as plug-ins, there is no need to put the results onto disk. This virtually eliminates the run-time and disk space costs associated with most RIB filters.

There are times when it is desireable to store the filtered RIB stream rather than render it. This turns out to be simple.

	% catrib -o filtered.rib -rif matcher.so -rifargs -pat ^her -rifend example.rib

Now the filtered RIB will magically appear in filtered.rib, and will not be rendered. For this version of example.rib:

	FrameBegin 1
	Display "test.tif" "tiff" "rgba"
	Format 512 512 1
	Projection "perspective" "fov" [45]
	Translate 0 0 10
	WorldBegin
	AttributeBegin
	Attribute "identifier" "name" "my sphere"
	Sphere .5 -.5 .5 360
		"constant float foo" [.1]
		"varying float bar" [0 1 2 3]
		"varying float[2] bar2" [0 0 1 1 2 2 3 3]
	Attribute "identifier" "name" "his cones"
	Cone 1 .5 360
	AttributeBegin
	Attribute "identifier" "name" "her cylinder and cone"
	Cylinder .5 -1 1 360
	Cone 2 .5 360
	AttributeEnd
	Cone 3 .5 360
	AttributeEnd
	Cone 4 .5 360
	Polygon "P" [0 0 0 1 1 1 2 2 2]
	WorldEnd
	FrameEnd

This is the result of the above filtering:

	##RenderMan RIB
	version 3.04
	FrameBegin 1
	Display "test.tif" "tiff" "rgba"
	Format 512 512 1
	Projection "perspective" "fov" [45]
	Translate 0 0 10
	WorldBegin
	AttributeBegin
	Attribute "identifier" "name" ["my sphere"]
	Attribute "identifier" "name" ["his cones"]
	AttributeBegin
	Attribute "identifier" "name" ["her cylinder and cone"]
	Cylinder 0.5 -1 1 360
	Cone 2 0.5 360
	AttributeEnd
	AttributeEnd
	Cone 4 .5 360
	Polygon "P" [0 0 0 1 1 1 2 2 2]
	WorldEnd
	FrameEnd

Note the disappearance of “my sphere” and “his cones” because they did not match the pattern. The matcher plugin is a useful utility that has many of the properties desirable of Ri filter plug-ins. First, it is a very compact plug-in with a tiny, easily maintained codebase. Second, its functionality is easy to understand and thus its presence in a chain of filters should offer no surprising side effects. You can even use multiple instances of matcher in a pipeline. As an exercise, run the following snippet to see how many geometric primitives result.

	% catrib -rif matcher.so -rifargs -pat cone -rifend \
		-rif matcher.so -rifargs -pat ^her -inv -rifend
In case your imagination hasn't already run wild, here are a few potential Ri filter applications to consider:


The Joy of Rx

The example above, while compact, has to keep track of its own AttributeBegin and AttributeEnd stack and note when an Attribute "identifier:name" matches. Fortunately RenderMan is already keeping track of the Attribute state for us. We can simply use the RxAttribute() call to query the "identifier:name" of a piece of geometry and determine its fate based on a match.

Let's say we instead want a filter that simply removes all spheres that have a name that matches the given pattern. We could re-write the matcher plug-in above and remove call backs to the Attribute functions and instead just add a single call back for RiSphere. We also set the default pass through mode to k_Continue.

        matcher::matcher(const char *expr, bool invert) :
                m_invert(invert),
                m_expression(expr)
        {
                // First we initialize our RifFilter structs with the (static) class methods
                m_rifFilter.SphereV = mySphereV;
                m_rifFilter.Filtering = RifFilter::k_Continue;

                // here we initialize other state and compile the regular expression
                int err = regcomp(&m_regexp, expr, REG_EXTENDED|REG_NOSUB);
        }

Now we just need an implementation for our Sphere callback.

        RtVoid
        matcher::mySphereV(RtFloat radius, RtFloat zmin, RtFloat zmax, RtFloat tmax,
                                RtInt n, RtToken tokens[], RtPointer parms[])
        {
                char **sphereName = (char **)alloca(1);
                RxInfoType_t type;
                RtInt resultcnt, size;
                matcher *obj = static_cast<matcher *>( RifGetCurrentPlugin() );

                // Ask PRMan for the identifier:name
                size = RxAttribute("identifier:name",sphereName,(sizeof(char*)),&type,&resultcnt);
                if ( !size && (0 == regexec ( &obj->m_regexp, *sphereName, 1, 0, 0)) )
                        RiSphereV(radius, zmin, zmax, tmax, n, tokens, parms);
        }

If we were to run this example on the RIB stream below, only "his sphere" would render and "her sphere" would be removed.

	FrameBegin 1
	Display "test.tif" "tiff" "rgba"
	Format 512 512 1
	Projection "perspective" "fov" [45]
	Translate 0 0 10
	WorldBegin
	AttributeBegin
	Attribute "identifier" "name" "his sphere"
	Color [0 0 1]
	Sphere .5 -.5 .5 360
		"constant float foo" [.1]
		"varying float bar" [0 1 2 3]
		"varying float[2] bar2" [0 0 1 1 2 2 3 3]
	AttributeEnd
	AttributeBegin
	Attribute "identifier" "name" "her sphere"
	Color [1 0 0]
	Sphere .9 -.9 .9 360
	AttributeEnd
	WorldEnd
	FrameEnd

However, if we want the result of this filter written to a file, we cannot use the catrib program anymore. Instead we have to now use the -catrib flag to the prman program, because only prman can keep track of the graphics state. The following is the result of running:

	% prman -catrib filtered.rib -rif matcher.so -rifargs -pat ^his -rifend example.rib

Once again, the filtered RIB will magically appear in filtered.rib and not be rendered, even though it will have maintained the correct graphics state:

	##RenderMan RIB
	version 3.04
	FrameBegin 1
	Display "test.tif" "tiff" "rgba"
	Format 512 512 1
	Projection "perspective" "fov" [45]
	Translate 0 0 10
	WorldBegin
	AttributeBegin
	Attribute "identifier" "name" "his sphere"
	Color [0 0 1]
	Sphere .5 -.5 .5 360
		"constant float foo" [.1]
		"varying float bar" [0 1 2 3]
		"varying float[2] bar2" [0 0 1 1 2 2 3 3]
	AttributeEnd
	AttributeBegin
	Attribute "identifier" "name" "her sphere"
	Color [1 0 0]
	AttributeEnd
	WorldEnd
	FrameEnd


Parsing Parameters

One thing developers of Ri filter plugins may want is to alter the parameter list of an RI call or modify the behavior of the current filter based on the contents of a parameter list. Fortunately there is a nice helper function that can be used to do this, RifGetDeclaration(). As an example, the Sphere matcher could examine the additional arguments to the Sphere call as follows:

        RtVoid
        matcher::mySphereV(RtFloat radius, RtFloat zmin, RtFloat zmax, RtFloat tmax,
                                RtInt n, RtToken tokens[], RtPointer parms[])
        {
                static char* types[] = {
                              "float", 
			      "point",
                              "color",
                              "integer",
                              "string",
                              "vector",
                              "normal",
                              "hpoint",
                              "matrix",
                              "mpoint" };
                static char* details[] = {
			     "constant",
                             "uniform",
                             "varying",
                             "vertex",
                             "varying"};		


		/* Echo parameter list as a comment into the RIB stream*/
		for(int i=0;i<n;i++)
		{
			RifTokenType type;
			RifTokenDetail detail;
			int arraylen;
			if( 0 == RifGetDeclaration(tokens[i], &type, &detail, &arraylen) )
			{
				RiArchiveRecord(RI_COMMENT, "\t%s: %s %s[%d]", 
					tokens[i], details[(int) detail], 
					types[(int) type], arraylen);
			}
    		}
        }


Archives and Procedurals

One of the nice features about the new Ri filters is that they work even if your RIB stream is composed of Procedurals and Archives, be they file based or in-line. In other words, all RIB that passes through to the renderer will also pass through the Ri filter pipeline.

Using our sphere matcher example above, the following RIB below:

	FrameBegin 1
	Display "test.tif" "tiff" "rgba"
	Format 512 512 1
	Projection "perspective" "fov" [45]
	Translate 0 0 10
	ArchiveBegin "thisworks"
	Sphere .5 -.5 .5 360
		"constant float foo" [.1]
		"varying float bar" [0 1 2 3]
		"varying float[2] bar2" [0 0 1 1 2 2 3 3]
	ArchiveEnd
	WorldBegin
	AttributeBegin
	Attribute "identifier" "name" "his sphere"
	Color [0 0 1]
	ReadArchive "thisworks"
	AttributeEnd
	AttributeBegin
	Attribute "identifier" "name" "her sphere"
	Color [1 0 0]
	Sphere .9 -.9 .9 360
	AttributeEnd
	WorldEnd
	FrameEnd
will become this resulting RIB:
	##RenderMan RIB
	version 3.04
	FrameBegin 1
	Display "test.tif" "tiff" "rgba"
	Format 512 512 1
	Projection "perspective" "fov" [45]
	Translate 0 0 10
	WorldBegin
	AttributeBegin
	Attribute "identifier" "name" "his sphere"
	Color [0 0 1]
	#RenderMan RIB
	Sphere .5 -.5 .5 360
		"constant float foo" [.1]
		"varying float bar" [0 1 2 3]
		"varying float[2] bar2" [0 0 1 1 2 2 3 3]
	AttributeEnd
	AttributeBegin
	Attribute "identifier" "name" "her sphere"
	Color [1 0 0]
	AttributeEnd
	WorldEnd
	FrameEnd

 

 

Pixar Animation Studios
(510) 752-3000 (voice)  (510) 752-3151 (fax)
Copyright © 1996- Pixar. All rights reserved.
RenderMan® is a registered trademark of Pixar.