Project BatCave: Data Mining and Process Control


What is Project BatCave?

Starting with RenderMan Artist Tools Version 6.5, Alfred and alfserver processes can log information into a MySQL database. Support for additional databases begins with RAT 7.0 with support for PostgreSQL. This real-time information can be used to keep track of the active jobs in your render farm more easily, or to go back and retrieve historical data for analysis. Project BatCave includes the MySQL components as well as a Web-based interface enabling you to access all of that information in one place. The interface offers simple views of the data in tables and graphs as well as an “expert” mode that accommodates customized SQL queries to adapt the BatCave to meet your site's unique needs.


Prerequisites

The following components are required by the BatCave, but are not part of this distribution:

 


Configuring the BatCave

Prior to use, the BatCave must be configured.  This requires properly setting up the MySQL database and configuring the Web server used to view the data.


Entering the BatCave


Database Details

The BatCave makes use of two database connections.  The primary database contains the live Alfred and alfserver data, using the following tables:

Alfred Live Data Tables
Table Name Purpose
Job Contains information for every Job started by a dispatcher.  The dispatcher host creates this entry and maintains it as information in the job changes.
Task Contains one entry for every Task.  The Task table includes the Job ID (“jid”) from the Job table, so that tasks can be related to jobs.  The dispatcher is responsible for creating the Task entries and updating information.
Cmd This table is used for historical analysis and contains a history of all executed commands.
Slot Contains one entry for every Slot in the alfred.schedule file.  The maitred creates this entry and is responsible for keeping it current.
Machine The Machine table contains one entry for every Machine, rather than one for every slot.  It is maintained by the alfserver and contains information associated with current metrics, operating system, et cetera.
Maitred The Maitred table contains information regarding any maitred that has been started.  As a maitred is started it registers with this table, and it updates information during operation as needed.
Dispatcher This table contains information on Alfred dispatchers.  An entry is created when a dispatcher starts and updates are made as needed. Information in this table is used to connect to the Alfred dispatcher HTTP port, or to send job control commands to the dispatcher.
MetricDef This table contains Metrics definitions. These entries are generally created by the Maitred, however, in the case where metrics are defined in an alfserver, the alfserver can create these entries as well.
Metrics The Metrics table collects all broadcast metrics.  These entries are entered by the alfserver and are automatically time-stamped to allow later visualization of metric information.
LimitDef This table contains limit definitions.  The limitid returned from this table is used by dispatchers to update the Limits table as Tasks are executed.
Limits The Limits table contains a current count of specific limits, by dispatcher (user & host). The summation of all named limits gives the current global usage.
StateName This table is used for reference only.  It contains English text of status used by the various components.

 

HTML Templates and Queries Tables
Table Name Purpose
Templates Data from the various tables above are displayed through HTML templates that define the format of the data to be displayed. 
Queries All access to the BatCave data are through queries stored in this table.  The Queries table also contains information about the php script and the templates used to display data retrieved by the query.

 

Pixar Animation Studios
(510) 752-3000 (voice)   (510) 752-3151 (fax)
Copyright © 1996- Pixar. All rights reserved.
RenderMan® is a registered trademark of Pixar.