- Codename "Project BatCave"
Alfred MySQL Logging and Web-based Database Browsing
The project codenamed "BatCave" is an effort to export alfred status
and task execution history to a generic database in a form which allows
people to develop monitoring and analysis tools which are specific
to their needs and interests.
Alfred components, such as the dispatcher, maitre-d, and alfserver
have been modified to (optionally) log various data and events directly
to a MySQL database.
The database provides a highly programmable environment for creating
custom status queries and for doing historical analysis. The database
server can also offload the live status reporting/formatting function
performed by the maitre-d in "watch-servers" mode, which can improve
the maitre-d performance at large sites.
A php-based web interface for viewing the records in this SQL database
is also shipped as part of the project. The web interface provides a
broad range of basic functions and is fully customizable, so new
site-specific queries and reports can easily be added.
Several new alfred.ini settings control the basic database logging
capability. Bootstrap scripts are also provided for creating the
database tables and the web interface.
This capability has broad implications for integrating alfred with
existing production databases, as well as with site control and
planning projects. Alfred features in this area will continue to
grow and evolve from the initial groundwork being released here.
The bulk of the new features, and all of the database and php setup scripts,
as well as their documentation, are currently part of the "RenderMan Artist
Tools" 6.5 release. Please contact
Pixar Customer Support if you need these materials in an alfserver-only
environment.
- New and Different settings in alfserver.ini
Alfred administrators are encouraged to browse through the
new alfserver.ini and perhaps "diff" it with the one
currently in use. There have been several changes and additions
which may be relevant.
Alfserver and the alfred maitre-d "discover" each other on the network
using multicast packets addressed to a particular multicast "session"
address.
Having found each other, alfservers then deliver periodic status
updates called metrics to the maitre-d (and now also to a MySQL
database, see the BatCave discussion above). These metrics are
used as a basic measure of server health and the values can be
used to make specific server assignment decisions. Starting with
Alfred 6.5, metrics are reported to the maitre-d using point-to-point
"unicast" udp packets. In previous releases the metrics were also
multicast back to the discovery address, for use by potentially many
interested listeners. The unicast approach can reduce some network
overhead at large sites, especially in situations where the one-to-many
nature of multicast traffic causes problems for smart network
switches that try to optimize one-to-one communications.
Routers on the network ensure that the mulicast discovery messages
are delivered to all "subscribed" systems. By default, alfred and
alfserver use the multicast "session" address 239.255.224.99, port
9002/udp. Sites can change this multicast address by adding the
hostname "alf-status" to the site nameserver (e.g. DNS, NIS,
/etc/hosts, etc), and picking a new multicast address for it from
the multicast range (224.0.0.0 - 239.255.255.255). Note that there
are IANA numbering conventions which apply to multicast addresses.
Alternatively, conventional "unicast" communications can be used
for both discovery as well as metrics delivery. This is done by
simply adding "alf-status" as a hostname alias for the maitre-d
host's regular IP address, rather than using a multicast address.
This approach is actually a way to bypass the "discovery" phase.
The alfserver metrics will be sent as standard UDP packets directly
to the named maitre-d. Note that this approach should not be used
with fallback maitre-ds, since alfservers would only know about
the one named host, and metrics would only be delivered to that
one host.
A new alfserver configuration setting, "metricsDelivery" can now
be set to "multicast" to force metrics to be sent to the multicast
address (as in releases prior to 6.5), so that any other interested
listeners can receive them simultaneously.
There is also a new way to deliver configuration overrides to all
alfservers from the maitre-d: Create a file called
$RATTREE/etc/alfsite.ini containing the overrides in
an ini location accessible to the maitre-d, its
contents will be sent by the maitre-d to the alfservers as part of
the discovery process, along with the site metrics definitions.
- Several problems were fixed related to the optional RMANCONFIG
environment variable and how it is used to find rendermn.ini settings,
for use by alfserver.
-
The "envkey" handlers in alfserver.ini that select
an appropriate RMANTREE and RATTREE have been upgraded to match
the keys sent by default installs of MTOR. Also, the sanity
checks applied to the selected trees now properly check for
Windows-style application names.
-
On Mac OS X, the automatic DYLD_LIBRARY_PATH setting chosen for
Maya plug-ins in alfserver.ini has been improved to be more
compatible with newer Maya releases.
-
A problem that caused Windows Alfservers in unicast mode to
sometimes cease delivering metrics after a maitre-d restart
has been addressed.
-
Some duplicate metrics unicast/multicast settings have been
removed from alfserver.ini.
-
Delivery of "alfsite.ini" configuration overrides from the
Alfred maitre-d has been improved and made more robust. Also,
a problem causing Alfserver to sometimes repeatedly request
alfsite.ini from the maitre-d has been fixed.
-
A potentially serious bug has been fixed regarding the way in
which very long metrics definitions are buffered during transmission
to Alfservers.
-
A problem with the handling of the alfserver.ini "signal
escalation delay" (timerAlfserverSignalEscalation)
setting has been fixed. This is the delay, in seconds, between
applying increasingly severe shutdown methods to processes launched
by Alfserver, typically because the associated Alfred job is deleted.
Previously the delay was very short, no matter what setting was
given, and some applications were unable to shut down cleanly in
the interval.
- There are several minimum version requirements for the servers
used for the "BatCave" functions. In particular, MySQL version
4.1 or higher is needed. Also a relatively recent PHP package needs to be
installed and be accessible in your site web server; php needs the MySQL
extensions, and
the "preg - the perl regular expression" extension, as well as
the "gd - image drawing" extension.
- Alfserver support for scriptable, key-based, per-command,
environment configuration was recently extended to include
support for "netrender -R key ..." This includes the ability
to select which user will own the resulting prman process.
Currently, the alfserver ownership mode "login" is not a viable option
for netrender connections. This is because netrender and prman exchange
some of their data on sockets which are connected to stdin and stdout;
these connections do not survive the login set-up. The alternative
"setuid" mode works as expected, and is frequently a better choice
anyway, from an administrative point of view. Note that "login"
continues to be supported for RemoteCmd usage, although again,
"setuid" is usually the more manageable choice.
|