Mastering LW ScreamerNet

1 [2] 3 4 5 6 7 8 9

LightWave’s Config Files are plain text files that store basic configuration information for LightWave and ScreamerNet. It’s very important that you know where these config files are located so that both LightWave and ScreamerNet can access them. It’s also important to understand which ScreamerNet relevant settings are stored in the config file, and how to use them.

• Updating LightWave’s Config Files
• LightWave’s Config File Path on Mac OS X
• Creating a New Set of Config Files
• Config Settings for LWSN
  • Content Directory
  • Command Directory
  • Default Segment Memory
  • Multithreading

Updating LightWave’s Config Files

LightWave only updates the config file when you quit the program. Therefore anytime you make changes to any of the config settings (other than menu or shortcut changes which are saved when you close the respective editor) you must be sure to quit LightWave to ensure that the changes are written to the config file itself. ScreamerNet only reads the config file when first launched. Therefore, if you wish to make changes to the config file for ScreamerNet and you are using a single config file for both Lightwave and ScreamerNet, you should perform the following steps:

1. Quit LWSN (or ScreamerNet UB).
   (Press command-period, before using command-Q if you’re using LW7.5, or LWSN will crash. This bug has been fixed in LW8.0 and beyond.)
2. Launch LightWave Layout.
3. Make changes to the desired config settings in LightWave, as discussed later.
4. Quit LightWave to write the changes out to the config file.
5. Re-launch LWSN (or ScreamerNet UB) so it reads the updated config file.
6. Re-launch LightWave if using the built-in network controller to manage ScreamerNet.

LightWave’s Config File Path on Mac OS X

Starting with LightWave 9.3 there are two different versions of LightWave that run on the Macintosh.

There’s the original LightWave PPC (PowerPC) for the PowerPC Macs such as the G4 and G5. This older version is also sometimes referred to as LightWave CFM (Code Fragment Manager, the legacy compatibility API it’s written for on Mac OS X) or LightWave CW (Code Warrior, the original development environment it was written in).

Now there’s the newer LightWave UB which is a universal binary optimized for both the PPC as well as the newer Intel based Macs. This new version is developed in Apple’s XCode and will eventually replace the older LightWave PPC entirely. They both currently ship however because plugins must be complied differently to work with the newer LightWave UB than they were for the older LightWave PPC. So in order to use older PPC plugins that have not yet been updated to UB versions, you can continue to use the older LightWave PPC.

Each version has its own set of config files stored in the current user’s preferences folder.

On Mac OS X the default path of the LW PPC config files is:

":Users:username:Library:Preferences"

and the default path of the LW UB config files is:

"/Users/username/Library/Preferences/LightWave3D/"

Replace username with your actual user name.

NOTE: Specify all paths for LightWave PPC using the Macintosh path separator ":" (colon), not the Unix separator "/" (forward slash) nor the DOS separator "\" (back slash). Specify all paths for LightWave UB using the Unix separator "/" (forward slash).

Keep in mind that, by default, the config files are stored in the Library:Preferences folder that’s inside your home folder, not in the Library:Preferences folder at the top level your system hard drive.

The config files for LW 9 PPC on Mac OS X are in the Preferences folder and named as follows:

LightWave Extensions 9 Prefs : Complete list of file paths to all available plug-ins

LightWave Hub 9 Prefs : Basic Hub Settings

LightWave Layout 9 Prefs : Basic Layout Settings

LightWave Modeler 9 Prefs : Basic Modeler Settings

The config files for LW 9 UB are in the Preferences/LightWave3D folder and named as follows:

Extensions 9 : List of file paths to user added plug-ins

Hub 9 : Basic Hub Settings

Layout 9 : Basic Layout Settings

Modeler 9 : Basic Modeler Settings

You don’t need to specify any of these config files by name, only the path to the directory where they are stored, and that’s only if you decide to keep your config files in a location other than the default. If you leave the config files in their default location, you do not need to specify their path for LightWave, Modeler, the Hub, or for LWSN/ScreamerNet. In this case each individual Mac OS X user would also have their own configs, since they would be stored in each user’s home folder.

You may wish to store your configs in different locations in some advanced situations, such as:

• When running different versions of LightWave on the same machine.
• When different users use the same Mac and you all wish to use a single set of configs.
• When running ScreamerNet over a network with a common set of configs for all nodes.
• When running ScreamerNet over a network and using separate configs for machines with varying resources, such as available RAM or number of processors/cores.

The easiest way to redirect the ScreamerNet UB config file is to use DLI_SNUB-Launcher, DreamLight Interactive’s ScreamerNet Universal Binary Launcher, an XCode Aqua GUI front end that interactively builds the necessary command lines to launch LightWave 3D’s ScreamerNet UB standalone, batch and network rendering command line component. DLI_SNUB-Launcher’s interactive interface makes configuring and launching LightWave 3D’s ScreamerNet UB much more user friendly and far less error prone than doing so manually. Download the FREE LITE VERSION today!

Creating a New Set of Config Files

The following steps will show you how you can easily create a fresh new set of config files for LightWave 9 PPC in a new location if desired.

1. Locate your LightWave installation, typically:
YourHD:Applications:NewTek:LightWave 3D 9.3
2. Inside the LightWave 3D 9.3 directory, create a new directory named Configs.
3. Locate the LightWave cmdLine file, typically installed in: "YourHD:Applications:NewTek:LightWave 3D 9.3:Programs"
4. Double click it, to open it in TextEdit.
5. Type the config parameter -c immediately followed by the full path to the new Configs directory, enclosed in quotes, as follows:
   -c"YourHD:Applications:NewTek:LightWave 3D 9.3:Configs"
   NOTE: Do not put a space between the config parameter -c and the config path. Also make sure you type the parameter -c using a lowercase c.
6. Open TextEdit->Preferences, Saving and turn OFF: Append ".txt" to plain text files. If there is a ".txt" extension added to the end of the LightWave cmdLine file name, it unfortunately won’t be recognized by LightWave.
   
7. Save the LightWave cmdLine file.
8. Copy and paste the config parameter, -c"YourHD:Applications:NewTek:LightWave 3D 9.3:Configs", into the Modeler cmdLine file and the Hub cmdLine file and save each of those files as well.
9. Launch & Quit Layout so that Layout creates its config files.
10. Launch & Quit Modeler so that Modeler creates its config file.
11. Quit the Hub so that it creates its config file.

Starting with LightWave version 9, included plugins are automatically scanned when LightWave is first run. If you are setting up a previous version of LightWave or wish to add custom third-party plugins then follow these additional steps.

1. Launch LightWave.
2. Open the Edit Plug-ins panel with Utilities->Plugins->Edit Plugins...
3. Click Scan Directory, & Choose your plugins directory or third-party plugins directory.

12. Close the Edit Plug-ins panel by clicking Done.

13. Quit LightWave, which will write out a new set of config files in the new Configs directory.

If you run into various problems with LightWave and suspect corrupted config files, you can delete them and use the above steps to recreate a fresh new set of config files in their current location.

As of the initial LightWave 9.3 UB, redirecting the UB config files for Layout, Modeler and/or Hub is not yet recommended by NewTek, though it is possible by launching LightWave UB via the Terminal or through an AppleScript. Later in this tutorial I will cover how to redirect the ScreamerNet UB configs since ScreamerNet UB is intended to be run from the Mac OS X Terminal.

If you need to create a fresh new set of LightWave UB preferences you can simply delete or move the "/Users/username/Library/Preferences/LightWave3D/" folder and LightWave UB will create a new one the next time you launch and quit it. To make a set of UB config files specifically for ScreamerNet UB, simply copy the "/Users/username/Library/Preferences/LightWave3D/" folder to a new location and give it a new name, something like: ScreamerNetConfigs.

Config Settings for ScreamerNet

The following settings are stored in the LightWave Layout 9 Prefs file and are used by ScreamerNet: Content Directory, Command Directory, Default Segment Memory, and Multithreading.

Content Directory

LightWave 9 PPC sample content directory entry in config file:

ContentDirectory YourHD:Applications:NewTek:LightWave 3D 9.3:Content:

LightWave 9 UB sample content directory entry in config file:

ContentDirectory /Users/username/Desktop/Content/

Along with a full compliment of paths to the Content Directory’s special subdirectories, such as Images, Objects, Scenes, etc. If you wish to use ScreamerNet successfully you must learn to properly use LightWave’s Content Directory structure to package your projects correctly.

A Content Directory in LightWave is very similar to a Web site’s root directory. When used properly, all necessary file paths are specified relative to the root directory, and all files are located in subdirectories of this root directory. This means the root directory is a self contained unit that may be moved as desired, including to other machines, without breaking all the dependent file/directory links.

In LightWave, the Content Directory itself may be named whatever you like, but there are some specific subdirectories that LightWave expects to find inside the Content Directory that you should not rename or move. These special subdirectories are named Images, Scenes & Objects. They are where you should store all your image, scene and object files respectively, for a project. You may name your files anything you wish (being sure to use proper file extensions), as long as you place them in the proper subdirectories.

There are two approaches to effectively organize the Content Directory. Using a separate Content Directory for each project, or using a single Content Director for all your projects.

Separate Content Directory per Project

If your projects don’t typically share assets and you don’t mind changing the Content Directory for each project you work on then you can keep each project in it’s own self-contained Content Directory. In this case simple put all your project’s objects in an Objects directory, all your projects images in an Images directory and all your project’s scenes in a Scenes directory within a project folder.

Here’s an example of a very simple separate Content Directory for a single project:

Shared Content Directory for all Projects

If your projects typically share assets or changing the Content Directory often would be inconvenient, such as when multiple users or render nodes all point to a common shared Content Directory, then you can organize a single Content Directory for all your projects to share. The trick to organizing your Content Directory in this manner is to make separate folders for each project inside the Images, Objects and Scenes folders. Those project specific sub folders may be named anything you like, but the enclosing folders at the top level of the Content Directory must be named Images, Objects and Scenes, unless you explicitly changed them in LightWave’s path preferences panel.

Here’s an example of a single Content Directory that is shared by multiple projects:

Setting the Content Directory in the Config File

The ContentDirectory entry is not actually written to the config file until you Quit LightWave after specifically setting a Content Directory or by allowing LightWave to automatically change the Content Directory when loading a scene file. You may explicitly set a Content Directory as follows.

1. Launch LightWave
2. Select: Edit->Set Content Directory...
   & Choose your desired Content Directory.
3. Quit LightWave, to save the config file.

Content Directory 128 Character Path Length BUG

In LightWave 7.5, if you are specifying your Content directory through LightWave and the Config file, rather than hard coding it into the LWSN command line, then you should make sure the path to the Content directory is short enough that no paths, for objects, images, etc. will be longer than 128 characters in total. If they are 128 characters or longer, LWSN may hang or crash when loading the scene, unless you either shorten the path by moving your Content directory higher up in the hierarchy (such as moving it to the desktop) or hard code the Content path directly into the LWSN cmdLine file. This bug has been fixed in LightWave 8.0.

Command Directory

LightWave 9 PPC sample command directory entry in config file:

CommandDirectory YourHD:Applications:NewTek:LightWave 3D 9.3:Programs:

LightWave 9 UB sample command directory entry in config file:

CommandDirectory /Users/username/Desktop/Content/Commands/

Believe it or not, LightWave really doesn’t do any “network” rendering. Neither LightWave nor ScreamerNet really has any idea that it may be communicating over a network or even that any network even exists. All LightWave does is write simple commands into a job command text file in a common command directory. Then each ScreamerNet render node simply reads those commands from the job text files, executes the commands and writes a reply into an ack (acknowledgment) file in the same directory. What makes this work over a network is setting it up so that each machine has access to the same command directory.

The Command Directory is the directory that contains the job command file and the acknowledgement file. These are two plain text files that the network controller uses to communicate with an instance of LWSN PPC or ScreamerNet UB during batch or network rendering.

The communication goes like this:

1. The network controller creates a job file.
2. The network controller writes a command into the job file.
3. LWSN PPC or ScreamerNet UB reads the command from the job file.
4. LWSN PPC or ScreamerNet UB attempts to perform the command.
5. LWSN PPC or ScreamerNet UB creates an acknowledgment file.
6. LWSN PPC or ScreamerNet UB writes a reply to the acknowledgement file.
7. The controller reads the reply from the acknowledgement file to decide what’s next.

These files are named: job# and ack# where # is replaced with the number of the LWSN node to control. If only one instance of LWSN PPC or ScreamerNet UB is being used these files would be named job1 and ack1. For a second instance of LWSN PPC or ScreamerNet UB, they would be named job2 and ack2, etc. As in config and cmdLine files, no ".txt" extension is used. Also note that there is no leading zero, space or anything else between the number and the words job or ack.

LightWave PPC defaults the Command Directory to the LightWave Programs Directory. LWSN on Mac OS X however uses the Content directory as the default Command Directory. LightWave UB defaults the Command Directory to the root directory (which is not a recommended place for such files on Mac OS X). This means ScreamerNet will not run as configured with the defaults, you must at least change one of these settings.

Both the user running the network controller (typically LightWave or a third party controller) and the user running LWSN PPC or ScreamerNet UB (if different) must have read/write access to the same Command Directory. Non-administrator users don’t typically have write access to the Applications directories on Mac OS X. Therefore, I prefer to use a shared Content Directory, with read/write access for all users, and keep everything relative to this shared Content Directory, including my ScreamerNet Command Directory.

I typically structure my Content Directories, with their own Commands directory like this:

Here’s a step-by-step example of how to configure such a simple self contained Content Directory with its own Command Directory.

1. Launch LightWave.
2. Set LightWave’s Content Directory with: Edit->Set Content Directory...
   to your desired Content directory.
3. Select: Render->Network Render
4. Click the Command Directory button.

   

5. Navigate to the same Content Directory you set in step 2.
6. Create a New Folder named Commands inside the Content Directory.
7. Click the Choose button to close the dialog and accept the changes.
8. If asked to initialize the ScreamerNet, click No.
9. Quit LightWave to save your changes to the config file.
10. When creating the LWSN PPC cmdLine files (fully explained later) use the following Content relative command file paths, replacing # with the number of the LWSN node:
    ":Commands:job#" ":Commands:ack#"

Default Segment Memory

DefaultSegmentMemory 32000000

This is the maximum number of bytes to use for the rendered image segment memory. It defaults to 32 million bytes which is just under 32 Megabytes. This only affects the memory allocated for image buffers used in rendering the image itself, it doesn’t affect the amount of RAM used to load objects, textures etc. The number you set is the maximum upper limit. LightWave will only use as much RAM as it needs for the actual render buffers, up to this limit. If more memory is needed, LightWave will break the image up into as many segments as necessary so that each fits within the segment memory limit. In general you want this number set high enough for an entire frame to be rendered within one segment, as long as it fits comfortably within your machine’s physical RAM capacity, while still leaving enough physical RAM free for objects, textures etc.

This value is set in the Render Globals panel (typically while a scene is open) but is stored in the Config file, only if you say yes when asked to use the changed value as a new default. It is never stored in the scene file, and is not stored anywhere if you click no when asked to use the changed value as a new default. In that case it will only affect a render in the current session of LightWave itself until you quit LightWave. The next time you launch LightWave the old value in the config file will be reinstated.

Here’s a simple step-by-step example of how to set the Default Segment Memory

1. Launch LightWave.
2. Open the Render->Options->Render Globals panel.
3. Set the resolution to whatever you wish, but beware this is the global resolution and while it can be used to calculate the proper segment memory limit, it won't be used by the camera unless you set the camera to use the global resolution.
4. Notice the readout directly under the Segment Memory Limit button.

   

5. If it reads Segments:1 you don’t need to change it, otherwise continue on...
6. Click the Segment Memory Limit button.
7. The Segment Memory Limit dialog will open, filled with the current DefaultSegmentMemory value from the config file.
   
   (FUN FACT: This number is in millions of bytes, not megabytes. The rounded integer shown is simply multiplied by 1,000,000 when stored in the config file. A true Megabyte of RAM is actually 1024*1024 = 1,048,576 bytes. )
8. Enter an integer that is equal to the old number multiplied by the number X that was displayed in the readout Segments:X from step 5. For instance, if the old value was 32, and the readout said: Segments:2, you could enter 64 into the field, to be sure to be able to render the entire frame in one segment.
9. Click the OK button.
10. You will now see a dialog that asks: "Should this value become the new default?" Yes/No. You must click the Yes button, or the new value will not be stored anywhere and will only persist during this session of LightWave until you quit LightWave.
11. The readout below the Segment Memory Limit button should now read: Segments:1
12. Quit LightWave to actually save the change to the config file.

You may also simply locate and edit the line DefaultSegmentMemory 32000000 in your config file directly in a text editor. Simply change the leading integer from 32 to whatever you desire (followed by six zeros) and save the config file. Make sure your text editor keeps the file as a plain text file and doesn’t add any file extension to the config’s filename.

Multithreading

RenderThreads 1

This is the number of concurrent threads (strings of commands) to execute during rendering. This setting is most often used on multiprocessor and/or multi-core machines, but in some instances may even speed up single processor rendering. It’s best to run various tests on your particular scenes and machines to determine the best possible settings in your case. In LW9 Layout, it’s best to set the render threads to the number of CPU’s or cores available on the render machine when using any of the new LW9 cameras other than the old classic camera. When setting render threads higher than 1, be sure to check test renders for any problems. Some older plug-ins may not work properly with multiple threads, and may crash. In LW9 the render threads setting initially defaults to the number of CPU’s or cores available but you may need to set it manually when setting up additional render nodes. Current tests show that ScreamerNet 9.3 UB. is not handling the threads the same way Layout 9.3.1 UB does, it’s actually rendering on one more than the number set in the config, so run tests to find the best settings for your particular situation.

Here are some benchmarks using various thread settings on various Mac’s.

The following times are for rendering frames 1-16 of the LW8Content/Scenes/Benchmark/Teapot.lws scene. I’ve changed the camera to the perspective camera and set 1 pass of adaptive sampling at 0.1.

The current ScreamerNet 9.3.1 UB appears to have a few problems with the way it handles render threads. Even when set to 1 thread, it appears to render on 2 threads according to the Activity Monitor. Each thread setting appears to be using one more full thread in Activity Monitor.

Below are the same tests run on an older 2GHz Dual Processor PPC G5 PowerMac which has two single core processors. The best results when rendering directly in LightWave 9.3.1 UB are with 2 threads as expected. However when using ScreamerNet 9.3.1 UB and the built-in LightWave render controller, the best results are with 2 instances of ScreamerNet, both running with 8 threads each. So when running ScreamerNet UB on a PPC, ScreamerNet UB appears to get best results with the render threads set higher than the number of actual cores or CPUs in contrast to Layout.

Below are some older benchmarks I ran on a Power Macintosh Dual Processor 2GHz G5, 2GB RAM, running Mac OS X.3.3 and LightWave 8.0 PPC which uses the classic camera.

These times are for rendering 2 frames of the Benchmark:Raytrace.lws scene.

On this machine, LightWave itself renders the fastest (in these tests) when set to use 8 threads. LWSN renders fastest with two instances of LWSN, each running a single render thread.

In general when using such a dual processor machine in a render farm, you’ll get the best results running two instances of LWSN PPC, each using one thread. One thing to keep in mind though is that running two instances of LWSN PPC means you’ll be using twice as much memory as a single instance. So if memory is an issue, such as when rendering high resolution images, you may actually get better results running one instance of LWSN PPC set to two or more threads. That way both CPUs and all available memory can be used for a single image.

Since the Multithreading setting is stored in the config file, and LightWave & ScreamerNet/LWSN run optimally with different render thread settings, it is often useful to configure LightWave and your ScreamerNet/LWSN nodes to use different config files. Otherwise, be sure to check/set the RenderThreads setting in LightWave (or directly in the config file with a text editor) prior to launching ScreamerNet/LWSN.

Here’s a step-by-step example of how to set the RenderThreads setting with LightWave:

1. Launch LightWave.
2. Open Render->Options->Render Globals
3. Click on the Render tab.
4. Set the Multithreading pop-up menu to 1, 2, 4, 8 or 16Threads, as desired.

   

5. Quit LightWave, to have LightWave write the change to the config file.

You may also simply locate and edit the line RenderThreads 1 in your config file directly in a text editor. Simply change the number to 1, 2, 4, 8, or 16 as desired and save the config file. Make sure your text editor keeps the file as a plain text file and doesn’t add any file extension to the config’s filename, or it will not load into LightWave.

1 [2] 3 4 5 6 7 8 9