Resources : Lightwave

Setting up Screamernet for Lightwave 10 on OS X
By Dan Sollis

OK, I think everyone knows that network rendering on Lightwave is a slightly painful process. There's some good third party solutions to take some of the pain out of it, but unfortunately most of them are Windows-only. To save you the grief, I thought I'd write up a mac-specific guide. It's also LW10 specific - though all of this should work for 9.6 (universal binary, non CFM version) and LW11. This assumes all the machines that you use in your renderfarm are running OSX. It is possible to run a mixed windows/mac farm, but to paraphrase "The Steve" (RIP), it's a bag of hurt and I'm not going to cover it here :)

Theory

I think the biggest stumbling block I had with screamernet setup was the basic conceptual stuff about how it works. Once you know this, it all makes a lot more sense. So here are the most important things you need to realise:

Firstly, screamernet does not communicate directly between the render controller (the screamernet panel in Lightwave) and the render nodes. Screamernet was developed almost 20 years ago, so it obviously predates any modern niceties like zero-conf or bonjour. Instead it uses a shared network folder into which it writes little text files that the nodes and controller read periodically to find the status of the controller or node. It's a staggeringly basic system, but it does work. What this means in terms of getting it working is that both the controller and all the render nodes need to be able to access the same folder on the network and be able to read and write to that folder. This all-important folder is known as the COMMAND folder. The screamernet controller writes "job" files for each node to this folder, which the render nodes read. In response each render node writes to an "ack" (acknowledgement) file. This is then read by the controller, which then overwrites the original job file with updated instructions. And so the process loops.

There's two other shared folders you'll need as well:

The CONFIGS folder, which holds the preferences files for the SN nodes. These preference files contain some very important settings like the size of the render buffer (only 32mb by default - which is too small for HD) as well as links to all plugins. Plugins are a very common point of failure, so I highly recommend having them installed in the same location as Lightwave's own native plugins, which is usually:
Applications / Newtek / Lightwave 3D10.0 / Support / Plugins
(
You'll need to clear your plugin list and re-scan this directory again if you're moving plugins)

The final shared folder you need is the aforementioned Content Directory containing the scene and all related files you need in order to render. This should be structured in the standard lightwave format of SCENES / OBJECTS / IMAGES / FRAMES if you want to avoid any undue pain!

The step-by-step guide

So, with the preceding ideas in mind, let's set up our network!

1. The most important first step - enable file-sharing on all machines that you want to use in the render network. This is done by going to System Preferences : Sharing and ticking on File Sharing. I'm doing this in 10.6 of OSX, other versions may vary slightly.

2. On your main machine (the one you'll use as the render controller), create a new folder at the root level of one of your hard drives. Call it something logical like "LWSN_shared". I strongly advise against putting any spaces or non-ASCI characters in the folder name (or names for anything else in this tutorial!) as it might break over the network and will make your life more complicated.

3. Inside the LWSN_shared folder, create two other folders: "LWSN_commands" and "LWSN_configs".

4. We now need to create the config files (pointing to the Content Directory). To generate this, open up Layout and choose the correct Content Directory (from the Edit popup). Then quit Layout as it only saves the config file when you quit. The location of config files changed during the LW10 to LW10.1 update, so:
(a) For LW10, In your Home user directory, go to Library : Preferences : Lightwave3D and copy "Extensions 10" and "Layout 10" to the LWSN_configs folder.
(b) For LW10.1 (and possibly 11 too, though that's in flux) the files are in Library : Application Support : Newtek : Lightwave : 10.0
or Library : Application Support : Newtek : Lightwave : 10.1 (thanks to Dwight Bonney for pointing this out!)

5. Copy your Lightwave3D 10.0 folder (containing the app and all plugins) to the LWSN_shared folder.

6. Finally, in system preferences open Sharing again. Click on "File Sharing" from the list and in the "Shared Folders" box, add your LWSN_Shared directory - with Read&Write enabled for all users.

OK, that's pretty much all the network setup you need to do for the moment. The next stage is to set up and launch your Screamernet nodes on all the machines you want to use for rendering. This can include the mac you're running the screamernet controller on too (yeah, in an ideal world you'll use a dedicated machine for the controller, but I can't afford to!). Screamernet is in the Lightwave3D 10.0 / bin folder, but you can't just double-click and launch it like any other app. It's a command-line app and therefore expects to be launched from the terminal with specific parameters passed to it at the same time.

You could do this manually for all machines, but it's really tedious - so instead I'm going to tell you to just go to here and download Mike Scaramozzino's DLI_SNUB-Launcher app. This is a fairly basic but bloody useful utility that does all the hard work for you. You can run it on each of your render nodes and use it to launch screamernet with the correct parameters set. With the free version, you can use it to launch up 2 nodes. But don't be a cheapskate - just buy the 10 node Standard Version, it's only 25 bucks.

Put the DLI_SNUB-Launcher app in your LWSN_shared : Lightwave3D 10.0 folder. We're now ready to get our nodes set up.

On each of the macs that is going to be a render node, you will need to mount the LWSN_Shared folder as well as the drive with the Content Directory. To do this, from the Finder menu select: Go : Connect to server and from the "recent servers" list (the clock icon on the right) select the mac which hosts these folders. It'll bring up a list of volumes attached to that machine - LWSN_Shared should be one of them. Mount that, then re-do the process to add any other drives that are needed for the render.

You might need to change permissions or put in administrator password in order to do so. Don't forget you need to be able to read AND write to this folder on the remote machines. If Screamernet's not working, chances are you haven't set read/write permissions correctly for LWSN_shared (and the other folders and files inside there too!). If in doubt, highlight the folder (when viewing it from the remote node), hit Apple + I and make sure you can read/write is enabled. You can also apply the same permissions to "All enclosed files" - which is a good bet!

Once the folder is open, launch DLI_SNUB-Launcher. To set it up correctly, make sure that all the different folder paths are set to the right places. This means:
Screamernet path - point it to LWSN_shared : Lightwave3D 10.0 : bin : ScreamerNet
Config folder - point to LWSN_shared : LWSN_configs
Content folder - point to your Content Directory
(and down at the bottom)
Command Folder - point to LWSN_shared : LWSN_commands

Finally, decide what node number you want that machine to be. Each machine needs to have a unique number, so keep track of it. You put it in the "First ScreamerNet Node #" box. If you want to enable more than one node per machine (useful for simpler renders), you can activate multiple nodes using the Through Last Node options. But don't bother for now. The 9.6 renderer is pretty well threaded these days and personally I'm not bothered by the minor speed gains verses the hassle of managing tons of nodes. With all the options set in DLI_SNUB-Launcher, click "Launch ScreamerNetUB Nodes" to launch Screamernet in the terminal. You should see something like:

sendack: Initializing
MachOMac

That node's now ready. Do the same for the remaining Macs on your network, putting a different node number in for each.

The final stage - Screamernet Controller

OK, we're finally ready to start rendering. I'm assuming you have already set up your scene ready to render. So all the output paths are pointing to somewhere in your Content Directory (in Frames if you want to stick to standards).

Open up Layout. Go to Render and click Network Render to open up the Screamernet Panel. In the panel, click on the Command Directory button to change the command directory to your LWSN_shared : LWSN_commands folder. If you've got more than 8 nodes launched, increase your Maximum CPU number. Otherwise leave it alone.

Click on "Screamer Init". With luck, all your nodes will appear after a few seconds in the list below. This can be a bit flakey (sometimes nodes don't respond quickly enough for the controller to recognise them), so if they don't appear, try again. When they're finally there, click "Add scene to List" to choose your scene for rendering. As you can see, it's possible to queue up multiple scenes - which is very useful for batch rendering over the weekend!

And finally, click Screamer Render to launch it all. YESSSSSS!!!!

Well, I hope that all worked for you. Please contact me (Dan Sollis) if you notice any mistakes here - email on the contact page