(Amiberry powered) Lightwave render farm, (pt 5)

By | 2020-06-07

It’s time to see how exactly we can use our shiny new render farm, that we’ve spent the last parts setting up. So without further delay, let’s jump right into it.

First things first

We will need something to render: A simple scene is enough, as long as it has multiple frames (remember, each render node will be assigned a separate frame from the scene to render), and of course: the option to actually save the resulting rendered frames must be enabled. You wouldn’t want to wait for the rendering to complete, only to discover that no output frames were saved!

You might want to check Muadib’s video tutorials on Lightwave, to learn more about setting up scenes, objects, texturing and lots more: https://www.twitch.tv/muadib3d/ (and while you’re there, consider contributing with some donation if you can!). For now, we’ll just assume we have at least one Scene we can use.

We’ll start with the controlling machine that will be running Lightwave. It could be a real Amiga, or an emulated one – it doesn’t really matter, as long as it has the same 3D content directory, and the same “command” directory for ScreamerNet as our render nodes. Assigns are perfect for this job.

For example in my case, I have a volume named “Projects:” mounted, which is the shared directory from my NAS that I want to use. It contains a “3D” folder, which holds all my 3D content (Scenes, Objects, Textures, etc) and it’s also mounted in all my Amiberry render nodes (using the autofs approach we discussed in the previous guides). It also contains an empty directory named “Command”, which we will tell ScreamerNet to use for storing the job and response files.

So to make sure things will go smoothly, I’ve assigned 3D: to Projects:3D and Command: to Projects:Command:

Assign 3D: Projects:3D
Assign Command: Projects:Command

The same assign is applied to all the render farm nodes as well, during their startup.

With that settled, we load up Lightwave on our controlling machine.

Lightwave’s Layout default view when opened

We will start by loading our Scene, to make sure there are no errors. All objects, textures etc. should load correctly with no warnings. If you do get errors, like a missing object, you’ll have to address those and re-save the Scene, before proceeding.

I’ve loaded a Scene in Layout, with no errors

Next, we’ll go to the Record panel, and enable “Save RGB Images”. We’ll provide a path and filename where we want our output frames to be saved. In my case, that’s in “Projects:frames”. Remember, it has to be a path that is accessible by all your render farm nodes. Save your Scene after this step, to make sure all your render nodes will pick up the changes!

Making sure the Save RGB Images is enabled

Next, we’ll open the SceamerNet panel (named “SN”) and change the “Net Rendering Method” to ScreamerNet II. It will warn you that this method should only be used if the networked computers can share directories, which is exactly what we have setup, so go ahead. Then make sure your “Command Directory” points to your “Command:” assign (which we pointed to our shared directory, remember?).

Lightwave’s ScreamerNet II panel

The next item to change, will be the “Maximum CPU Number“. This indicates how many render nodes should ScreamerNet expect to find in our render farm. If you set a higher number than your actual computers here, ScreamerNet will just spend more time waiting for them to respond. So ideally, we want a number that is equal to the number of computers we have in our render farm – which in my case, is 6.

Now we can start adding the Scene(s) we want to render in the List, by using the relevant button. You can queue up as many scenes as you want, but I’ll just use one for this example.

I’ve loaded one Scene to be rendered in my List

Once you have your Scene(s) loaded in the List, it’s time to press “Screamer Init” once. This will generate job files, one for each “CPU” you specified (so again, in my case a total of 6 files) and wait for a response from the machines which should pick these up. Since we have no render nodes running at the moment, it will come back with a response saying “No available ScreamerNet CPUs were detected”. This is normal, and desired at this point.

No CPUs detected on Init

The reasoning behind this, is that we want to have the job files created in order for the nodes of our render farm to pick them up when starting. Once they pick them up, they will start a loop of waiting for new commands, and then we’ll use the “Screamer Init” button once more, to detect them (and this time, they will be ready to respond).

Start your engines

It’s time to fire up our render farm nodes! I will start Amiberry on each machine, using my “RenderNode” config which is designed to load up a minimal AROS distribution, including Lightwave’s ScreamerNet client and a little script I wrote to automatically assign a unique number to each one. You can do this from the command line:

./amiberry -f conf/rendernode-aros.uae -G

Or you can of course open it up from the GUI, load the config and hit Start.

The -G command line option in Amiberry, tells it to skip opening the GUI and instead start immediately using the specified config file. Alternatively, you can have the option “Show GUI on Startup” disabled inside the config file, in which case you don’t need to specify “-G” at all. And additionally, you could rename your config filename to “default.uae” and Amiberry will load it up automatically, without even having to specify that!

I’ll do this for each of my 6 nodes in my render farm, and to demonstrate it easier, I did it over VNC so I can show you what happens:

All 6 nodes in my render farm have started and are waiting for a job

And now, we can click on Screamer Init once more. This time, it should find the computers in the render farm, and show them in the list.

Screamer Init found my 6 computers

One more step remaining: Just click on Screamer Render, and the Scenes in the List will be loaded and rendered, one frame at a time (in parallel of course). Here’s another screenshot of the rendering in action:

My 6-node render farm, busy rendering frames…

And this is how it looks like from Lightwave’s side, while the rendering is going on:

As soon as a frame is finished rendering, it’s saved in the output directory you specified in the Scene. If you have multiple Scenes in the list, it will automatically proceed to the next one, until they are all finished.

Once the list of Scenes is finished, congratulations, your job is done! You can gracefully shut down all the render farm nodes, by using the “Screamer Shutdown” button. This will instruct all the Lightwave ScreamerNet clients to cleanup and quit. I’ve also added an extra step in my custom script, to quit the emulator as well when that happens (by using WinUAE’s “UAEQuit” tool) – so when you shut ScreamerNet down, you will also quit Amiberry in one step.

This concludes the guide, on how to setup an Amiberry-powered Lightwave render farm. I will try to keep this guide up-to-date as things evolve, and based on any feedback that it might generate. I hope it helps and inspires some people to be a little creative!