[Sidefx-houdini-list] render farms, pipelines and file systems

Andrew D Lyons tstexture at gmail.com
Tue Jan 6 22:43:36 EST 2009

Hi Drew,

I'm not an expert from the hardware end, but here's some ideas from
the farm manager end.

> Do you generate IFD's out on the farm or on artists workstations, and
> if so how are resources synced across to where they are needed ?

- Break renders up into two steps - IFD generation + Render. This
releases batch licenses asap. Mantra licenses are free.
- Generate IFD's on the farm. Have IFD's include existing delayed
geo/rib files where possible.
- On multi-core nodes - try and ensure that all cores are doing IFD
generation to minimize per node license usage.
- Have IFD's write to machine local temp drive. Have renderer look
local. Cleanup afterwards.
- only images/geos go back to central server.

> Is it the case that you purchase expensive global file systems and
> maintain a single shared file space ? If so, are they robust and
> performant ?

Not an expert - but this seems to be the way to go. $$$

> Obviously their are significant performance issues to consider. Eg you
> are working with Gb's of texture per frame which you don't want to
> ship across to a render node every time a frame is rendered. There are
> many possible solutions to this problem but I'd love to hear about
> experiences in the trenches ...

Some systems track such dependencies and do a full copy to each node
prior to render rather than trickle them across, and then chunk a
certain number of frames per "job". Hard to avoid a hit.

Good Luck!


Andrew D Lyons | Digital Artist | http://www.tstex.com

More information about the Sidefx-houdini-list mailing list