[Sidefx-houdini-list] disappearing HDA parameters

Antoine Durr antoine at floqfx.com
Fri Jun 6 14:23:14 EDT 2008

Some very cool points below.  I think any job/seq/shot/task type 
metadata is going to need site-settable fields, as you call it a shot, 
and I call it a scene, etc.  Or I might have one more field than you.  
The issue of ownership definitely needs to be addressed: I want to have 
temporary ownership, and during that time, no-one else can muck with it.

Maybe what might be easiest is that the set of menu entries that deal 
with make-editable, match-to-definition, etc. should come from another 
OPmenu file which can be modified.  SESI could add some example plugins, 
eg. copy to local area, rcs checkout/checkin,etc.  Those plugin scripts 
would set particular metadata on the OTL in question,i.e. an arbitrary 
set of key/value pairs.  Having just those two facilities would address 
80% of the issues.

As far as dependencies go, here's a question: if you are interested in 
releasing an OTL  that uses subsidiary otlA version 1.2, otlB version 
2.5, etc., why not bake them all into the top level .otl file?  Not 
pretty, but that would guarantee that the correct things got loaded.  
Instead, what I'm hearing is that all the issues regarding dependencies, 
and particularly about *finding* the darned things if they're not there 
in the first place should be addressed.  I'm not convinced that Houdini 
should be doing that, though I think asset versioning would be most helpful.

What's interesting to note, now that I'm doing a freelance gig her at 
Digital Domain, is that back at Rhythm & Hues, we had pipeline set up 
where you could inherit a node structure, i.e. someone publishes a batch 
of OBJects/SOPs, etc., you subscribe to it, and presto, it shows up in 
your .hip file.  Here at Digital Domain, we do the same thing but buried 
inside an OTL.  I.e. the OTL itself creates the pipeline.  Is this what 
most people are doing?   The big limitation with this approach is that 
once you unlock, you're on your own.  And you have to unlock a lot, 
because inevitably your shot is different than the next, and sure 
enough, the default setup doesn't quite cover what you need.  If OTLs 
are consistently being used that way, then I'd love to see some work 
being done to better support that workflow.

-- Antoine

Andrew D Lyons wrote:
> Hi,
> There are a lot of good revision contols systems out there for *nix,
> and SESI is better off providing hooks for those rather than trying to
> embed one for now (like the system in the video). Here's some more
> ideas for providing some basic support:
> - Tighten up hooks for versioning HDAs so people don't have to bake
> version numbers into hda file names to avoid conflicts in a hipfile.
>    - Have hda version readable and writable (with version write/lock
> permissions) from disk. ie expand hotl interface to permit definition
> metadata manipulation.
>    - Have HDA manager in Houdini recognise versions.
>    - Have Galleries recognise and support hda versions.
> - Swap out the filesystem oriented tree in the HDA manager for a more
> production-oriented tree in both hda and gallery. ie.
> job-sequence-shot-task.
>    - Allow hdas to be tagged with job-sequence-shot-task metadata.
>    - Allow galleries to be tagged with job-sequence-shot-task metadata.
>    - A hotl, hom api, and gu interface for "promoting" and "demoting"
> a hda version in the production-oriented tree.
>    - Automatically have the manager give priority to topmost
> production-oriented tree defs when instantiating new ops - even if
> that means ignoring versions.
>    - keep disk location visible and manageable, but shift emphasis to
> the production abstraction.
> - Provide hooks for calling a revision control system on an otl file:
>    - provide a checkout, checkin hook etc.
>    - provide the hda manager a hook to test the lock status of an otl file
> - Multiple instances of the same HDA (name) in a scene hda manager
> causes huge problems. Currently otl file location differences are the
> mechanism that permits this.
>    - disallow multiple hda definitions to be loaded into a hip file
> that have the same version and production tree setting - even if they
> come from different disk directories.
>    - An otl file should only ever contain a single hda. otls that
> store multiple hdas have always created more problems than they solved
> IMHO, and should be flagged with a warning.
> - A new "Digital Asset Set" graph structure with accompanying file
> format. To allow people to manage "sets" of otls.
>    - Sets should work with the production-oriented tree position,
> version values, and disk location.
>    - Set Manager API for HOM
>    - Set manager GUI (in standalone mode as well perhaps?)
> That's it for now.
> Cheers
> 2008/6/6 paul simpson <paul at realisestudio.com>:
>> hi robert,
>> thanks for putting this link up.  i think this summarises the magnitude of
>> the misunderstanding over the last few years.  it seems most people
>> (including sesi?) are getting confused with digital asset management (ie,
>> textures, external shaders, geometery) and HDA package management (ie,
>> digital asset dependences).  to me, the former is an open ended potential
>> quagmire that i would rather sesi leave alone for now.  the latter is imho
>> an essential tool that we needed (4 years ago) in order to create and
>> package HDA's that in turn use other HDA's within them.  the two are *very*
>> different subjects!  (you could - in time treat all external files as
>> digital assets - buts that an entirely different topic for now).
>> to reiterate - i see the latter as very important as i would like to share
>> assets from old jobs, other shops and houdini exchange without having to
>> worry about which other assets each one is using.  really, we need all the
>> assets to be included in a search path - and each asset can use either the
>> latest - or an earlier version all happily and transparently.  i'm thinking
>> about continuity between jobs here.  i see this as a universal requirement.
>>  i would like this logic to be implemented initially in a bare bones fashion
>> by sesi - and made open so individual shops can then taylor to their own
>> needs.  (please look at debian package management for a great example of
>> what i mean).
>> the video you point us to refers to the former.  so, to answer your question
>> - i would rather any have the hda dependency system implemented rather than
>> this tool you refer to.
>> we all need to be clear what we're trying to talk about.  part of this
>> problem is sesis fault naming HDA's or (OTL's) as "digital assets".  it's a
>> very generic term within the industry which often refers to texture maps,
>> geometry files - as in the 3dsmax example you posted.  nomenclature is very
>> important - and needs to be 'normalised' somewhat - or at least
>> de-obfuscated.  sesi needs to take the bull by the horns and get more
>> production savvy asap.
>> this subject has been talked about at cross purposes for over five years
>> now!  :-)
>> -paul
>> 2008/6/6 Robert Magee <rmagee at sidefx.com>:
>>> A couple of releases ago, Autodesk put out this movie for their asset
>>> tracking in Max8. It starts out as a single window where all the
>>> dependencies in a scene are laid out and their current status is
>>> outlined (links broken, etc..). From this window links can be re-
>>> established and even swapped out for other versions as needed. This
>>> system offers a one-stop-shopping for working with local assets. They
>>> can then extend that to a network as a "stage 2" of the system:
>>> http://download.autodesk.com/media/3dsmax/asset_tracking_max8_380k.mov
>>> Without going to a full blow asset management system (aka tactic),
>>> would something like this meet the needs outlined in this thread? It
>>> would be very helpful to figure out exactly how far people would want
>>> us to go. When we spoke with customers on this in the past, the
>>> requests seemed to lean towards a system that was open and not tied
>>> too tightly with Houdini. A Houdini-centric set of tools might be
>>> easier in the short term to get people started.
>>> Robert
>>> On 6-Jun-08, at 10:21 AM, Ken Ouellette wrote:
>>>> Paul's problems (I'm assuming here, but based on a hunch of how he has
>>>> worked in the past) is partly on how he is using HDAs. To that
>>>> regard I
>>>> don't think he is out of line asking for a bit more support to make
>>>> the
>>>> packaging and transport/sharing of HDAs easier.
>>>> Yes for a small centralized studio it is easier to self manage and
>>>> regulate
>>>> the changes. For a larger pipeline people will naturally build tools
>>>> to
>>>> support some sort of revision system, even it is just a slight twist
>>>> on the
>>>> autoBackup of HDAs and symlinking those to a none versioned reference.
>>>> Opdepend was an helpful addition for getting some help, but the
>>>> process can
>>>> still become complex quickly.
>>>> There could be some things that Sesi could do to support
>>>> encapsulated HDA in
>>>> HDAs a bit easier. That has always been a bit of a problem as
>>>> finding what
>>>> is inside one and understanding its dependencies can be a bit of a
>>>> pain.
>>>> Then add to the fact that finding certain disk based references
>>>> (textures,
>>>> geometry, etc) can be a again a bit of a haul to make it work easily.
>>>> Hopefully some of those basics are easier to handle with python. I
>>>> haven't
>>>> looked, but even if you made up your own methods to handle all the
>>>> cases it
>>>> would take you a while. It all depends on how absolute you want to
>>>> be in
>>>> your version of the asset, not just the HDA. The HDA itself is a
>>>> container
>>>> with parameters, you can inject some of its components (geometry,
>>>> and low
>>>> rez textures) but other items like high rez textures, and rib/ifd
>>>> files
>>>> either don't work in the context (I know Renderman doesn't like it,
>>>> not sure
>>>> about Mantra). So when it comes down to it there are a lot of
>>>> support files
>>>> you need to consider when you are planning out your pipe.
>>>> It gets more complex if you start to encapsulate HDAs inside HDAs
>>>> and then
>>>> it can become very important for the order in the loading/
>>>> instantiation and
>>>> then add the dependency issues across these nested HDAs. The
>>>> flexibility in
>>>> Houdini makes it really difficult to contain. Having had a large
>>>> role in
>>>> building two asset management systems for Houdini, it is very
>>>> difficult to
>>>> keep things agnostic and actually requires more structure then you
>>>> would
>>>> expect. There are many ways to create HDAs  and it becomes a job
>>>> just to
>>>> support the different methods. That is part of a natural process of
>>>> having
>>>> and maintaining a pipeline vision and should be expect with every
>>>> system
>>>> though. I do feel it is more of a challenge in Houdini than in other
>>>> products ( <cough>Maya<cough>)
>>>> When I saw what Sesi was doing with python and H9 I really stressed
>>>> to them
>>>> that with a hand full of people any studio could quickly flesh out a
>>>> very
>>>> sospicated pipeline if they had some experience in doing it in
>>>> previous
>>>> versions. Having Sesi provide blue print type examples on how to set
>>>> up a
>>>> system would be great but also starts a slippery slope for them.
>>>> Where does
>>>> providing added value stop with support? That is a difficult problem
>>>> for any
>>>> software vendor and one most avoid. I can't blame them
>>>> The embedded browser is very helpful to develop an app to handle the
>>>> management of the assets, but there are challenges that inherit to
>>>> using
>>>> that environment. Thankfully Sidefx has made huge efforts to remove
>>>> a lot of
>>>> these but there are still cycles to be spent finding ways to around
>>>> them. In
>>>> the end it is the same problem of Sidefx entering a slippery slope
>>>> of where
>>>> to draw the line for support.
>>>> For "a irritated Georg"
>>>> Encapsulation for a entire can be easy to implement but has serious
>>>> draw
>>>> backs for updates, both HDA wise and parameters. On the Wild we had
>>>> a fairly
>>>> strict rule that we supported only HDAs, which added its own
>>>> complexity to
>>>> making things (think of how hard it is/was to make a dynamic
>>>> parenting HDA).
>>>> We knew what  HDA's whet into a hipfile and what verison they were,
>>>> we also
>>>> saved out all the animation for the files. In some cases we wrote out
>>>> geometry and had special HDAs to handle those cases.
>>>> Saving and versioning entire hipfiles and keeping track of the
>>>> versions of
>>>> any HDAs inside....  for a production pipeline is a bit easier and
>>>> can offer
>>>> some benefits, but still has some drawbacks. That method that was
>>>> used as
>>>> well post "the Wild" in a number of shows.  There are always
>>>> challenges
>>>> about updating versions and handling animation.cmd files.....
>>>> keeping track
>>>> of changes between saved hipfiles when it comes to non-HDA elements
>>>> anyone
>>>> can create in Houdini. Connections parenting all that jazz
>>>> The real power is to find a method to manage all the information and
>>>> push
>>>> changes out on the fly. Then you complete the circle of the pipeline
>>>> and you
>>>> have the ability to make large sweeping changes effecting renders
>>>> going to
>>>> the farm.
>>>> I should stop now.. that is long enough to read :)
>>>> -k
>>>> On Fri, Jun 6, 2008 at 7:48 AM, Pablo Giménez Pizarro <
>>> pablogipi at gmail.com
>>>> wrote:
>>>>> El 06/06/2008, a las 13:33, Peter Robbinson escribió:
>>>>>> Hi,
>>>>>> It had a lot to do with the asset manager.
>>>>>> There were specialists, Michael being an important one, who were the
>>>>>> responsible for developing and maintaining the digital assets.
>>>>>> The artists on the production floor accessed the appropriate assets
>>>>>> for
>>>>>> a given scene through the embedded browser. Core used the embedded
>>>>>> browser to present an interface to the asset manager. Artists only
>>>>>> had
>>>>>> access to the approved assets for an assigned scene. There were ways
>>>>>> to
>>>>>> access earlier versions of assets, but only when problems were
>>>>>> encountered. The assest manager allowed artists to report problems
>>>>>> (bug
>>>>>> reports) and these were automatically reported to the appropriate
>>>>>> department.
>>>>>> Michael play a big part in getting that system to work, it was quite
>>>>>> remarkable. That's not to say it worked well all the time, there
>>>>>> were
>>>>>> plenty of work-arounds necessary and it took time to get working
>>>>>> right,
>>>>>> but it was certainly the best pipeline I had ever encountered.
>>>>>> Keep in
>>>>>> mind Core used version 6 of houdini on that production and digital
>>>>>> assets were brand new.
>>>>> Thanks for the explanation Peter.
>>>>> Now I think that you are agree with me that to take the most from HDA
>>>>> you need some management tool.
>>>>> I am not talking to have this complex system developed at CORE but
>>>>> something simple and easy for the small studios that use tools that
>>>>> can be used by big houses to develop their own management systems
>>>>> (like the otlversion command for example)
>>>>> In my opinion SESI did really well with the character picker and pose
>>>>> library tools, something like this, in the browser will be great. The
>>>>> pose library for example is not the most advanced tool, but it works
>>>>> out of the box and for a small studio do the job and if you need to
>>>>> extent it you can.
>>>>>> PeterR
>>>>>> Georg Dümlein wrote:
>>>>>>> Michael Goldfarb <goldfarb at coredp.com> schrieb:
>>>>>>>> The Wild was made almost entirely (like 95%+) without hip files...
>>>>>>> Sorry for the noobish question , but:
>>>>>>> How does this work?
>>>>>>> Are you using HDA that contain the complete scene?
>>>>>>> a irritated Georg
>>>>>>> _______________________________________________
>>>>>>> Sidefx-houdini-list mailing list
>>>>>>> Sidefx-houdini-list at sidefx.com
>>>>>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>>>>>> --
>>>>>> "gravity is not a force, it is a boundary layer"
>>>>>> "everything is coincident"
>>>>>> "Love: the state of suspended anticipation"
>>>>>> "To get any appreciable distance from the Earth in
>>>>>> a sensible amount of time, you must lie."
>>>>>> "Easy, is not how you make a living."
>>>>>> _______________________________________________
>>>>>> Sidefx-houdini-list mailing list
>>>>>> Sidefx-houdini-list at sidefx.com
>>>>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>>>>> ---------
>>>>> Un saludo
>>>>> Best Regards
>>>>> Pablo Giménez
>>>>> pablogipi at gmail.com
>>>>> _______________________________________________
>>>>> Sidefx-houdini-list mailing list
>>>>> Sidefx-houdini-list at sidefx.com
>>>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>>>> _______________________________________________
>>>> Sidefx-houdini-list mailing list
>>>> Sidefx-houdini-list at sidefx.com
>>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>>> ----
>>> Robert Magee
>>> Product Marketing Manager
>>> Side Effects Software Inc.
>>> www.sidefx.com
>>> 416 504-9876 x327
>>> _______________________________________________
>>> Sidefx-houdini-list mailing list
>>> Sidefx-houdini-list at sidefx.com
>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> _______________________________________________
>> Sidefx-houdini-list mailing list
>> Sidefx-houdini-list at sidefx.com
>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list

More information about the Sidefx-houdini-list mailing list