[Sidefx-houdini-list] disappearing HDA parameters

Pablo Giménez Pizarro pablogipi at gmail.com
Fri Jun 6 14:46:20 EDT 2008

El 06/06/2008, a las 20:23, Antoine Durr escribió:

> Some very cool points below.  I think any job/seq/shot/task type
> metadata is going to need site-settable fields, as you call it a shot,
> and I call it a scene, etc.  Or I might have one more field than you.
> The issue of ownership definitely needs to be addressed: I want to  
> have
> temporary ownership, and during that time, no-one else can muck with  
> it.
I think that Houdini doesn't have to define your metadata, instead it  
have to give you a tool to create metadata and create hyerarcies of  
metadata, that will be saved very straighforward using and XML section  
into the HDA.
For example a tool in the properties that allows you to create a key  
called Project, a chield called Shot, and so on, then you can define  
your pipeline and fill the key values with the needed names.
Basically an editor for metadata that allows to save templates an then  
you can create you own projects when needed.
> Maybe what might be easiest is that the set of menu entries that deal
> with make-editable, match-to-definition, etc. should come from another
> OPmenu file which can be modified.  SESI could add some example  
> plugins,
> eg. copy to local area, rcs checkout/checkin,etc.  Those plugin  
> scripts
> would set particular metadata on the OTL in question,i.e. an arbitrary
> set of key/value pairs.  Having just those two facilities would  
> address
> 80% of the issues.
I think these are the hooks that Andrew have pointed.
> As far as dependencies go, here's a question: if you are interested in
> releasing an OTL  that uses subsidiary otlA version 1.2, otlB version
> 2.5, etc., why not bake them all into the top level .otl file?  Not
> pretty, but that would guarantee that the correct things got loaded.
> Instead, what I'm hearing is that all the issues regarding  
> dependencies,
> and particularly about *finding* the darned things if they're not  
> there
> in the first place should be addressed.  I'm not convinced that  
> Houdini
> should be doing that, though I think asset versioning would be most  
> helpful.
I think that to always have the option to pack all your dependencies  
in the HDA is good for debug or to solve problems in production,  
similar to the option that allows you to put all your HDA definitions  
into the hip file.
But it will be only to solve problems or debugging not the way to work.
> What's interesting to note, now that I'm doing a freelance gig her at
> Digital Domain, is that back at Rhythm & Hues, we had pipeline set up
> where you could inherit a node structure, i.e. someone publishes a  
> batch
> of OBJects/SOPs, etc., you subscribe to it, and presto, it shows up in
> your .hip file.  Here at Digital Domain, we do the same thing but  
> buried
> inside an OTL.  I.e. the OTL itself creates the pipeline.  Is this  
> what
> most people are doing?   The big limitation with this approach is that
> once you unlock, you're on your own.  And you have to unlock a lot,
> because inevitably your shot is different than the next, and sure
> enough, the default setup doesn't quite cover what you need.  If OTLs
> are consistently being used that way, then I'd love to see some work
> being done to better support that workflow.
> -- Antoine
> Andrew D Lyons wrote:
>> Hi,
>> There are a lot of good revision contols systems out there for *nix,
>> and SESI is better off providing hooks for those rather than trying  
>> to
>> embed one for now (like the system in the video). Here's some more
>> ideas for providing some basic support:
>> - Tighten up hooks for versioning HDAs so people don't have to bake
>> version numbers into hda file names to avoid conflicts in a hipfile.
>>   - Have hda version readable and writable (with version write/lock
>> permissions) from disk. ie expand hotl interface to permit definition
>> metadata manipulation.
>>   - Have HDA manager in Houdini recognise versions.
>>   - Have Galleries recognise and support hda versions.
>> - Swap out the filesystem oriented tree in the HDA manager for a more
>> production-oriented tree in both hda and gallery. ie.
>> job-sequence-shot-task.
>>   - Allow hdas to be tagged with job-sequence-shot-task metadata.
>>   - Allow galleries to be tagged with job-sequence-shot-task  
>> metadata.
>>   - A hotl, hom api, and gu interface for "promoting" and "demoting"
>> a hda version in the production-oriented tree.
>>   - Automatically have the manager give priority to topmost
>> production-oriented tree defs when instantiating new ops - even if
>> that means ignoring versions.
>>   - keep disk location visible and manageable, but shift emphasis to
>> the production abstraction.
>> - Provide hooks for calling a revision control system on an otl file:
>>   - provide a checkout, checkin hook etc.
>>   - provide the hda manager a hook to test the lock status of an  
>> otl file
>> - Multiple instances of the same HDA (name) in a scene hda manager
>> causes huge problems. Currently otl file location differences are the
>> mechanism that permits this.
>>   - disallow multiple hda definitions to be loaded into a hip file
>> that have the same version and production tree setting - even if they
>> come from different disk directories.
>>   - An otl file should only ever contain a single hda. otls that
>> store multiple hdas have always created more problems than they  
>> solved
>> IMHO, and should be flagged with a warning.
>> - A new "Digital Asset Set" graph structure with accompanying file
>> format. To allow people to manage "sets" of otls.
>>   - Sets should work with the production-oriented tree position,
>> version values, and disk location.
>>   - Set Manager API for HOM
>>   - Set manager GUI (in standalone mode as well perhaps?)
>> That's it for now.
>> Cheers
>> 2008/6/6 paul simpson <paul at realisestudio.com>:
>>> hi robert,
>>> thanks for putting this link up.  i think this summarises the  
>>> magnitude of
>>> the misunderstanding over the last few years.  it seems most people
>>> (including sesi?) are getting confused with digital asset  
>>> management (ie,
>>> textures, external shaders, geometery) and HDA package management  
>>> (ie,
>>> digital asset dependences).  to me, the former is an open ended  
>>> potential
>>> quagmire that i would rather sesi leave alone for now.  the latter  
>>> is imho
>>> an essential tool that we needed (4 years ago) in order to create  
>>> and
>>> package HDA's that in turn use other HDA's within them.  the two  
>>> are *very*
>>> different subjects!  (you could - in time treat all external files  
>>> as
>>> digital assets - buts that an entirely different topic for now).
>>> to reiterate - i see the latter as very important as i would like  
>>> to share
>>> assets from old jobs, other shops and houdini exchange without  
>>> having to
>>> worry about which other assets each one is using.  really, we need  
>>> all the
>>> assets to be included in a search path - and each asset can use  
>>> either the
>>> latest - or an earlier version all happily and transparently.  i'm  
>>> thinking
>>> about continuity between jobs here.  i see this as a universal  
>>> requirement.
>>> i would like this logic to be implemented initially in a bare  
>>> bones fashion
>>> by sesi - and made open so individual shops can then taylor to  
>>> their own
>>> needs.  (please look at debian package management for a great  
>>> example of
>>> what i mean).
>>> the video you point us to refers to the former.  so, to answer  
>>> your question
>>> - i would rather any have the hda dependency system implemented  
>>> rather than
>>> this tool you refer to.
>>> we all need to be clear what we're trying to talk about.  part of  
>>> this
>>> problem is sesis fault naming HDA's or (OTL's) as "digital  
>>> assets".  it's a
>>> very generic term within the industry which often refers to  
>>> texture maps,
>>> geometry files - as in the 3dsmax example you posted.   
>>> nomenclature is very
>>> important - and needs to be 'normalised' somewhat - or at least
>>> de-obfuscated.  sesi needs to take the bull by the horns and get  
>>> more
>>> production savvy asap.
>>> this subject has been talked about at cross purposes for over five  
>>> years
>>> now!  :-)
>>> -paul
>>> 2008/6/6 Robert Magee <rmagee at sidefx.com>:
>>>> A couple of releases ago, Autodesk put out this movie for their  
>>>> asset
>>>> tracking in Max8. It starts out as a single window where all the
>>>> dependencies in a scene are laid out and their current status is
>>>> outlined (links broken, etc..). From this window links can be re-
>>>> established and even swapped out for other versions as needed. This
>>>> system offers a one-stop-shopping for working with local assets.  
>>>> They
>>>> can then extend that to a network as a "stage 2" of the system:
>>>> http://download.autodesk.com/media/3dsmax/asset_tracking_max8_380k.mov
>>>> Without going to a full blow asset management system (aka tactic),
>>>> would something like this meet the needs outlined in this thread?  
>>>> It
>>>> would be very helpful to figure out exactly how far people would  
>>>> want
>>>> us to go. When we spoke with customers on this in the past, the
>>>> requests seemed to lean towards a system that was open and not tied
>>>> too tightly with Houdini. A Houdini-centric set of tools might be
>>>> easier in the short term to get people started.
>>>> Robert
>>>> On 6-Jun-08, at 10:21 AM, Ken Ouellette wrote:
>>>>> Paul's problems (I'm assuming here, but based on a hunch of how  
>>>>> he has
>>>>> worked in the past) is partly on how he is using HDAs. To that
>>>>> regard I
>>>>> don't think he is out of line asking for a bit more support to  
>>>>> make
>>>>> the
>>>>> packaging and transport/sharing of HDAs easier.
>>>>> Yes for a small centralized studio it is easier to self manage and
>>>>> regulate
>>>>> the changes. For a larger pipeline people will naturally build  
>>>>> tools
>>>>> to
>>>>> support some sort of revision system, even it is just a slight  
>>>>> twist
>>>>> on the
>>>>> autoBackup of HDAs and symlinking those to a none versioned  
>>>>> reference.
>>>>> Opdepend was an helpful addition for getting some help, but the
>>>>> process can
>>>>> still become complex quickly.
>>>>> There could be some things that Sesi could do to support
>>>>> encapsulated HDA in
>>>>> HDAs a bit easier. That has always been a bit of a problem as
>>>>> finding what
>>>>> is inside one and understanding its dependencies can be a bit of a
>>>>> pain.
>>>>> Then add to the fact that finding certain disk based references
>>>>> (textures,
>>>>> geometry, etc) can be a again a bit of a haul to make it work  
>>>>> easily.
>>>>> Hopefully some of those basics are easier to handle with python. I
>>>>> haven't
>>>>> looked, but even if you made up your own methods to handle all the
>>>>> cases it
>>>>> would take you a while. It all depends on how absolute you want to
>>>>> be in
>>>>> your version of the asset, not just the HDA. The HDA itself is a
>>>>> container
>>>>> with parameters, you can inject some of its components (geometry,
>>>>> and low
>>>>> rez textures) but other items like high rez textures, and rib/ifd
>>>>> files
>>>>> either don't work in the context (I know Renderman doesn't like  
>>>>> it,
>>>>> not sure
>>>>> about Mantra). So when it comes down to it there are a lot of
>>>>> support files
>>>>> you need to consider when you are planning out your pipe.
>>>>> It gets more complex if you start to encapsulate HDAs inside HDAs
>>>>> and then
>>>>> it can become very important for the order in the loading/
>>>>> instantiation and
>>>>> then add the dependency issues across these nested HDAs. The
>>>>> flexibility in
>>>>> Houdini makes it really difficult to contain. Having had a large
>>>>> role in
>>>>> building two asset management systems for Houdini, it is very
>>>>> difficult to
>>>>> keep things agnostic and actually requires more structure then you
>>>>> would
>>>>> expect. There are many ways to create HDAs  and it becomes a job
>>>>> just to
>>>>> support the different methods. That is part of a natural process  
>>>>> of
>>>>> having
>>>>> and maintaining a pipeline vision and should be expect with every
>>>>> system
>>>>> though. I do feel it is more of a challenge in Houdini than in  
>>>>> other
>>>>> products ( <cough>Maya<cough>)
>>>>> When I saw what Sesi was doing with python and H9 I really  
>>>>> stressed
>>>>> to them
>>>>> that with a hand full of people any studio could quickly flesh  
>>>>> out a
>>>>> very
>>>>> sospicated pipeline if they had some experience in doing it in
>>>>> previous
>>>>> versions. Having Sesi provide blue print type examples on how to  
>>>>> set
>>>>> up a
>>>>> system would be great but also starts a slippery slope for them.
>>>>> Where does
>>>>> providing added value stop with support? That is a difficult  
>>>>> problem
>>>>> for any
>>>>> software vendor and one most avoid. I can't blame them
>>>>> The embedded browser is very helpful to develop an app to handle  
>>>>> the
>>>>> management of the assets, but there are challenges that inherit to
>>>>> using
>>>>> that environment. Thankfully Sidefx has made huge efforts to  
>>>>> remove
>>>>> a lot of
>>>>> these but there are still cycles to be spent finding ways to  
>>>>> around
>>>>> them. In
>>>>> the end it is the same problem of Sidefx entering a slippery slope
>>>>> of where
>>>>> to draw the line for support.
>>>>> For "a irritated Georg"
>>>>> Encapsulation for a entire can be easy to implement but has  
>>>>> serious
>>>>> draw
>>>>> backs for updates, both HDA wise and parameters. On the Wild we  
>>>>> had
>>>>> a fairly
>>>>> strict rule that we supported only HDAs, which added its own
>>>>> complexity to
>>>>> making things (think of how hard it is/was to make a dynamic
>>>>> parenting HDA).
>>>>> We knew what  HDA's whet into a hipfile and what verison they  
>>>>> were,
>>>>> we also
>>>>> saved out all the animation for the files. In some cases we  
>>>>> wrote out
>>>>> geometry and had special HDAs to handle those cases.
>>>>> Saving and versioning entire hipfiles and keeping track of the
>>>>> versions of
>>>>> any HDAs inside....  for a production pipeline is a bit easier and
>>>>> can offer
>>>>> some benefits, but still has some drawbacks. That method that was
>>>>> used as
>>>>> well post "the Wild" in a number of shows.  There are always
>>>>> challenges
>>>>> about updating versions and handling animation.cmd files.....
>>>>> keeping track
>>>>> of changes between saved hipfiles when it comes to non-HDA  
>>>>> elements
>>>>> anyone
>>>>> can create in Houdini. Connections parenting all that jazz
>>>>> The real power is to find a method to manage all the information  
>>>>> and
>>>>> push
>>>>> changes out on the fly. Then you complete the circle of the  
>>>>> pipeline
>>>>> and you
>>>>> have the ability to make large sweeping changes effecting renders
>>>>> going to
>>>>> the farm.
>>>>> I should stop now.. that is long enough to read :)
>>>>> -k
>>>>> On Fri, Jun 6, 2008 at 7:48 AM, Pablo Giménez Pizarro <
>>>> pablogipi at gmail.com
>>>>> wrote:
>>>>>> El 06/06/2008, a las 13:33, Peter Robbinson escribió:
>>>>>>> Hi,
>>>>>>> It had a lot to do with the asset manager.
>>>>>>> There were specialists, Michael being an important one, who  
>>>>>>> were the
>>>>>>> responsible for developing and maintaining the digital assets.
>>>>>>> The artists on the production floor accessed the appropriate  
>>>>>>> assets
>>>>>>> for
>>>>>>> a given scene through the embedded browser. Core used the  
>>>>>>> embedded
>>>>>>> browser to present an interface to the asset manager. Artists  
>>>>>>> only
>>>>>>> had
>>>>>>> access to the approved assets for an assigned scene. There  
>>>>>>> were ways
>>>>>>> to
>>>>>>> access earlier versions of assets, but only when problems were
>>>>>>> encountered. The assest manager allowed artists to report  
>>>>>>> problems
>>>>>>> (bug
>>>>>>> reports) and these were automatically reported to the  
>>>>>>> appropriate
>>>>>>> department.
>>>>>>> Michael play a big part in getting that system to work, it was  
>>>>>>> quite
>>>>>>> remarkable. That's not to say it worked well all the time, there
>>>>>>> were
>>>>>>> plenty of work-arounds necessary and it took time to get working
>>>>>>> right,
>>>>>>> but it was certainly the best pipeline I had ever encountered.
>>>>>>> Keep in
>>>>>>> mind Core used version 6 of houdini on that production and  
>>>>>>> digital
>>>>>>> assets were brand new.
>>>>>> Thanks for the explanation Peter.
>>>>>> Now I think that you are agree with me that to take the most  
>>>>>> from HDA
>>>>>> you need some management tool.
>>>>>> I am not talking to have this complex system developed at CORE  
>>>>>> but
>>>>>> something simple and easy for the small studios that use tools  
>>>>>> that
>>>>>> can be used by big houses to develop their own management systems
>>>>>> (like the otlversion command for example)
>>>>>> In my opinion SESI did really well with the character picker  
>>>>>> and pose
>>>>>> library tools, something like this, in the browser will be  
>>>>>> great. The
>>>>>> pose library for example is not the most advanced tool, but it  
>>>>>> works
>>>>>> out of the box and for a small studio do the job and if you  
>>>>>> need to
>>>>>> extent it you can.
>>>>>>> PeterR
>>>>>>> Georg Dümlein wrote:
>>>>>>>> Michael Goldfarb <goldfarb at coredp.com> schrieb:
>>>>>>>>> The Wild was made almost entirely (like 95%+) without hip  
>>>>>>>>> files...
>>>>>>>> Sorry for the noobish question , but:
>>>>>>>> How does this work?
>>>>>>>> Are you using HDA that contain the complete scene?
>>>>>>>> a irritated Georg
>>>>>>>> _______________________________________________
>>>>>>>> Sidefx-houdini-list mailing list
>>>>>>>> Sidefx-houdini-list at sidefx.com
>>>>>>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>>>>>>> --
>>>>>>> "gravity is not a force, it is a boundary layer"
>>>>>>> "everything is coincident"
>>>>>>> "Love: the state of suspended anticipation"
>>>>>>> "To get any appreciable distance from the Earth in
>>>>>>> a sensible amount of time, you must lie."
>>>>>>> "Easy, is not how you make a living."
>>>>>>> _______________________________________________
>>>>>>> Sidefx-houdini-list mailing list
>>>>>>> Sidefx-houdini-list at sidefx.com
>>>>>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini- 
>>>>>>> list
>>>>>> ---------
>>>>>> Un saludo
>>>>>> Best Regards
>>>>>> Pablo Giménez
>>>>>> pablogipi at gmail.com
>>>>>> _______________________________________________
>>>>>> Sidefx-houdini-list mailing list
>>>>>> Sidefx-houdini-list at sidefx.com
>>>>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>>>>> _______________________________________________
>>>>> Sidefx-houdini-list mailing list
>>>>> Sidefx-houdini-list at sidefx.com
>>>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>>>> ----
>>>> Robert Magee
>>>> Product Marketing Manager
>>>> Side Effects Software Inc.
>>>> www.sidefx.com
>>>> 416 504-9876 x327
>>>> _______________________________________________
>>>> Sidefx-houdini-list mailing list
>>>> Sidefx-houdini-list at sidefx.com
>>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>>> _______________________________________________
>>> Sidefx-houdini-list mailing list
>>> Sidefx-houdini-list at sidefx.com
>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
> _______________________________________________
> Sidefx-houdini-list mailing list
> Sidefx-houdini-list at sidefx.com
> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list

Un saludo
Best Regards
Pablo Giménez
pablogipi at gmail.com

More information about the Sidefx-houdini-list mailing list