[Sidefx-houdini-list] disappearing HDA parameters
Andrew D Lyons
tstexture at gmail.com
Fri Jun 6 12:27:24 EDT 2008
There are a lot of good revision contols systems out there for *nix,
and SESI is better off providing hooks for those rather than trying to
embed one for now (like the system in the video). Here's some more
ideas for providing some basic support:
- Tighten up hooks for versioning HDAs so people don't have to bake
version numbers into hda file names to avoid conflicts in a hipfile.
- Have hda version readable and writable (with version write/lock
permissions) from disk. ie expand hotl interface to permit definition
- Have HDA manager in Houdini recognise versions.
- Have Galleries recognise and support hda versions.
- Swap out the filesystem oriented tree in the HDA manager for a more
production-oriented tree in both hda and gallery. ie.
- Allow hdas to be tagged with job-sequence-shot-task metadata.
- Allow galleries to be tagged with job-sequence-shot-task metadata.
- A hotl, hom api, and gu interface for "promoting" and "demoting"
a hda version in the production-oriented tree.
- Automatically have the manager give priority to topmost
production-oriented tree defs when instantiating new ops - even if
that means ignoring versions.
- keep disk location visible and manageable, but shift emphasis to
the production abstraction.
- Provide hooks for calling a revision control system on an otl file:
- provide a checkout, checkin hook etc.
- provide the hda manager a hook to test the lock status of an otl file
- Multiple instances of the same HDA (name) in a scene hda manager
causes huge problems. Currently otl file location differences are the
mechanism that permits this.
- disallow multiple hda definitions to be loaded into a hip file
that have the same version and production tree setting - even if they
come from different disk directories.
- An otl file should only ever contain a single hda. otls that
store multiple hdas have always created more problems than they solved
IMHO, and should be flagged with a warning.
- A new "Digital Asset Set" graph structure with accompanying file
format. To allow people to manage "sets" of otls.
- Sets should work with the production-oriented tree position,
version values, and disk location.
- Set Manager API for HOM
- Set manager GUI (in standalone mode as well perhaps?)
That's it for now.
2008/6/6 paul simpson <paul at realisestudio.com>:
> hi robert,
> thanks for putting this link up. i think this summarises the magnitude of
> the misunderstanding over the last few years. it seems most people
> (including sesi?) are getting confused with digital asset management (ie,
> textures, external shaders, geometery) and HDA package management (ie,
> digital asset dependences). to me, the former is an open ended potential
> quagmire that i would rather sesi leave alone for now. the latter is imho
> an essential tool that we needed (4 years ago) in order to create and
> package HDA's that in turn use other HDA's within them. the two are *very*
> different subjects! (you could - in time treat all external files as
> digital assets - buts that an entirely different topic for now).
> to reiterate - i see the latter as very important as i would like to share
> assets from old jobs, other shops and houdini exchange without having to
> worry about which other assets each one is using. really, we need all the
> assets to be included in a search path - and each asset can use either the
> latest - or an earlier version all happily and transparently. i'm thinking
> about continuity between jobs here. i see this as a universal requirement.
> i would like this logic to be implemented initially in a bare bones fashion
> by sesi - and made open so individual shops can then taylor to their own
> needs. (please look at debian package management for a great example of
> what i mean).
> the video you point us to refers to the former. so, to answer your question
> - i would rather any have the hda dependency system implemented rather than
> this tool you refer to.
> we all need to be clear what we're trying to talk about. part of this
> problem is sesis fault naming HDA's or (OTL's) as "digital assets". it's a
> very generic term within the industry which often refers to texture maps,
> geometry files - as in the 3dsmax example you posted. nomenclature is very
> important - and needs to be 'normalised' somewhat - or at least
> de-obfuscated. sesi needs to take the bull by the horns and get more
> production savvy asap.
> this subject has been talked about at cross purposes for over five years
> now! :-)
> 2008/6/6 Robert Magee <rmagee at sidefx.com>:
>> A couple of releases ago, Autodesk put out this movie for their asset
>> tracking in Max8. It starts out as a single window where all the
>> dependencies in a scene are laid out and their current status is
>> outlined (links broken, etc..). From this window links can be re-
>> established and even swapped out for other versions as needed. This
>> system offers a one-stop-shopping for working with local assets. They
>> can then extend that to a network as a "stage 2" of the system:
>> Without going to a full blow asset management system (aka tactic),
>> would something like this meet the needs outlined in this thread? It
>> would be very helpful to figure out exactly how far people would want
>> us to go. When we spoke with customers on this in the past, the
>> requests seemed to lean towards a system that was open and not tied
>> too tightly with Houdini. A Houdini-centric set of tools might be
>> easier in the short term to get people started.
>> On 6-Jun-08, at 10:21 AM, Ken Ouellette wrote:
>> > Paul's problems (I'm assuming here, but based on a hunch of how he has
>> > worked in the past) is partly on how he is using HDAs. To that
>> > regard I
>> > don't think he is out of line asking for a bit more support to make
>> > the
>> > packaging and transport/sharing of HDAs easier.
>> > Yes for a small centralized studio it is easier to self manage and
>> > regulate
>> > the changes. For a larger pipeline people will naturally build tools
>> > to
>> > support some sort of revision system, even it is just a slight twist
>> > on the
>> > autoBackup of HDAs and symlinking those to a none versioned reference.
>> > Opdepend was an helpful addition for getting some help, but the
>> > process can
>> > still become complex quickly.
>> > There could be some things that Sesi could do to support
>> > encapsulated HDA in
>> > HDAs a bit easier. That has always been a bit of a problem as
>> > finding what
>> > is inside one and understanding its dependencies can be a bit of a
>> > pain.
>> > Then add to the fact that finding certain disk based references
>> > (textures,
>> > geometry, etc) can be a again a bit of a haul to make it work easily.
>> > Hopefully some of those basics are easier to handle with python. I
>> > haven't
>> > looked, but even if you made up your own methods to handle all the
>> > cases it
>> > would take you a while. It all depends on how absolute you want to
>> > be in
>> > your version of the asset, not just the HDA. The HDA itself is a
>> > container
>> > with parameters, you can inject some of its components (geometry,
>> > and low
>> > rez textures) but other items like high rez textures, and rib/ifd
>> > files
>> > either don't work in the context (I know Renderman doesn't like it,
>> > not sure
>> > about Mantra). So when it comes down to it there are a lot of
>> > support files
>> > you need to consider when you are planning out your pipe.
>> > It gets more complex if you start to encapsulate HDAs inside HDAs
>> > and then
>> > it can become very important for the order in the loading/
>> > instantiation and
>> > then add the dependency issues across these nested HDAs. The
>> > flexibility in
>> > Houdini makes it really difficult to contain. Having had a large
>> > role in
>> > building two asset management systems for Houdini, it is very
>> > difficult to
>> > keep things agnostic and actually requires more structure then you
>> > would
>> > expect. There are many ways to create HDAs and it becomes a job
>> > just to
>> > support the different methods. That is part of a natural process of
>> > having
>> > and maintaining a pipeline vision and should be expect with every
>> > system
>> > though. I do feel it is more of a challenge in Houdini than in other
>> > products ( <cough>Maya<cough>)
>> > When I saw what Sesi was doing with python and H9 I really stressed
>> > to them
>> > that with a hand full of people any studio could quickly flesh out a
>> > very
>> > sospicated pipeline if they had some experience in doing it in
>> > previous
>> > versions. Having Sesi provide blue print type examples on how to set
>> > up a
>> > system would be great but also starts a slippery slope for them.
>> > Where does
>> > providing added value stop with support? That is a difficult problem
>> > for any
>> > software vendor and one most avoid. I can't blame them
>> > The embedded browser is very helpful to develop an app to handle the
>> > management of the assets, but there are challenges that inherit to
>> > using
>> > that environment. Thankfully Sidefx has made huge efforts to remove
>> > a lot of
>> > these but there are still cycles to be spent finding ways to around
>> > them. In
>> > the end it is the same problem of Sidefx entering a slippery slope
>> > of where
>> > to draw the line for support.
>> > For "a irritated Georg"
>> > Encapsulation for a entire can be easy to implement but has serious
>> > draw
>> > backs for updates, both HDA wise and parameters. On the Wild we had
>> > a fairly
>> > strict rule that we supported only HDAs, which added its own
>> > complexity to
>> > making things (think of how hard it is/was to make a dynamic
>> > parenting HDA).
>> > We knew what HDA's whet into a hipfile and what verison they were,
>> > we also
>> > saved out all the animation for the files. In some cases we wrote out
>> > geometry and had special HDAs to handle those cases.
>> > Saving and versioning entire hipfiles and keeping track of the
>> > versions of
>> > any HDAs inside.... for a production pipeline is a bit easier and
>> > can offer
>> > some benefits, but still has some drawbacks. That method that was
>> > used as
>> > well post "the Wild" in a number of shows. There are always
>> > challenges
>> > about updating versions and handling animation.cmd files.....
>> > keeping track
>> > of changes between saved hipfiles when it comes to non-HDA elements
>> > anyone
>> > can create in Houdini. Connections parenting all that jazz
>> > The real power is to find a method to manage all the information and
>> > push
>> > changes out on the fly. Then you complete the circle of the pipeline
>> > and you
>> > have the ability to make large sweeping changes effecting renders
>> > going to
>> > the farm.
>> > I should stop now.. that is long enough to read :)
>> > -k
>> > On Fri, Jun 6, 2008 at 7:48 AM, Pablo Giménez Pizarro <
>> pablogipi at gmail.com
>> > >
>> > wrote:
>> >> El 06/06/2008, a las 13:33, Peter Robbinson escribió:
>> >>> Hi,
>> >>> It had a lot to do with the asset manager.
>> >>> There were specialists, Michael being an important one, who were the
>> >>> responsible for developing and maintaining the digital assets.
>> >>> The artists on the production floor accessed the appropriate assets
>> >>> for
>> >>> a given scene through the embedded browser. Core used the embedded
>> >>> browser to present an interface to the asset manager. Artists only
>> >>> had
>> >>> access to the approved assets for an assigned scene. There were ways
>> >>> to
>> >>> access earlier versions of assets, but only when problems were
>> >>> encountered. The assest manager allowed artists to report problems
>> >>> (bug
>> >>> reports) and these were automatically reported to the appropriate
>> >>> department.
>> >>> Michael play a big part in getting that system to work, it was quite
>> >>> remarkable. That's not to say it worked well all the time, there
>> >>> were
>> >>> plenty of work-arounds necessary and it took time to get working
>> >>> right,
>> >>> but it was certainly the best pipeline I had ever encountered.
>> >>> Keep in
>> >>> mind Core used version 6 of houdini on that production and digital
>> >>> assets were brand new.
>> >> Thanks for the explanation Peter.
>> >> Now I think that you are agree with me that to take the most from HDA
>> >> you need some management tool.
>> >> I am not talking to have this complex system developed at CORE but
>> >> something simple and easy for the small studios that use tools that
>> >> can be used by big houses to develop their own management systems
>> >> (like the otlversion command for example)
>> >> In my opinion SESI did really well with the character picker and pose
>> >> library tools, something like this, in the browser will be great. The
>> >> pose library for example is not the most advanced tool, but it works
>> >> out of the box and for a small studio do the job and if you need to
>> >> extent it you can.
>> >>> PeterR
>> >>> Georg Dümlein wrote:
>> >>>> Michael Goldfarb <goldfarb at coredp.com> schrieb:
>> >>>>> The Wild was made almost entirely (like 95%+) without hip files...
>> >>>> Sorry for the noobish question , but:
>> >>>> How does this work?
>> >>>> Are you using HDA that contain the complete scene?
>> >>>> a irritated Georg
>> >>>> _______________________________________________
>> >>>> Sidefx-houdini-list mailing list
>> >>>> Sidefx-houdini-list at sidefx.com
>> >>>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >>> --
>> >>> "gravity is not a force, it is a boundary layer"
>> >>> "everything is coincident"
>> >>> "Love: the state of suspended anticipation"
>> >>> "To get any appreciable distance from the Earth in
>> >>> a sensible amount of time, you must lie."
>> >>> "Easy, is not how you make a living."
>> >>> _______________________________________________
>> >>> Sidefx-houdini-list mailing list
>> >>> Sidefx-houdini-list at sidefx.com
>> >>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> ---------
>> >> Un saludo
>> >> Best Regards
>> >> Pablo Giménez
>> >> pablogipi at gmail.com
>> >> _______________________________________________
>> >> Sidefx-houdini-list mailing list
>> >> Sidefx-houdini-list at sidefx.com
>> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> > _______________________________________________
>> > Sidefx-houdini-list mailing list
>> > Sidefx-houdini-list at sidefx.com
>> > https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> Robert Magee
>> Product Marketing Manager
>> Side Effects Software Inc.
>> 416 504-9876 x327
>> Sidefx-houdini-list mailing list
>> Sidefx-houdini-list at sidefx.com
> Sidefx-houdini-list mailing list
> Sidefx-houdini-list at sidefx.com
Andrew D Lyons | Digital Artist | http://www.tstex.com
More information about the Sidefx-houdini-list