[Sidefx-houdini-list] HDK: op:/ magic

Olex P hoknamahn at gmail.com
Fri Jan 21 12:44:54 EST 2011


Hi Szymon,

VEX itself has nothing to do with multithreading. But VEX engine (standard
contexts or your own CVEX one) does. So in the case you mentioned (VEX SOP
that is reading a point cloud and assigning a filtered value to a point) it
works as (simplified):
1. VEX engine (SOP context) creates a flat array and fills it in with the
attributes from geometry
2. VEX engine spawns several threads and runs one instance of VEX procedure
in each thread
3. Each instance of VEX procedure (single threaded) filters a value from
point cloud and writes into it's chunk of flat array
4. VEX engine writes all the values back from flat array into a GDP using
one thread
Honestly I haven't tried multithreading with GDP but I see no problems with
it as long as you are not modifying the data. And normally VEX function
itself doesn't modify a GDP (but VEX engine does).
Well that's how I understand it and I can be wrong :)
There are some bits about thread safety in HDK
http://www.sidefx.com/docs/hdk11.0/hdk_changes_11_0.html#HDKC11_0_ThreadSafety
I probably have to do some tests too :)

On Fri, Jan 21, 2011 at 5:13 PM, Szymon Kapeniak
<szymon.kapeniak at gmail.com>wrote:

> Hey Olex,
> I'm writing custom vex, and I'm sure vex has some standard methods to
> deal with this. I'm only interested in read access from several
> threads which I don't have any control on, as they are spawned by VEX
> SOP.
>
> I started to play with it by checking minimal scenarios and it seems
> it's enough to declare one static GU_Detail instance and assign
> pointers to it from a couple of threads to crash Houdini even without
> loading any geometry.
> Also by checking threads ids I realized, that mantra can run the same
> code multi-threaded because vex in surface context call init()
> (standard procedure) per thread, unlike in sop context, where it seems
> init() is called only once like in fact it's described in docs.
>
> Surly there are thread-save vex functions (reading gdp) which operate
> in sop context, like pcopen().  I'm guessing the trick is not to use
> pure gdp, but some  thread-safe memory schema. Lack of documentation
> doesn't play well with my ignorance...
>
> skk.
>
> btw  I'm referring to this talk:
>
> http://forums.odforce.net/index.php?/topic/12434-how-to-replicate-this-ice-network-in-vops/page__st__12
>
> 2011/1/21 Olex P <hoknamahn at gmail.com>:
> > Hey Szymon,
> >
> > You can read in multiple threads but multithread writing as far as I know
> is
> > limited to volume primitives (each thread can write into it's own tile)
> as
> > far as I know. You could create own GU/GEO_Detail object for each thread,
> > write into it and then merge all the pieces of puzzle into one object
> (merge
> > will single thread).
> > But from your words I'm not sure what kind of parallelism you are looking
> > for...
> > Let's say sampling from volume in VEX. 1. You are running the same single
> > thread code on different chunks of data using multiple threads. 2. But
> this
> > also could be done in a different way: vex function internally spawns
> > several threads which modify the data somehow. You probably talking about
> 2
> > (as you mentioned multithread access to GU_Detail). I'm not sure but
> maybe
> > it makes sense to use 1 (i.e. CVEX). Sure it will work only for data
> > parallel algorithms.
> >
> > On Thu, Jan 20, 2011 at 10:20 PM, Szymon Kapeniak <
> szymon.kapeniak at gmail.com
> >> wrote:
> >
> >> ...sorry for spamming, but I've just realized that the same dso works
> >> in mantra, but not in (threading) sop context. Kind of strange.
> >>
> >> 2011/1/20 Szymon Kapeniak <szymon.kapeniak at gmail.com>:
> >> > Thanks for a response, I didn't know I have to deal with it manually.
> >> >
> >> > Second problem is threading in vex sop. After playing with my function
> >> > a little, I'm getting a feeling my approach was too naive, so to
> >> > speak.
> >> >
> >> > Is multi-threaded access to a static GU_Detail * possible at all, or
> >> > do I have to use some caches like UT_ThreadSafeCache or anything
> >> > similar? What is a general way of doing this in HDK?
> >> >
> >> > thanks,
> >> > skk.
> >> >
> >> >
> >> > 2011/1/19 Olex P <hoknamahn at gmail.com>:
> >> >> It kind of does writing an op-referenced texture into an IFD :)
> >> >>
> >> >> On Wed, Jan 19, 2011 at 1:48 PM, Mark Elendt <mark at sidefx.com>
> wrote:
> >> >>
> >> >>> For mantra, even Houdini's VEX doesn't support the op: syntax :)
> >> >>>
> >> >>> For VEX functions in OPs, you should be able to access the SOP by
> >> >>> grabbing the OP director and finding the SOP.
> >> >>>
> >> >>> On Wednesday Jan 19 at 13:39, Szymon Kapeniak wrote:
> >> >>> > thanks Mark,
> >> >>> > but how about custom vex dso? My func() opens file from disk, but
> >> >>> > doesn't handle op: syntax. Is it doable in custom code?
> >> >>> >
> >> >>> > thanks,
> >> >>> > skk.
> >> >>> >
> >> >>> > 2011/1/19 Mark Elendt <mark at sidefx.com>:
> >> >>> > > Unfortunately, at the current time op: is handled as a special
> >> case,
> >> >>> > > so it's not part of the generalized loader.
> >> >>> > >
> >> >>> > > You'll have to reference the geometry directly (i.e. cook the
> SOP).
> >> >>> > > If you need your own copy, you'll have to merge it.
> >> >>> > >
> >> >>> > > On the other hand, opdef: is handled properly.
> >> >>> > >
> >> >>> > > On Wednesday Jan 19 at 12:11, Olex P wrote:
> >> >>> > >> Maybe you right Szymon. Most of the usage of op: syntax for me
> was
> >> >>> sampling
> >> >>> > >> from point clouds/volumes/textures and I have never tries to
> load
> >> >>> geometry
> >> >>> > >> using op: syntax. File SOP doesn't work with it and I think
> it's
> >> >>> because of
> >> >>> > >> the fact that load() expects a proper file format. Would be
> great
> >> to
> >> >>> hear
> >> >>> > >> about this from SESI.
> >> >>> > >>
> >> >>> > >> On Wed, Jan 19, 2011 at 11:58 AM, Szymon Kapeniak <
> >> >>> szymon.kapeniak at gmail.com
> >> >>> > >> > wrote:
> >> >>> > >>
> >> >>> > >> > hi Olex,
> >> >>> > >> > thanks for fast response. Well, I must be doing something
> >> stupid,
> >> >>> > >> > because it doesn't.
> >> >>> > >> >
> >> >>> > >> > GU_Detail gdp;
> >> >>> > >> > const char *file   = (const char *) argv[1];
> >> >>> > >> > if (gdp.load(file, 0) < 0) return ;
> >> >>> > >> > ...
> >> >>> > >> >
> >> >>> > >> > while works fine with disk paths, doesn't work with op:/'ed
> one.
> >> >>> > >> >
> >> >>> > >> >
> >> >>> > >> > 2011/1/19 Olex P <hoknamahn at gmail.com>:
> >> >>> > >> > > Hi Szymon,
> >> >>> > >> > >
> >> >>> > >> > > I don't think that you need any magic. op: syntax has to be
> >> >>> handled
> >> >>> > >> > > automagically when you using
> >> >>> > >> > >
> >> >>> > >> > > [code]virtual int          load(const char *, const
> UT_Options
> >> >>> > >> > > *options);[/code]
> >> >>> > >> > >
> >> >>> > >> > > method.
> >> >>> > >> > >
> >> >>> > >> > > On Wed, Jan 19, 2011 at 11:41 AM, Szymon Kapeniak <
> >> >>> > >> > szymon.kapeniak at gmail.com
> >> >>> > >> > >> wrote:
> >> >>> > >> > >
> >> >>> > >> > >> greetings!
> >> >>> > >> > >>
> >> >>> > >> > >> what is a magic to let  my GU_Detail class to ::load()
> both
> >> files
> >> >>> from
> >> >>> > >> > >> disk as operator's streams with op:/path syntax? I can see
> >> load()
> >> >>> > >> > >> with an UT_IStream as argument, but I have no idea how to
> use
> >> it.
> >> >>> Does
> >> >>> > >> > >> any of UT_Stream versions handle both files and steams or
> the
> >> >>> logic is
> >> >>> > >> > >> on my side?
> >> >>> > >> > >>
> >> >>> > >> > >> thanks,
> >> >>> > >> > >> skk.
> >> >>> > >> > >> _______________________________________________
> >> >>> > >> > >> Sidefx-houdini-list mailing list
> >> >>> > >> > >> Sidefx-houdini-list at sidefx.com
> >> >>> > >> > >>
> >> >>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
> >> >>> > >> > >>
> >> >>> > >> > > _______________________________________________
> >> >>> > >> > > Sidefx-houdini-list mailing list
> >> >>> > >> > > Sidefx-houdini-list at sidefx.com
> >> >>> > >> > >
> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
> >> >>> > >> > >
> >> >>> > >> > _______________________________________________
> >> >>> > >> > Sidefx-houdini-list mailing list
> >> >>> > >> > Sidefx-houdini-list at sidefx.com
> >> >>> > >> >
> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
> >> >>> > >> >
> >> >>> > >> _______________________________________________
> >> >>> > >> Sidefx-houdini-list mailing list
> >> >>> > >> Sidefx-houdini-list at sidefx.com
> >> >>> > >>
> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
> >> >>> > >
> >> >>> > > --
> >> >>> > > Mark Elendt
> >> >>> > > _______________________________________________
> >> >>> > > Sidefx-houdini-list mailing list
> >> >>> > > Sidefx-houdini-list at sidefx.com
> >> >>> > >
> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
> >> >>> > >
> >> >>> > _______________________________________________
> >> >>> > Sidefx-houdini-list mailing list
> >> >>> > Sidefx-houdini-list at sidefx.com
> >> >>> > https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
> >> >>>
> >> >>> --
> >> >>> Mark Elendt
> >> >>> _______________________________________________
> >> >>> Sidefx-houdini-list mailing list
> >> >>> Sidefx-houdini-list at sidefx.com
> >> >>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
> >> >>>
> >> >> _______________________________________________
> >> >> Sidefx-houdini-list mailing list
> >> >> Sidefx-houdini-list at sidefx.com
> >> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
> >> >>
> >> >
> >> _______________________________________________
> >> Sidefx-houdini-list mailing list
> >> Sidefx-houdini-list at sidefx.com
> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
> >>
> > _______________________________________________
> > Sidefx-houdini-list mailing list
> > Sidefx-houdini-list at sidefx.com
> > https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
> >
> _______________________________________________
> Sidefx-houdini-list mailing list
> Sidefx-houdini-list at sidefx.com
> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>



More information about the Sidefx-houdini-list mailing list