[Sidefx-houdini-list] HDK: op:/ magic

Szymon Kapeniak szymon.kapeniak at gmail.com
Fri Jan 21 13:30:45 EST 2011


The problem is I don't have a chance to access any attribute, read,
write, what so ever. I declare gdp as static pointer and create new
GU_Detail inside init(), which meant to run once before threads start,
then I point threads to that gdp as advised in docs. It's enough to
crash Houdini in sop context.

2011/1/21 Olex P <hoknamahn at gmail.com>:
> Hi Szymon,
>
> VEX itself has nothing to do with multithreading. But VEX engine (standard
> contexts or your own CVEX one) does. So in the case you mentioned (VEX SOP
> that is reading a point cloud and assigning a filtered value to a point) it
> works as (simplified):
> 1. VEX engine (SOP context) creates a flat array and fills it in with the
> attributes from geometry
> 2. VEX engine spawns several threads and runs one instance of VEX procedure
> in each thread
> 3. Each instance of VEX procedure (single threaded) filters a value from
> point cloud and writes into it's chunk of flat array
> 4. VEX engine writes all the values back from flat array into a GDP using
> one thread
> Honestly I haven't tried multithreading with GDP but I see no problems with
> it as long as you are not modifying the data. And normally VEX function
> itself doesn't modify a GDP (but VEX engine does).
> Well that's how I understand it and I can be wrong :)
> There are some bits about thread safety in HDK
> http://www.sidefx.com/docs/hdk11.0/hdk_changes_11_0.html#HDKC11_0_ThreadSafety
> I probably have to do some tests too :)
>
> On Fri, Jan 21, 2011 at 5:13 PM, Szymon Kapeniak
> <szymon.kapeniak at gmail.com>wrote:
>
>> Hey Olex,
>> I'm writing custom vex, and I'm sure vex has some standard methods to
>> deal with this. I'm only interested in read access from several
>> threads which I don't have any control on, as they are spawned by VEX
>> SOP.
>>
>> I started to play with it by checking minimal scenarios and it seems
>> it's enough to declare one static GU_Detail instance and assign
>> pointers to it from a couple of threads to crash Houdini even without
>> loading any geometry.
>> Also by checking threads ids I realized, that mantra can run the same
>> code multi-threaded because vex in surface context call init()
>> (standard procedure) per thread, unlike in sop context, where it seems
>> init() is called only once like in fact it's described in docs.
>>
>> Surly there are thread-save vex functions (reading gdp) which operate
>> in sop context, like pcopen().  I'm guessing the trick is not to use
>> pure gdp, but some  thread-safe memory schema. Lack of documentation
>> doesn't play well with my ignorance...
>>
>> skk.
>>
>> btw  I'm referring to this talk:
>>
>> http://forums.odforce.net/index.php?/topic/12434-how-to-replicate-this-ice-network-in-vops/page__st__12
>>
>> 2011/1/21 Olex P <hoknamahn at gmail.com>:
>> > Hey Szymon,
>> >
>> > You can read in multiple threads but multithread writing as far as I know
>> is
>> > limited to volume primitives (each thread can write into it's own tile)
>> as
>> > far as I know. You could create own GU/GEO_Detail object for each thread,
>> > write into it and then merge all the pieces of puzzle into one object
>> (merge
>> > will single thread).
>> > But from your words I'm not sure what kind of parallelism you are looking
>> > for...
>> > Let's say sampling from volume in VEX. 1. You are running the same single
>> > thread code on different chunks of data using multiple threads. 2. But
>> this
>> > also could be done in a different way: vex function internally spawns
>> > several threads which modify the data somehow. You probably talking about
>> 2
>> > (as you mentioned multithread access to GU_Detail). I'm not sure but
>> maybe
>> > it makes sense to use 1 (i.e. CVEX). Sure it will work only for data
>> > parallel algorithms.
>> >
>> > On Thu, Jan 20, 2011 at 10:20 PM, Szymon Kapeniak <
>> szymon.kapeniak at gmail.com
>> >> wrote:
>> >
>> >> ...sorry for spamming, but I've just realized that the same dso works
>> >> in mantra, but not in (threading) sop context. Kind of strange.
>> >>
>> >> 2011/1/20 Szymon Kapeniak <szymon.kapeniak at gmail.com>:
>> >> > Thanks for a response, I didn't know I have to deal with it manually.
>> >> >
>> >> > Second problem is threading in vex sop. After playing with my function
>> >> > a little, I'm getting a feeling my approach was too naive, so to
>> >> > speak.
>> >> >
>> >> > Is multi-threaded access to a static GU_Detail * possible at all, or
>> >> > do I have to use some caches like UT_ThreadSafeCache or anything
>> >> > similar? What is a general way of doing this in HDK?
>> >> >
>> >> > thanks,
>> >> > skk.
>> >> >
>> >> >
>> >> > 2011/1/19 Olex P <hoknamahn at gmail.com>:
>> >> >> It kind of does writing an op-referenced texture into an IFD :)
>> >> >>
>> >> >> On Wed, Jan 19, 2011 at 1:48 PM, Mark Elendt <mark at sidefx.com>
>> wrote:
>> >> >>
>> >> >>> For mantra, even Houdini's VEX doesn't support the op: syntax :)
>> >> >>>
>> >> >>> For VEX functions in OPs, you should be able to access the SOP by
>> >> >>> grabbing the OP director and finding the SOP.
>> >> >>>
>> >> >>> On Wednesday Jan 19 at 13:39, Szymon Kapeniak wrote:
>> >> >>> > thanks Mark,
>> >> >>> > but how about custom vex dso? My func() opens file from disk, but
>> >> >>> > doesn't handle op: syntax. Is it doable in custom code?
>> >> >>> >
>> >> >>> > thanks,
>> >> >>> > skk.
>> >> >>> >
>> >> >>> > 2011/1/19 Mark Elendt <mark at sidefx.com>:
>> >> >>> > > Unfortunately, at the current time op: is handled as a special
>> >> case,
>> >> >>> > > so it's not part of the generalized loader.
>> >> >>> > >
>> >> >>> > > You'll have to reference the geometry directly (i.e. cook the
>> SOP).
>> >> >>> > > If you need your own copy, you'll have to merge it.
>> >> >>> > >
>> >> >>> > > On the other hand, opdef: is handled properly.
>> >> >>> > >
>> >> >>> > > On Wednesday Jan 19 at 12:11, Olex P wrote:
>> >> >>> > >> Maybe you right Szymon. Most of the usage of op: syntax for me
>> was
>> >> >>> sampling
>> >> >>> > >> from point clouds/volumes/textures and I have never tries to
>> load
>> >> >>> geometry
>> >> >>> > >> using op: syntax. File SOP doesn't work with it and I think
>> it's
>> >> >>> because of
>> >> >>> > >> the fact that load() expects a proper file format. Would be
>> great
>> >> to
>> >> >>> hear
>> >> >>> > >> about this from SESI.
>> >> >>> > >>
>> >> >>> > >> On Wed, Jan 19, 2011 at 11:58 AM, Szymon Kapeniak <
>> >> >>> szymon.kapeniak at gmail.com
>> >> >>> > >> > wrote:
>> >> >>> > >>
>> >> >>> > >> > hi Olex,
>> >> >>> > >> > thanks for fast response. Well, I must be doing something
>> >> stupid,
>> >> >>> > >> > because it doesn't.
>> >> >>> > >> >
>> >> >>> > >> > GU_Detail gdp;
>> >> >>> > >> > const char *file   = (const char *) argv[1];
>> >> >>> > >> > if (gdp.load(file, 0) < 0) return ;
>> >> >>> > >> > ...
>> >> >>> > >> >
>> >> >>> > >> > while works fine with disk paths, doesn't work with op:/'ed
>> one.
>> >> >>> > >> >
>> >> >>> > >> >
>> >> >>> > >> > 2011/1/19 Olex P <hoknamahn at gmail.com>:
>> >> >>> > >> > > Hi Szymon,
>> >> >>> > >> > >
>> >> >>> > >> > > I don't think that you need any magic. op: syntax has to be
>> >> >>> handled
>> >> >>> > >> > > automagically when you using
>> >> >>> > >> > >
>> >> >>> > >> > > [code]virtual int          load(const char *, const
>> UT_Options
>> >> >>> > >> > > *options);[/code]
>> >> >>> > >> > >
>> >> >>> > >> > > method.
>> >> >>> > >> > >
>> >> >>> > >> > > On Wed, Jan 19, 2011 at 11:41 AM, Szymon Kapeniak <
>> >> >>> > >> > szymon.kapeniak at gmail.com
>> >> >>> > >> > >> wrote:
>> >> >>> > >> > >
>> >> >>> > >> > >> greetings!
>> >> >>> > >> > >>
>> >> >>> > >> > >> what is a magic to let  my GU_Detail class to ::load()
>> both
>> >> files
>> >> >>> from
>> >> >>> > >> > >> disk as operator's streams with op:/path syntax? I can see
>> >> load()
>> >> >>> > >> > >> with an UT_IStream as argument, but I have no idea how to
>> use
>> >> it.
>> >> >>> Does
>> >> >>> > >> > >> any of UT_Stream versions handle both files and steams or
>> the
>> >> >>> logic is
>> >> >>> > >> > >> on my side?
>> >> >>> > >> > >>
>> >> >>> > >> > >> thanks,
>> >> >>> > >> > >> skk.
>> >> >>> > >> > >> _______________________________________________
>> >> >>> > >> > >> Sidefx-houdini-list mailing list
>> >> >>> > >> > >> Sidefx-houdini-list at sidefx.com
>> >> >>> > >> > >>
>> >> >>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >>> > >> > >>
>> >> >>> > >> > > _______________________________________________
>> >> >>> > >> > > Sidefx-houdini-list mailing list
>> >> >>> > >> > > Sidefx-houdini-list at sidefx.com
>> >> >>> > >> > >
>> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >>> > >> > >
>> >> >>> > >> > _______________________________________________
>> >> >>> > >> > Sidefx-houdini-list mailing list
>> >> >>> > >> > Sidefx-houdini-list at sidefx.com
>> >> >>> > >> >
>> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >>> > >> >
>> >> >>> > >> _______________________________________________
>> >> >>> > >> Sidefx-houdini-list mailing list
>> >> >>> > >> Sidefx-houdini-list at sidefx.com
>> >> >>> > >>
>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >>> > >
>> >> >>> > > --
>> >> >>> > > Mark Elendt
>> >> >>> > > _______________________________________________
>> >> >>> > > Sidefx-houdini-list mailing list
>> >> >>> > > Sidefx-houdini-list at sidefx.com
>> >> >>> > >
>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >>> > >
>> >> >>> > _______________________________________________
>> >> >>> > Sidefx-houdini-list mailing list
>> >> >>> > Sidefx-houdini-list at sidefx.com
>> >> >>> > https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >>>
>> >> >>> --
>> >> >>> Mark Elendt
>> >> >>> _______________________________________________
>> >> >>> Sidefx-houdini-list mailing list
>> >> >>> Sidefx-houdini-list at sidefx.com
>> >> >>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >>>
>> >> >> _______________________________________________
>> >> >> Sidefx-houdini-list mailing list
>> >> >> Sidefx-houdini-list at sidefx.com
>> >> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >>
>> >> >
>> >> _______________________________________________
>> >> Sidefx-houdini-list mailing list
>> >> Sidefx-houdini-list at sidefx.com
>> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >>
>> > _______________________________________________
>> > Sidefx-houdini-list mailing list
>> > Sidefx-houdini-list at sidefx.com
>> > https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >
>> _______________________________________________
>> Sidefx-houdini-list mailing list
>> Sidefx-houdini-list at sidefx.com
>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>>
> _______________________________________________
> Sidefx-houdini-list mailing list
> Sidefx-houdini-list at sidefx.com
> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>



More information about the Sidefx-houdini-list mailing list