[Sidefx-houdini-list] HDK: op:/ magic

Szymon Kapeniak szymon.kapeniak at gmail.com
Fri Jan 21 19:17:11 EST 2011


That would be fantastic! But is your geometry enough heavy to start
multi-threading? Grid 100x100 should be fine, but anything smaller
doesn't use threads even if you set it on a vex node.

Secondly, do you allocate new GU_Detail and load bgeo inside init()? I
would love to do that, but I don't know how to pass user arguments
there. It doesn't seem to take any, so all I have must take place
inside main function. I may be wrong also in this point though...

I'll try your example tomorrow.

Thank you Olex!

2011/1/22 Olex P <hoknamahn at gmail.com>:
> Hi Szymon,
>
> I've made a simple VEX dso which creates a static pointer to GU_Detail
>
> static GU_Detail* geo = NULL;
>
> allocates new object on Init()
>
> if(!geo)
> {
>  geo = new GU_Detail();
>  geo->load("/tmp/something.bgeo", 0);
> }
>
> takes an attribute value on Evaluate and returns it
>
> GEO_Point* pt = geo->points()[ptnum];
> GB_AttributeRef somethingIndex = geo->findPointAttrib("something",
> GB_ATTRIB_FLOAT);
> const float* something = pt->castAttribData<float>(somethingIndex);
> *result = *something;
>
> VEX op itself looks like:
>
> sop
> vextest(export float newsomething = 0)
> {
>    newsomething = myfun((float)ptnum);
> }
>
> So we create a geometry, apply an attribute on it, save somewhere. Then
> apply this VEX op to clean (without an attribute geo) and spreadsheet show
> exactly the same values as where written into bgeo.
> It works fine.
> On Fri, Jan 21, 2011 at 6:30 PM, Szymon Kapeniak
> <szymon.kapeniak at gmail.com>wrote:
>
>> The problem is I don't have a chance to access any attribute, read,
>> write, what so ever. I declare gdp as static pointer and create new
>> GU_Detail inside init(), which meant to run once before threads start,
>> then I point threads to that gdp as advised in docs. It's enough to
>> crash Houdini in sop context.
>>
>> 2011/1/21 Olex P <hoknamahn at gmail.com>:
>> > Hi Szymon,
>> >
>> > VEX itself has nothing to do with multithreading. But VEX engine
>> (standard
>> > contexts or your own CVEX one) does. So in the case you mentioned (VEX
>> SOP
>> > that is reading a point cloud and assigning a filtered value to a point)
>> it
>> > works as (simplified):
>> > 1. VEX engine (SOP context) creates a flat array and fills it in with the
>> > attributes from geometry
>> > 2. VEX engine spawns several threads and runs one instance of VEX
>> procedure
>> > in each thread
>> > 3. Each instance of VEX procedure (single threaded) filters a value from
>> > point cloud and writes into it's chunk of flat array
>> > 4. VEX engine writes all the values back from flat array into a GDP using
>> > one thread
>> > Honestly I haven't tried multithreading with GDP but I see no problems
>> with
>> > it as long as you are not modifying the data. And normally VEX function
>> > itself doesn't modify a GDP (but VEX engine does).
>> > Well that's how I understand it and I can be wrong :)
>> > There are some bits about thread safety in HDK
>> >
>> http://www.sidefx.com/docs/hdk11.0/hdk_changes_11_0.html#HDKC11_0_ThreadSafety
>> > I probably have to do some tests too :)
>> >
>> > On Fri, Jan 21, 2011 at 5:13 PM, Szymon Kapeniak
>> > <szymon.kapeniak at gmail.com>wrote:
>> >
>> >> Hey Olex,
>> >> I'm writing custom vex, and I'm sure vex has some standard methods to
>> >> deal with this. I'm only interested in read access from several
>> >> threads which I don't have any control on, as they are spawned by VEX
>> >> SOP.
>> >>
>> >> I started to play with it by checking minimal scenarios and it seems
>> >> it's enough to declare one static GU_Detail instance and assign
>> >> pointers to it from a couple of threads to crash Houdini even without
>> >> loading any geometry.
>> >> Also by checking threads ids I realized, that mantra can run the same
>> >> code multi-threaded because vex in surface context call init()
>> >> (standard procedure) per thread, unlike in sop context, where it seems
>> >> init() is called only once like in fact it's described in docs.
>> >>
>> >> Surly there are thread-save vex functions (reading gdp) which operate
>> >> in sop context, like pcopen().  I'm guessing the trick is not to use
>> >> pure gdp, but some  thread-safe memory schema. Lack of documentation
>> >> doesn't play well with my ignorance...
>> >>
>> >> skk.
>> >>
>> >> btw  I'm referring to this talk:
>> >>
>> >>
>> http://forums.odforce.net/index.php?/topic/12434-how-to-replicate-this-ice-network-in-vops/page__st__12
>> >>
>> >> 2011/1/21 Olex P <hoknamahn at gmail.com>:
>> >> > Hey Szymon,
>> >> >
>> >> > You can read in multiple threads but multithread writing as far as I
>> know
>> >> is
>> >> > limited to volume primitives (each thread can write into it's own
>> tile)
>> >> as
>> >> > far as I know. You could create own GU/GEO_Detail object for each
>> thread,
>> >> > write into it and then merge all the pieces of puzzle into one object
>> >> (merge
>> >> > will single thread).
>> >> > But from your words I'm not sure what kind of parallelism you are
>> looking
>> >> > for...
>> >> > Let's say sampling from volume in VEX. 1. You are running the same
>> single
>> >> > thread code on different chunks of data using multiple threads. 2. But
>> >> this
>> >> > also could be done in a different way: vex function internally spawns
>> >> > several threads which modify the data somehow. You probably talking
>> about
>> >> 2
>> >> > (as you mentioned multithread access to GU_Detail). I'm not sure but
>> >> maybe
>> >> > it makes sense to use 1 (i.e. CVEX). Sure it will work only for data
>> >> > parallel algorithms.
>> >> >
>> >> > On Thu, Jan 20, 2011 at 10:20 PM, Szymon Kapeniak <
>> >> szymon.kapeniak at gmail.com
>> >> >> wrote:
>> >> >
>> >> >> ...sorry for spamming, but I've just realized that the same dso works
>> >> >> in mantra, but not in (threading) sop context. Kind of strange.
>> >> >>
>> >> >> 2011/1/20 Szymon Kapeniak <szymon.kapeniak at gmail.com>:
>> >> >> > Thanks for a response, I didn't know I have to deal with it
>> manually.
>> >> >> >
>> >> >> > Second problem is threading in vex sop. After playing with my
>> function
>> >> >> > a little, I'm getting a feeling my approach was too naive, so to
>> >> >> > speak.
>> >> >> >
>> >> >> > Is multi-threaded access to a static GU_Detail * possible at all,
>> or
>> >> >> > do I have to use some caches like UT_ThreadSafeCache or anything
>> >> >> > similar? What is a general way of doing this in HDK?
>> >> >> >
>> >> >> > thanks,
>> >> >> > skk.
>> >> >> >
>> >> >> >
>> >> >> > 2011/1/19 Olex P <hoknamahn at gmail.com>:
>> >> >> >> It kind of does writing an op-referenced texture into an IFD :)
>> >> >> >>
>> >> >> >> On Wed, Jan 19, 2011 at 1:48 PM, Mark Elendt <mark at sidefx.com>
>> >> wrote:
>> >> >> >>
>> >> >> >>> For mantra, even Houdini's VEX doesn't support the op: syntax :)
>> >> >> >>>
>> >> >> >>> For VEX functions in OPs, you should be able to access the SOP by
>> >> >> >>> grabbing the OP director and finding the SOP.
>> >> >> >>>
>> >> >> >>> On Wednesday Jan 19 at 13:39, Szymon Kapeniak wrote:
>> >> >> >>> > thanks Mark,
>> >> >> >>> > but how about custom vex dso? My func() opens file from disk,
>> but
>> >> >> >>> > doesn't handle op: syntax. Is it doable in custom code?
>> >> >> >>> >
>> >> >> >>> > thanks,
>> >> >> >>> > skk.
>> >> >> >>> >
>> >> >> >>> > 2011/1/19 Mark Elendt <mark at sidefx.com>:
>> >> >> >>> > > Unfortunately, at the current time op: is handled as a
>> special
>> >> >> case,
>> >> >> >>> > > so it's not part of the generalized loader.
>> >> >> >>> > >
>> >> >> >>> > > You'll have to reference the geometry directly (i.e. cook the
>> >> SOP).
>> >> >> >>> > > If you need your own copy, you'll have to merge it.
>> >> >> >>> > >
>> >> >> >>> > > On the other hand, opdef: is handled properly.
>> >> >> >>> > >
>> >> >> >>> > > On Wednesday Jan 19 at 12:11, Olex P wrote:
>> >> >> >>> > >> Maybe you right Szymon. Most of the usage of op: syntax for
>> me
>> >> was
>> >> >> >>> sampling
>> >> >> >>> > >> from point clouds/volumes/textures and I have never tries to
>> >> load
>> >> >> >>> geometry
>> >> >> >>> > >> using op: syntax. File SOP doesn't work with it and I think
>> >> it's
>> >> >> >>> because of
>> >> >> >>> > >> the fact that load() expects a proper file format. Would be
>> >> great
>> >> >> to
>> >> >> >>> hear
>> >> >> >>> > >> about this from SESI.
>> >> >> >>> > >>
>> >> >> >>> > >> On Wed, Jan 19, 2011 at 11:58 AM, Szymon Kapeniak <
>> >> >> >>> szymon.kapeniak at gmail.com
>> >> >> >>> > >> > wrote:
>> >> >> >>> > >>
>> >> >> >>> > >> > hi Olex,
>> >> >> >>> > >> > thanks for fast response. Well, I must be doing something
>> >> >> stupid,
>> >> >> >>> > >> > because it doesn't.
>> >> >> >>> > >> >
>> >> >> >>> > >> > GU_Detail gdp;
>> >> >> >>> > >> > const char *file   = (const char *) argv[1];
>> >> >> >>> > >> > if (gdp.load(file, 0) < 0) return ;
>> >> >> >>> > >> > ...
>> >> >> >>> > >> >
>> >> >> >>> > >> > while works fine with disk paths, doesn't work with
>> op:/'ed
>> >> one.
>> >> >> >>> > >> >
>> >> >> >>> > >> >
>> >> >> >>> > >> > 2011/1/19 Olex P <hoknamahn at gmail.com>:
>> >> >> >>> > >> > > Hi Szymon,
>> >> >> >>> > >> > >
>> >> >> >>> > >> > > I don't think that you need any magic. op: syntax has to
>> be
>> >> >> >>> handled
>> >> >> >>> > >> > > automagically when you using
>> >> >> >>> > >> > >
>> >> >> >>> > >> > > [code]virtual int          load(const char *, const
>> >> UT_Options
>> >> >> >>> > >> > > *options);[/code]
>> >> >> >>> > >> > >
>> >> >> >>> > >> > > method.
>> >> >> >>> > >> > >
>> >> >> >>> > >> > > On Wed, Jan 19, 2011 at 11:41 AM, Szymon Kapeniak <
>> >> >> >>> > >> > szymon.kapeniak at gmail.com
>> >> >> >>> > >> > >> wrote:
>> >> >> >>> > >> > >
>> >> >> >>> > >> > >> greetings!
>> >> >> >>> > >> > >>
>> >> >> >>> > >> > >> what is a magic to let  my GU_Detail class to ::load()
>> >> both
>> >> >> files
>> >> >> >>> from
>> >> >> >>> > >> > >> disk as operator's streams with op:/path syntax? I can
>> see
>> >> >> load()
>> >> >> >>> > >> > >> with an UT_IStream as argument, but I have no idea how
>> to
>> >> use
>> >> >> it.
>> >> >> >>> Does
>> >> >> >>> > >> > >> any of UT_Stream versions handle both files and steams
>> or
>> >> the
>> >> >> >>> logic is
>> >> >> >>> > >> > >> on my side?
>> >> >> >>> > >> > >>
>> >> >> >>> > >> > >> thanks,
>> >> >> >>> > >> > >> skk.
>> >> >> >>> > >> > >> _______________________________________________
>> >> >> >>> > >> > >> Sidefx-houdini-list mailing list
>> >> >> >>> > >> > >> Sidefx-houdini-list at sidefx.com
>> >> >> >>> > >> > >>
>> >> >> >>>
>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >> >>> > >> > >>
>> >> >> >>> > >> > > _______________________________________________
>> >> >> >>> > >> > > Sidefx-houdini-list mailing list
>> >> >> >>> > >> > > Sidefx-houdini-list at sidefx.com
>> >> >> >>> > >> > >
>> >> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >> >>> > >> > >
>> >> >> >>> > >> > _______________________________________________
>> >> >> >>> > >> > Sidefx-houdini-list mailing list
>> >> >> >>> > >> > Sidefx-houdini-list at sidefx.com
>> >> >> >>> > >> >
>> >> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >> >>> > >> >
>> >> >> >>> > >> _______________________________________________
>> >> >> >>> > >> Sidefx-houdini-list mailing list
>> >> >> >>> > >> Sidefx-houdini-list at sidefx.com
>> >> >> >>> > >>
>> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >> >>> > >
>> >> >> >>> > > --
>> >> >> >>> > > Mark Elendt
>> >> >> >>> > > _______________________________________________
>> >> >> >>> > > Sidefx-houdini-list mailing list
>> >> >> >>> > > Sidefx-houdini-list at sidefx.com
>> >> >> >>> > >
>> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >> >>> > >
>> >> >> >>> > _______________________________________________
>> >> >> >>> > Sidefx-houdini-list mailing list
>> >> >> >>> > Sidefx-houdini-list at sidefx.com
>> >> >> >>> >
>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >> >>>
>> >> >> >>> --
>> >> >> >>> Mark Elendt
>> >> >> >>> _______________________________________________
>> >> >> >>> Sidefx-houdini-list mailing list
>> >> >> >>> Sidefx-houdini-list at sidefx.com
>> >> >> >>>
>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >> >>>
>> >> >> >> _______________________________________________
>> >> >> >> Sidefx-houdini-list mailing list
>> >> >> >> Sidefx-houdini-list at sidefx.com
>> >> >> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >> >>
>> >> >> >
>> >> >> _______________________________________________
>> >> >> Sidefx-houdini-list mailing list
>> >> >> Sidefx-houdini-list at sidefx.com
>> >> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >>
>> >> > _______________________________________________
>> >> > Sidefx-houdini-list mailing list
>> >> > Sidefx-houdini-list at sidefx.com
>> >> > https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >> >
>> >> _______________________________________________
>> >> Sidefx-houdini-list mailing list
>> >> Sidefx-houdini-list at sidefx.com
>> >> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >>
>> > _______________________________________________
>> > Sidefx-houdini-list mailing list
>> > Sidefx-houdini-list at sidefx.com
>> > https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>> >
>> _______________________________________________
>> Sidefx-houdini-list mailing list
>> Sidefx-houdini-list at sidefx.com
>> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>>
> _______________________________________________
> Sidefx-houdini-list mailing list
> Sidefx-houdini-list at sidefx.com
> https://lists.sidefx.com:443/mailman/listinfo/sidefx-houdini-list
>



More information about the Sidefx-houdini-list mailing list