| 03:07.52 | Twingy | s/humans/carbon based life forms? |
| 06:19.31 | *** join/#brlcad Twingy (n=justin@74.92.144.217) [NETSPLIT VICTIM] | |
| 14:02.43 | ``Erik | heh |
| 16:09.35 | *** join/#brlcad SWPadnos (n=Me@dsl245.esjtvtli.sover.net) | |
| 17:27.16 | Twingy | Robotic Deer to Fight Illegal Hunting |
| 17:27.24 | Twingy | ... |
| 17:32.56 | ``Erik | does it shoot back? |
| 17:33.19 | ``Erik | cuz that'd be kinda gnarly |
| 17:33.31 | ``Erik | heh, milton would come into work all bandaged up :D |
| 17:34.02 | Maloeran | Also disturbing : http://news.bbc.co.uk/2/hi/south_asia/6209613.stm |
| 17:35.22 | ``Erik | heh |
| 17:37.50 | Maloeran | I'm trying to understand how the thing can be that slow, getting there slowly |
| 17:37.53 | ``Erik | hahahahaha |
| 17:38.12 | ``Erik | starting to understand the stupid at the lab? :D |
| 17:38.31 | Maloeran | I guess you could put it that way :) |
| 17:38.35 | ``Erik | call stacks so deep, they might as well be bottomless, repacking of data for shits and giggles, ... |
| 17:38.50 | Maloeran | And constant memory allocation/freeing... to shoot a single ray! |
| 17:39.09 | Maloeran | Memory allocation and free()ing just for _one_ ray! |
| 17:39.22 | ``Erik | uh huh |
| 17:39.29 | ``Erik | memory pools? what are those? |
| 17:39.43 | ``Erik | oh, but hell project is ALL about resource pools, the opposite extreme |
| 17:39.50 | Maloeran | Ahah |
| 17:40.06 | ``Erik | want to send data? don't keep your own socket... get one from the pool! oh, and when you've sent your FUCKING FOUR BYTES, put it back in the pool! |
| 17:40.44 | Maloeran | Ah, cool |
| 17:41.24 | ``Erik | I'll be, uh, very not here for the next week |
| 17:41.25 | ``Erik | :) |
| 17:41.42 | Maloeran | Girlfriend? |
| 17:41.47 | ``Erik | mmhmm |
| 17:42.18 | brlcad | erm |
| 17:42.25 | brlcad | librt doesn't repack data |
| 17:42.44 | ``Erik | heh |
| 17:42.45 | ``Erik | s2 sure does |
| 17:42.46 | brlcad | maybe some specific app foo does |
| 17:42.53 | brlcad | s2 does |
| 17:43.26 | brlcad | comparing what s2 does to an optimized ray-tracer isn't even apples to oranges |
| 17:43.38 | ``Erik | if it were built as a 'shader' and ran inside of a brlcad app, it'd be pretty damn fast, evne on librt |
| 17:43.58 | ``Erik | rf has to feed s2 :( that's how we got wendy to sign off on it, iirc |
| 17:44.10 | Maloeran | What is s2? |
| 17:44.29 | ``Erik | the old important app, the one the hell project is supposed to replace |
| 17:44.37 | brlcad | sure it'll eventually feed it, so yeah most of the performance benefit is going to get slaughtered |
| 17:44.48 | brlcad | but it'll still be an improvement for many cases |
| 17:45.04 | Maloeran | Any hope of just fixing/rewriting that s2 thing? |
| 17:45.37 | ``Erik | not really... twingy rewrote the build system to be MUCH better, and they didn't understand it, so they didn't accept it |
| 17:45.42 | brlcad | Maloeran: it's really not that simple -- there are a couple decades of feature-packing in there that has to be replicated |
| 17:46.00 | brlcad | any rewrite, even the one being attempted, can't replace that functionality in any time reasonbly soon |
| 17:46.15 | brlcad | akin to rewriting any major code base |
| 17:46.24 | ``Erik | they also fought plugging adrt into it tooth and nail... like, they compiled adrt with floats, librt with doubles, and refused to acceptt adrt because the output numbers weren't bit identical... durrrr |
| 17:46.26 | brlcad | it's rather naive to think it just takes a simple rewrite |
| 17:47.09 | ``Erik | dude, I think 90% of that project is unnecessary fluff, crap left over from dirty hacks that just dont' get used... based on mrb output and talking to analysts, it might really be a 'simple rewrite' :( |
| 17:47.10 | Maloeran | I see, I just have no idea what s2 or the hell project exist for |
| 17:48.10 | brlcad | ``Erik: in all fairness, though, all of the differences WEREN'T just float/double fuzz -- the behavior hadn't been verified and a couple real issues were found (i.e. honest differences in behavior) |
| 17:48.25 | ``Erik | ok, the float vs double one really REALLY got under my skin |
| 17:48.29 | brlcad | that kind of stuff just can't happen for that code |
| 17:48.39 | Maloeran | You will find serious differences in behavior and results with rayforce too |
| 17:48.41 | ``Erik | because she (km) said that the problem 'disappeared' when she compiled adrt with double |
| 17:49.01 | brlcad | that was one case, there were other issues |
| 17:49.07 | ``Erik | but she HAD to to hunt down the cause for the difference! it was a whole .001 mm! |
| 17:49.15 | ``Erik | <-- strusfrated with some of their bs :) |
| 17:49.18 | brlcad | some were issues just being masked that were different behaviors |
| 17:49.56 | brlcad | Maloeran: to be expected to various degrees -- but they have to be explainable other than hand waving |
| 17:50.05 | ``Erik | obviously, the values wont' be the exact same, they cope with things differently... I'm just irked because in the grand scheme of things, it doesn't matter... |
| 17:50.14 | brlcad | at least if the differences are significant |
| 17:50.36 | ``Erik | I mean, it's all grotesque punting, the just cling to regression numbers because they don't understand the algorithms or reasoning |
| 17:50.41 | ``Erik | grar. |
| 17:50.50 | Maloeran | brlcad: Right, but a part of that is based on the ray cutting into segments, and the different intersection code with triangles ; you lose a bit of precision on the way |
| 17:51.03 | brlcad | he was done with adrt, at that time it really was a lot of hand waving going on, and they were understandably not satisfied imho |
| 17:51.08 | Maloeran | The differences should just be properly handled for them not to matter |
| 17:51.13 | ``Erik | and ch seems like the only one with his feet under him... km a tiny bit, but *sigh*... and they gave them that poor java coder hehehe |
| 17:51.48 | ``Erik | <-- not impressed with that crew :/ |
| 17:51.59 | ``Erik | also, I'm a strusfrated bitter person for anything up that alley :) |
| 17:52.22 | brlcad | there's plenty to complain about with respect to s2, but their demands of having to validate any change are a good one in general -- even if they do or don't understand what's going on |
| 17:52.47 | ``Erik | their stance of not ACCEPTING any change, whether or not it's valid or makes sense, isn't excusable |
| 17:53.07 | brlcad | Maloeran: most of the "significant" differences aren't just precision in itself, though perhaps provoked by precision |
| 17:53.12 | brlcad | edge conditions |
| 17:53.18 | brlcad | do you go left or right |
| 17:53.34 | Maloeran | Indeed, brlcad |
| 17:53.35 | brlcad | you might arbitrarily (albeit consistently "go right", per se) |
| 17:53.43 | ``Erik | like the air overlap swapping from the change jra made in the scenegraph |
| 17:54.16 | ``Erik | they refused a code improvement due to a 'broken' model... :) irritating |
| 17:54.26 | brlcad | ``Erik: true but then they never were shown a validation that could be accepted in that instance |
| 17:54.46 | ``Erik | the breakage in their test was pointed out |
| 17:55.11 | ``Erik | <-- likes regression testing and consistancy in results... but consistantly wrong results are still wrong :( |
| 17:55.32 | brlcad | dude, you're pulling out one instance.. there were like a half dozen different issues |
| 17:55.57 | ``Erik | *shrug* what can I say? I'm bitter and irate and on a tyrade :) |
| 17:56.16 | brlcad | a couple were breakage in their code, or at least bad assumptions, others were fuzz, a couple others were logic differences |
| 17:56.16 | ``Erik | or tirade |
| 17:56.18 | ``Erik | your choihce |
| 17:56.38 | ``Erik | <-- thinks they're too anal and lb is too lenient o.O |
| 17:57.06 | brlcad | the latter were the ones that mattered that would have otherwise gone unnoticed, even if they did amount to like a couple lines of code |
| 17:57.50 | ``Erik | (also, I have a low opinion of the theory and information that program is built on... stinky of hackiness to me) |
| 17:58.13 | brlcad | Maloeran: the overall "concept" isn't hard at all, especially from a code perspective -- it's an approximated physics simulation |
| 17:58.46 | brlcad | the code is also probably the least significant aspect of the purpose for it's existance |
| 17:59.00 | ``Erik | mal: http://www.bahdayton.com/SURVIAC/archive/surviac_bulletin/bulletin_9501/bull_9501.html#MUVES |
| 17:59.17 | Maloeran | For vulnerability analysis? |
| 17:59.23 | Maloeran | Ah, right |
| 17:59.28 | brlcad | for a variety of analysis purposes |
| 18:00.20 | Maloeran | Quote : "MUVES employs the latest software technologies both in design and implementation to optimize useability, [...]" Mmhmm :) |
| 18:00.31 | ``Erik | it's old code |
| 18:01.10 | brlcad | http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA245322 |
| 18:02.08 | brlcad | Maloeran: quoted from probably before you were born |
| 18:02.11 | ``Erik | huh, with a few authors I recognize, go figure |
| 18:02.14 | brlcad | when it was very much probably true |
| 18:02.46 | brlcad | it's a software implementation of a simulation that was actually carried out by hand for decades before it |
| 18:03.08 | brlcad | where it fails isn't really in terms of performance |
| 18:03.22 | brlcad | or utility -- it certainly does the job it was designed to do |
| 18:03.26 | ``Erik | a step int eh right direction... but the stepping stopped, and they got stuck :( |
| 18:03.46 | ``Erik | imho |
| 18:03.53 | Maloeran | Well, it's C and Unix, they got that right |
| 18:04.02 | brlcad | it fails in terms of usability and production polish/use -- the 80% of code in a project that has nothing to do with performance |
| 18:04.24 | ``Erik | ok, I'm done ranting and being a little bitch :) imma go slap my shoes on and head to bwi... catch ya'll later :) |
| 18:04.44 | Maloeran | Have fun! |
| 18:04.48 | brlcad | cheers |
| 18:05.49 | brlcad | much in the same way mged fails to provide an useful interface to new users -- yet most of the "geometry kernel" logic that is required by any solid modeling system is there underneath |
| 18:06.19 | brlcad | s2 users scream for a new interface, the usability is painful |
| 18:07.21 | Maloeran | I see, interesting. So performance isn't quite the main issue at this point |
| 18:07.52 | brlcad | it's one of many issues, but only the biggest from a production standpoint -- not from the "big picture" |
| 18:09.24 | brlcad | it's like saying I need to get from here to montreal, and I have my bike .. and even though I have a really good bike, it can still only go so fast |
| 18:09.36 | brlcad | so you're working on a fancy jet engine to propel the bike faster |
| 18:10.15 | brlcad | when in the big picture.. there are other problems (like others that need to get to montreal that don't know how to ride a bicycle, or are disabled, etc) |
| 18:10.55 | Maloeran | Good metaphor :). There's just the problem of focusing so much on the raytracer performance, if 95% of the processing time is spent in the "shaders" |
| 18:11.33 | brlcad | most problems can be simplified down to bicycles and cars :) |
| 18:12.42 | brlcad | 95% isn't representative |
| 18:13.45 | brlcad | it's very model and analysis dependent -- some are really highly ray-trace bound, others are stuck in shader-land |
| 18:15.08 | Maloeran | I guess so. Simple flat rendering "shaders" consume 40% of the processing time here, the actual ray-tracing is... cheap |
| 18:15.34 | brlcad | on average, I think it's pretty conservative estimate to say more than 50% .. more close to 80-90 probably on average |
| 18:16.25 | brlcad | yeah, that is one thing that rayforce definitely has over librt -- librt hooks into liboptical for shading, and that isn't nearly as optimized |
| 18:17.43 | brlcad | it was written to be run-time flexible, pluggable instead, which greatly impacts the design and performance |
| 18:18.32 | Maloeran | Not that much if rays are processed into large batches, but... yes, flexibility and processing of one ray at a time is just awful |
| 18:19.14 | brlcad | it would be interesting to see what rayforce's single-ray cost is post-prep |
| 18:19.31 | Maloeran | Horrible. It's not designed for that |
| 18:19.32 | brlcad | since that will be more in line with what would be expected from s2 |
| 18:19.48 | brlcad | horrible to you.. but still perhaps a vast boost overal |
| 18:20.03 | brlcad | perhaps not, just would be interesting |
| 18:20.06 | Maloeran | You have to set up a buffer, set the buffer environment variables to it builds the proper pipeline optimized for the settings, queue the job, have a thread acquire and run it... for a single ray? |
| 18:20.33 | Maloeran | It could be made faster, but it isn't really designed for that |
| 18:20.43 | brlcad | post-prep -- some of that could be construed as part of the prep |
| 18:22.04 | Maloeran | Really, it's like a C++ graphical library making a overly complex class "pixel", you don't want to process one ray at a time :) |
| 18:22.51 | brlcad | sounds like you're describing X ;) |
| 18:23.29 | Maloeran | Eheh, close enough yes |
| 18:25.01 | brlcad | it might be too much overhead, but it also might still be quite an improvement .. and is in line with how an analysis currently progresses |
| 18:25.41 | brlcad | i mean there are other considerations that become possible -- shooting in packets, faster optical/pictures, finer resolution at no additional expense |
| 18:26.14 | Maloeran | Do you really need to trace individual rays? |
| 18:26.31 | brlcad | the analysis codes, though, generally repeatedly ask the question "what objects are along this path" |
| 18:27.28 | brlcad | millions of times.. and sometimes the paths coincide, sometimes they do not at all -- more like solving the ray-tracing question for secondary rays only |
| 18:27.54 | Maloeran | Batches of secondary, incoherent rays are fine too |
| 18:28.16 | Maloeran | If single rays really are required, I guess I should add an "immediate mode" or so, without job/thread management, where the calling thread blocks and receives raw results |
| 18:30.34 | brlcad | this same issue came up with adrt, and of course it's even more significant for rayforce .. but what really makes sense as a first step is to drop it in as the implementation behind librt's ray api |
| 18:31.08 | brlcad | which means basically two function calls -- a scene prep routine, and a ray shooter routine |
| 18:31.18 | brlcad | the latter being called over and over |
| 18:31.58 | Maloeran | Would such calls be made from numerous threads? |
| 18:32.04 | brlcad | adrt is going to be tested in this fashion now that an initial integration is up and running |
| 18:32.45 | brlcad | yes, numerous threads and/or procs depending on the platform |
| 18:33.53 | Maloeran | I don't like the idea design-wise, but I guess it's the easiest first step |
| 18:34.26 | brlcad | for librt, those two actual calls are rt_prep_parallel() and rt_shootray() for most applications, including for s2 |
| 18:34.59 | Maloeran | Yes I saw, the rayforce API is a bit more complex |
| 18:35.40 | brlcad | the idea would be to detect in rt_prep_parallel if the model being prep'd is triangles, to prep using adrt and/or later using rayforce.. then during rt_shootray() calls, to invoke the other engine instead of librt for those rays |
| 18:35.47 | brlcad | at least as an initial pass |
| 18:36.53 | brlcad | that way, there are dozens of projects and applications that could get any performance benefits "for free" without requiring a rewrite (or even a recompile in some instances) |
| 18:37.01 | Maloeran | Does rt_shootray() block or it queues a callback? |
| 18:37.07 | brlcad | heh |
| 18:37.25 | brlcad | blocks |
| 18:37.42 | Maloeran | So there's no way to just fill up buffers to process rays properly |
| 18:38.05 | brlcad | thought about adding a non-blocking callback version, but that gets back to the same problem of rewriting all the user codes |
| 18:38.20 | brlcad | and if/when they're willing to go that far.. there may be better options |
| 18:38.54 | Maloeran | Using the rayforce native interface? :) It's generic, flexible, meant to be scalable |
| 18:39.31 | Maloeran | It was written with the possibility in mind that any other raytracer could easily plug itself behind the API |
| 18:39.34 | brlcad | apparently not flexible enough to have a single prep and single shotline inquiry interface? :) |
| 18:39.57 | Maloeran | Correct, that's just gross and inefficient :) |
| 18:40.18 | brlcad | and an excuse -- it's about as simple an interface as possible |
| 18:40.20 | Maloeran | It could be made, it just removes much of the acceleration potential that could go on behind the API |
| 18:40.44 | Maloeran | May it be efficient packet processing, SIMD processing, a hardware solution |
| 18:41.22 | Maloeran | The rayforce API can accomodate about any kind of underlying raytracing engine, hardware of FPGA included |
| 18:41.26 | Maloeran | or* FPGA |
| 18:41.30 | *** join/#brlcad docelic (n=docelic@212.91.114.94) | |
| 18:42.31 | brlcad | yeah, and those are certainly taking off like hotcakes.. :) |
| 18:42.54 | Maloeran | One day, perhaps, and that implies that the API will still be usable at that point :) |
| 18:42.58 | brlcad | that's great, but for the more immediate.. it really will probably boil down to just two or three levels of integration |
| 18:43.37 | Maloeran | I realize that but I must say I prefer properly designed APIs, seeing farther than the immediate needs |
| 18:43.53 | brlcad | first mimicking the api and seeing what boosts can be gained, then looking at what user-app changes would be required to get the biggest boost whether it be queuing rays or callbacks or what have you |
| 18:44.25 | brlcad | "immediate" in this context is the next two years or so |
| 18:45.24 | Maloeran | There's no way to do any decent SSE with single rays by the way, performance would be cut by a factor of 2-10 by design |
| 18:46.02 | brlcad | "properly designed APIs" can all mean drasticly different things -- librt's API is entirely proper given the purpose |
| 18:46.40 | Maloeran | It definitely wasn't designed to adopt different raytracing engines, interfaces or solutions |
| 18:46.41 | brlcad | depends on the user, the hardware, the purpose, etc |
| 18:47.05 | Maloeran | There's no way to use raytracing hardware with a blocking rt_shootray() for example |
| 18:47.38 | brlcad | sure, that was never a purpose of the library -- it was specifically written to be hardware agnostic |
| 18:48.13 | brlcad | hardware has changed a half dozen times since it was started... had the best "fad" of that day been used, librt would have been obsolete a decade ago |
| 18:48.56 | brlcad | and by obsolete, i mean written in a manner where it *could not* function on current hardware |
| 18:49.34 | brlcad | as it is, it still runs on just about anything, whether it's got acceleration hardware or not, special computation units etc |
| 18:49.37 | Maloeran | I don't mean it should be tailored to any specific hardware, I mean it should be both flexible and efficient, not minimalistic |
| 18:49.51 | Maloeran | OpenGL was designed in a flexible and decent manner, and it remained flexible and efficient for decades |
| 18:50.20 | brlcad | heh, you keep forgetting/ignoring the purpose though? |
| 18:50.39 | brlcad | for opengl's purpose, it was flexible and "decent" as you call it, but entirely useless for other domains |
| 18:50.49 | Maloeran | Well, the librt minimalistic interface causes problems today |
| 18:52.28 | brlcad | technically only the rt_shootray() interface has that limitation |
| 18:52.37 | brlcad | librt doesn't boil down to just those two calls |
| 18:53.01 | brlcad | they are just the most prevalently used, *because* they are so simple to use |
| 18:53.13 | brlcad | there are routines for shooting bundles of rays in the library |
| 18:53.21 | brlcad | nobody uses them |
| 18:53.22 | Maloeran | The problem isn't with librt really, but with the interface ; the interface that all the software now uses |
| 18:53.27 | Maloeran | Oh :) |
| 18:56.19 | Maloeran | I guess there wasn't much to gain with bundles of rays with librt |
| 18:56.57 | brlcad | it gets back to what I started off by saying -- you can't just suddenly change an API out from under dozens/hundreds of projects when you have faithful users without risking loosing your users -- even if the benefits "seem" sufficient |
| 18:57.12 | brlcad | actually, there was a lot to be gained -- that's why it was implemented |
| 18:57.39 | brlcad | getting performance boosts by bundling data accesses together is by far nothing new -- heck cray had a monopoly on that two decades ago |
| 18:59.16 | Maloeran | And people haven't updated the software to use the more efficient calls, I see |
| 19:00.19 | brlcad | not in the least |
| 19:01.00 | brlcad | actually writing and even using production-use software is usually not the biggest business expense |
| 19:01.40 | brlcad | managing that software, training users, documenting everything, managing the workflow process -- that all takes considerably more time |
| 19:03.13 | brlcad | so while there's an interface that is faster -- the investment is already complete, someone will have to take several weeks to review the new interface, integrate/code to it, validate it, test it, document the change, share that knowledge with others on the team(s) |
| 19:04.36 | Maloeran | So a programmer can't just switch the code to the better interface on an evening? :) Such inefficiency |
| 19:04.37 | brlcad | so the motivation to make the change, one that has NOT been validated in particular, is a risk and the benefit generally has to be distinctly clear and outweigh the other costs involved in making the change |
| 19:05.00 | brlcad | of course they can't |
| 19:05.27 | brlcad | well they can to the code, but not to the application that is in production use |
| 19:05.53 | brlcad | someone(tm) has to make sure it actually works .. and what happens when there's a problem, when someone does make a mistake? |
| 19:06.12 | brlcad | and mistakes DO happen.. they happen a damn lot, especially with programmers |
| 19:07.06 | Maloeran | The coding process sounds awfully slow and restricted in such a controlled environment |
| 19:08.41 | brlcad | in some instances it is, some it isn't.. depends on the impact and purpose |
| 19:09.08 | brlcad | shotline analysis impacts lives, people do live and die by the results |
| 19:10.15 | brlcad | that's why just saying "oh yeah, it's faster" is a useless statement when there is a difference in a given case that can't be exactly explained |
| 19:10.17 | Maloeran | Right, that isn't some software you can send patches to users later on |
| 19:10.58 | brlcad | the difference between a bug and floating point fuzz, for example, could be just one character .. and both giving the same result 99.99% of the time |
| 19:11.58 | brlcad | due diligence requires getting down to the actual science, explaining the difference and verifying that it's not a bug for example or not a difference to worry about is paramount |
| 19:13.01 | brlcad | most programmers are incredibly lazy, hand-waving excuses are usually very "bad" even if the "gut feeling" is sane |
| 19:14.39 | Maloeran | Well, I can already say that you'll find major differences between rayforce and adrt results. Just between two different graph builds, results differ |
| 19:15.07 | brlcad | yep, to be expected |
| 19:16.06 | brlcad | whether the results are significant depends on a lot of factors, and will be part of the future integration/validation part |
| 19:23.12 | *** join/#brlcad docelic (n=docelic@212.91.112.103) | |
| 20:24.05 | *** join/#brlcad docelic (n=docelic@212.91.112.103) | |
| 20:42.59 | Maloeran | Strange, I'm not finding any header for GCC's __builtin_bswap32 and __builtin_bswap64, to avoid implicit declarations |