Stream: brlcad

Topic: in-database support for vol objects

Sumagna Das (Sep 14 2020 at 19:36):

hey @Sean i have done the following things for in-DB support for vol

Sumagna Das (Sep 14 2020 at 19:37):

(deleted)

Sumagna Das (Sep 14 2020 at 19:37):

should i submit it as a patch?

Sumagna Das (Sep 14 2020 at 19:37):

or do something more and then submit it as a patch?

Sumagna Das (Sep 14 2020 at 19:38):

things i have done till now for in-DB support -
tasks.txt

Sean (Sep 14 2020 at 19:39):

holy moly!

Sumagna Das (Sep 14 2020 at 19:39):

what?

Sean (Sep 14 2020 at 19:40):

just that you've been making progress, that's fantastic!

Sumagna Das (Sep 14 2020 at 19:40):

did i do something wrong?

Sean (Sep 14 2020 at 19:40):

and looks like you're on the right track

Sean (Sep 14 2020 at 19:40):

i wouldn't post it to sf.net as a patch until it can be committed

Sumagna Das (Sep 14 2020 at 19:41):

ok

Sean (Sep 14 2020 at 19:41):

but you can post it here and can see if there's any feedback or issues

Sumagna Das (Sep 14 2020 at 19:41):

i am right now building it with target all

Sean (Sep 14 2020 at 19:42):

since you implemented the prompt for the filename, that will affect the ability to merge it (e.g., if it prompts now and then we post a release, that'd be a problem of course since people would expect if its prompting it must be doing that thing

Sean (Sep 14 2020 at 19:42):

but yeah, this is fantastic

Sumagna Das (Sep 14 2020 at 19:43):

thats why i was asking if i should post it or not right now

Sean (Sep 14 2020 at 19:43):

yeah, just share it here if you like

Sumagna Das (Sep 14 2020 at 19:43):

i saw the release thread

Sean (Sep 14 2020 at 19:43):

or if you want to extract the portions that can be committed, you could submit those

Sumagna Das (Sep 14 2020 at 19:44):

ok. let the build finish. then i will post it here

Sean (Sep 14 2020 at 19:44):

i.e., not the typein.c changes

Sumagna Das (Sep 14 2020 at 19:44):

ok

Sumagna Das (Sep 14 2020 at 19:45):

(deleted)

Sumagna Das (Sep 14 2020 at 19:59):

patch.patch

Sumagna Das (Sep 14 2020 at 20:00):

this include most of the changes i did for in-db support

Sumagna Das (Sep 14 2020 at 20:00):

except for the typein.c changes

Sumagna Das (Sep 14 2020 at 20:01):

this was all i could do in tonight's time

Sean (Sep 14 2020 at 20:01):

/me looks

Sumagna Das (Sep 14 2020 at 20:02):

good night

Sean (Sep 14 2020 at 20:02):

okay, see you in your morning ;)

Sumagna Das (Sep 14 2020 at 20:02):

tell me if it needs more changes in the other files

Sean (Sep 14 2020 at 20:10):

I just looked through the patch file and it looks spot-on to me.

Sean (Sep 14 2020 at 20:11):

no changes necessary to the ones you made -- it of course depends what you do with those changes, but looks like you're following what ebm is doing.

Sean (Sep 14 2020 at 20:11):

notice that dsp is slightly different (and more simple in some ways) as i t uses vls strings and doesn't have as much type toggling

Sumagna Das (Sep 14 2020 at 20:28):

I am going from ebm because I have made one and saw how it works

Sumagna Das (Sep 14 2020 at 20:28):

So i want vol to work the same way

Sean (Sep 14 2020 at 20:28):

it's fine, the differences are subtle

Sean (Sep 14 2020 at 20:30):

both dsp and ebm essentially do the same thing. main difference is little things like ebm stores the object name in a name field that's a char[] whereas dsp stores it in a bu_vls string.

Sumagna Das (Sep 14 2020 at 20:31):

Ok

Sean (Sep 14 2020 at 20:31):

so you'll see differences accessing the object name

Sean (Sep 14 2020 at 20:32):

ebm can just use the name string, whereas dsp has to call bu_vls_cstr() or bu_vls_addr() to get the name string. of course, ebm has to have overflow/buffer checks though, and dsp does not -- dsp memory is semiautomatically managed.

Sean (Sep 14 2020 at 20:32):

when it comes to accessing the data, you'll want to follow what ebm is doing anyway

Sean (Sep 14 2020 at 20:53):

so it looks like this can be committed. i haven't tested it to make sure it compiles, but if you can test that, you could submit it to sf.net as your second patch!

Sumagna Das (Sep 15 2020 at 03:11):

it will not build actually

Sumagna Das (Sep 15 2020 at 03:12):

a change i made in include/rt/geom.h was changing the char file variable to char name.

Sumagna Das (Sep 15 2020 at 03:13):

because of which a lot of problems will crop up during build in typein.c

Sumagna Das (Sep 15 2020 at 03:13):

i will submit it later to sf.net after making some more necessary changes

Sumagna Das (Sep 15 2020 at 03:23):

so i tested the 'f' part of the question and it still works

Sumagna Das (Sep 15 2020 at 03:23):

only part that is left is the 'o' part

Sumagna Das (Sep 15 2020 at 07:08):

@Sean tasks.txt
done upto this. now the only thing that is left is build it and test it if it works

Sumagna Das (Sep 15 2020 at 07:25):

(deleted)

Sumagna Das (Sep 15 2020 at 07:25):

(deleted)

Sumagna Das (Sep 15 2020 at 07:26):

its building successfully but not working

Sumagna Das (Sep 15 2020 at 07:34):

now its printing the following stuff:

ERROR: Binary object 'dbobj' has invalid data (expected type 4, found 4).
       Expecting 160000 8-bit unsigned char (nuc) integer data values.
       Encountered 480000 unsigned 8-bit ints

Couldn't find the associated file/object dbobj
ERROR: Binary object 'dbobj' has invalid data (expected type 4, found 4).
       Expecting 160000 8-bit unsigned char (nuc) integer data values.
       Encountered 480000 unsigned 8-bit ints

Couldn't find the associated file/object dbobj
ERROR: Binary object 'dbobj' has invalid data (expected type 4, found 4).
       Expecting 160000 8-bit unsigned char (nuc) integer data values.
       Encountered 480000 unsigned 8-bit ints

Couldn't find the associated file/object dbobj

Sumagna Das (Sep 15 2020 at 07:35):

this is the patch patch.patch

Sumagna Das (Sep 15 2020 at 07:35):

Couldn't find the associated file/object dbobj

i added this line for debugging purposes

Sumagna Das (Sep 15 2020 at 10:53):

its now working perfectly

Sumagna Das (Sep 15 2020 at 10:53):

but it is not showing up on the graphic window

Sumagna Das (Sep 15 2020 at 13:00):

tasks.txt

Sumagna Das (Sep 15 2020 at 13:01):

this is the things i have done in there

Sumagna Das (Sep 15 2020 at 13:36):

and this is the patch after fixing the problems patch.patch

Sean (Sep 15 2020 at 14:07):

That's great. Did you post the patch of the bits that were working to sf.net without the typein user prompting?

Sumagna Das (Sep 15 2020 at 14:08):

actually i implemented it fully

Sumagna Das (Sep 15 2020 at 14:08):

except for the visual part which i am trying to fix

Sean (Sep 15 2020 at 14:08):

seriously? that's awesome!

Sumagna Das (Sep 15 2020 at 14:09):

file part is working properly

Sumagna Das (Sep 15 2020 at 14:09):

only object part's drawing has to be fixed

Sumagna Das (Sep 15 2020 at 14:09):

its also building properly

Sean (Sep 15 2020 at 14:34):

well building is the first and typically easiest step. lots of ways for something to compile and still be quite wrong. :)

Sean (Sep 15 2020 at 14:35):

so you got the binunif aspect sorted out with the right data types?

Sean (Sep 15 2020 at 14:36):

bw (i.e., 8-bit unsigned char) is the native format for vol slices and it will need to read either multiple bw or one bw with multiple layers (you probably fed the latter in your test?)

Sumagna Das (Sep 15 2020 at 16:48):

Yea

Sumagna Das (Sep 15 2020 at 16:48):

I fed one bw with multiple layers

Sumagna Das (Sep 15 2020 at 16:57):

Should I do something else instead of this?

Sean (Sep 15 2020 at 17:34):

No, I think that's fine @Sumagna Das

Sean (Sep 15 2020 at 17:34):

I mean it might be nice if the user didn't have to combine them all, that gcv did it for them, but the internal guts in src/librt/primitives/vol should just be one data object

Sean (Sep 15 2020 at 17:36):

we'll likely change / extend that data object later with a proper "image" object instead of a binunif, but that's future-talk

Sean (Sep 15 2020 at 17:36):

maybe your third set of commits ;)

Sean (Sep 15 2020 at 17:36):

going to submit the patch soon?

Sumagna Das (Sep 15 2020 at 18:12):

Yea

Sumagna Das (Sep 15 2020 at 18:12):

After fixing the drawing issue

Sumagna Das (Sep 15 2020 at 20:26):

static int
get_obj_data(struct rt_vol_internal *vip, const struct db_i *dbip)
{
    struct rt_binunif_internal *bip;
    int ret;
    int nbytes;

    BU_ALLOC(vip->bip, struct rt_db_internal);

    ret = rt_retrieve_binunif(vip->bip, dbip, vip->name);
    if (ret)
    return -1;

    if (RT_G_DEBUG & RT_DEBUG_HF) {
    bu_log("db_internal magic: 0x%08x  major: %d  minor: %d\n",
        vip->bip->idb_magic,
        vip->bip->idb_major_type,
        vip->bip->idb_minor_type);
    }

    bip = (struct rt_binunif_internal *)vip->bip->idb_ptr;

    if (RT_G_DEBUG & RT_DEBUG_HF)
    bu_log("binunif magic: 0x%08x  type: %d count:%zu data[0]:%u\n",
        bip->magic, bip->type, bip->count, bip->u.uint8[0]);

    if (bip->type != DB5_MINORTYPE_BINU_8BITINT_U
    || (size_t)bip->count != (size_t)(vip->xdim*vip->ydim*vip->zdim))
    {
    size_t i = 0;
    size_t size;
  // size_t ret = 0;
    struct bu_vls binudesc = BU_VLS_INIT_ZERO;
    rt_binunif_describe(&binudesc, vip->bip, 0, dbip->dbi_base2local);

    /* skip the first title line*/
    size = bu_vls_strlen(&binudesc);
    while (size > 0 && i < size && bu_vls_cstr(&binudesc)[0] != '\n') {
        bu_vls_nibble(&binudesc, 1);
    }
    if (bu_vls_cstr(&binudesc)[0] == '\n')
        bu_vls_nibble(&binudesc, 1);

    bu_log("ERROR: Binary object '%s' has invalid data (expected type %d, found %d).\n"
           "       Expecting %zu 8-bit unsigned char (nuc) integer data values.\n"
           "       Encountered %s\n",
           vip->name,
           DB5_MINORTYPE_BINU_8BITINT_U,
           bip->type,
           (size_t)(vip->xdim*vip->ydim*vip->zdim),
           bu_vls_cstr(&binudesc));
    return -2;
    }

    nbytes = (vip->xdim+VOL_XWIDEN*2)*(vip->ydim+VOL_YWIDEN*2)*(vip->zdim+VOL_ZWIDEN*2);

    if (!vip->buf) {
    size_t x, y, z;
    unsigned char* cp;

    /* Prevent a multi-processor race */
    bu_semaphore_acquire(RT_SEM_MODEL);
    if (vip->buf) {
        /* someone else beat us, nothing more to do */
        bu_semaphore_release(RT_SEM_MODEL);
        return 0;
    }
    vip->buf = (unsigned char *)bu_calloc(1, nbytes, "rt_vol_import4 bitmap");
    bu_semaphore_release(RT_SEM_MODEL);

    /* Because of in-memory padding, read each scanline separately */
    cp = (unsigned char *)bip->u.uint8;
  for (z = 0; z < vip->zdim; z++) {
       for (x = 0; x < vip->xdim; x++) {
          /* bit() addresses into buf */
           memcpy(vol_bit(vip, x, 0), cp, vip->ydim*vip->xdim);
       }
     cp += vip->xdim;
  }
    }
    return 0;
}

this is the function for taking binary object as a data source for vol

Sumagna Das (Sep 15 2020 at 20:26):

can you tell me whats wrong

Sumagna Das (Sep 15 2020 at 20:58):

After fixing the drawing issue

Sean (Sep 15 2020 at 21:40):

I don't see anything obvoius

Sean (Sep 15 2020 at 21:42):

oh, looks like the memcpy is wrong in the middle of that loop

Sean (Sep 15 2020 at 21:42):

copying the same cp over and over

Sean (Sep 15 2020 at 21:43):

if it's doing what I think you're intending, you may need to pull the cp += up into the loop

Sumagna Das (Sep 16 2020 at 03:09):

Sean said:

if it's doing what I think you're intending, you may need to pull the cp += up into the loop

its in the correct loop?

Sumagna Das (Sep 16 2020 at 03:10):

but instead of at the end. it should be at first of the same loop?

Sean (Sep 16 2020 at 03:10):

think about what those loops are doing

Sean (Sep 16 2020 at 03:11):

the inner loop is copying a row of pixels

Sumagna Das (Sep 16 2020 at 03:11):

outer one is for each layer, and the inner one is for each pixel

Sumagna Das (Sep 16 2020 at 03:12):

i added one more loop for y dimensions but it slowed down the making of the vol very much

Sumagna Das (Sep 16 2020 at 03:12):

Sean said:

the inner loop is copying a row of pixels

yea

Sean (Sep 16 2020 at 03:12):

not each pixel, it's each row.. so that loop basically copies one image

Sumagna Das (Sep 16 2020 at 03:13):

yea

Sean (Sep 16 2020 at 03:13):

so as it iterates, it's copying the bytes from cp into vip

Sumagna Das (Sep 16 2020 at 03:14):

yea

Sean (Sep 16 2020 at 03:16):

well it's copying the same cp every iteration of the loop, and the loop is only incrementing x, which is the index into vip's array (via vol_bit())

Sean (Sep 16 2020 at 03:16):

but not incrementing cp because it's outside the loop

Sean (Sep 16 2020 at 03:17):

bring it in, and it should be copying the bytes correctly.

Sumagna Das (Sep 16 2020 at 03:17):

i think something else be added to cp

Sumagna Das (Sep 16 2020 at 03:17):

i just dont know what should be added

Sean (Sep 16 2020 at 03:18):

huh?

Sean (Sep 16 2020 at 03:18):

what else? you loop is right there...

Sumagna Das (Sep 16 2020 at 03:19):

zdim is actually adding 3 each time

Sean (Sep 16 2020 at 03:19):

there's only one line of code in the loop and the for condition that could possibly increment it

Sean (Sep 16 2020 at 03:20):

Sumagna Das said:

zdim is actually adding 3 each time

and is that correct?

Sumagna Das (Sep 16 2020 at 03:20):

i dont think it is correct

Sean (Sep 16 2020 at 03:21):

the inner loop is copying one layer i.e., one image out of the combined bw

Sumagna Das (Sep 16 2020 at 03:22):

Sean said:

Sumagna Das said:

zdim is actually adding 3 each time

and is that correct?

zdim is actually adding 3 for me because i am using the same picture to test it

Sean (Sep 16 2020 at 03:22):

adding 3 to what though

Sumagna Das (Sep 16 2020 at 03:22):

cp

Sean (Sep 16 2020 at 03:23):

and why would you have it do that?

Sumagna Das (Sep 16 2020 at 03:23):

i dont know actually. i just was thinking about changing it but forgot

Sumagna Das (Sep 16 2020 at 03:23):

let me test

Sumagna Das (Sep 16 2020 at 03:24):

memcpy(vol_bit(vip, x, 0), cp, vip->ydim*vip->xdim);

vip->ydim*vip->xdim this is correct for the number of bytes right?

Sean (Sep 16 2020 at 03:25):

is it?

Sean (Sep 16 2020 at 03:25):

what is xdim?

Sumagna Das (Sep 16 2020 at 03:25):

the x dimensions of the file i.e. the width of the image

Sean (Sep 16 2020 at 03:27):

and ydim?

Sean (Sep 16 2020 at 03:27):

"dimensions of the file" <-- technically incorrect

Sumagna Das (Sep 16 2020 at 03:27):

y dimensions of the file i.e. the height of the image

Sean (Sep 16 2020 at 03:27):

also technically incorrect

Sean (Sep 16 2020 at 03:28):

not file

Sumagna Das (Sep 16 2020 at 03:28):

i meant the image file

Sean (Sep 16 2020 at 03:28):

what is the input?

Sumagna Das (Sep 16 2020 at 03:29):

input to the binary object or input to the vol?

Sean (Sep 16 2020 at 03:30):

they are one in the same, no?

Sumagna Das (Sep 16 2020 at 03:30):

actually yea

Sumagna Das (Sep 16 2020 at 03:31):

the bw file of an image is inputted to the binary object and the binary object is inputted to the vol

Sean (Sep 16 2020 at 03:31):

right

Sean (Sep 16 2020 at 03:32):

so think of the bw file as a big array of bytes, because it is that

Sean (Sep 16 2020 at 03:33):

how many bytes is it?

Sumagna Das (Sep 16 2020 at 03:33):

400 * 400 *3(for me)

Sumagna Das (Sep 16 2020 at 03:33):

x dimension * y dimensions *z dimensions

Sean (Sep 16 2020 at 03:34):

which is how many bytes?

Sumagna Das (Sep 16 2020 at 03:35):

480000

Sean (Sep 16 2020 at 03:35):

great

Sean (Sep 16 2020 at 03:39):

so at the end of the day, this function you've written -- this get_obj_data() function -- is making a copy of the data putting it into its vip structure

Sumagna Das (Sep 16 2020 at 03:39):

it should do that

Sean (Sep 16 2020 at 03:40):

so you sort of copied what you saw ebm was doing , right?

Sumagna Das (Sep 16 2020 at 03:40):

but the code i have written is not doing that correctly

Sean (Sep 16 2020 at 03:40):

it's not, but it's important to understand what you're trying to do first

Sumagna Das (Sep 16 2020 at 03:41):

Sean said:

so you sort of copied what you saw ebm was doing , right?

somewhat. i also saw how the vol is being made from a file in vol_from_file() function

Sean (Sep 16 2020 at 03:41):

ah, excellent

Sumagna Das (Sep 16 2020 at 03:42):

but as it is reading a file directly, i cant follow it much

Sean (Sep 16 2020 at 03:43):

okay, no problem. I don't think you need to.

Sean (Sep 16 2020 at 03:43):

so the first question you should be asking is why are we even making a copy of the data at all given we could just access the data directly in (unsigned char *)bip->u.uint8;

Sumagna Das (Sep 16 2020 at 03:44):

that also can be done.

Sumagna Das (Sep 16 2020 at 03:44):

then extra memory wont be used

Sean (Sep 16 2020 at 03:44):

but that's a question for another day -- we see that ebm and dsp are making copies, so just for sake of argument lets assume there's some reason why

Sean (Sep 16 2020 at 03:45):

so assuming we need to copy the data out of bip->u.uint8, where are we copying it into?

Sumagna Das (Sep 16 2020 at 03:46):

from cp to vip?

Sean (Sep 16 2020 at 03:46):

don't abstract it, what's the actual to and from arrays or pointers?

Sean (Sep 16 2020 at 03:47):

cp is a pointer to the actual "from" array

Sumagna Das (Sep 16 2020 at 03:47):

from a void pointer to another void pointer?

Sean (Sep 16 2020 at 03:48):

no, those are types -- i mean what is the actual container, the actual array of bytes?

Sean (Sep 16 2020 at 03:48):

the "from" container is what cp is pointing at -- the "from" is bip->u.uint8

Sean (Sep 16 2020 at 03:49):

where are we copying "to" ?

Sumagna Das (Sep 16 2020 at 03:49):

the "to" pointer is pointing to vip->buf

Sean (Sep 16 2020 at 03:49):

excellent

Sean (Sep 16 2020 at 03:50):

how much are we copying?

Sumagna Das (Sep 16 2020 at 03:51):

i changed it right now so its copying vip->xdim number of bytes(or 400 for me)

Sean (Sep 16 2020 at 03:52):

yes, to copy by line, that is one fix

Sumagna Das (Sep 16 2020 at 03:53):

its in the inner loop btw-> for (y = 0; y < vip->ydim; y++)

Sean (Sep 16 2020 at 03:53):

yep

Sean (Sep 16 2020 at 03:54):

but if you continue with the question, you'll see this all probably simplifies greatly

Sean (Sep 16 2020 at 03:54):

how many bytes need to get copied from bip's array into vip's array?

Sumagna Das (Sep 16 2020 at 03:55):

overall or each time?

Sean (Sep 16 2020 at 03:55):

overall

Sean (Sep 16 2020 at 03:55):

ie.., what's this functions ultimate job

Sumagna Das (Sep 16 2020 at 03:56):

for me its probably about 480000 bytes

Sean (Sep 16 2020 at 03:56):

why "about 480000" ?

Sumagna Das (Sep 16 2020 at 03:57):

oh sorry it is 480000 bytes

Sean (Sep 16 2020 at 03:57):

better

Sean (Sep 16 2020 at 03:57):

so why not just do that?

Sumagna Das (Sep 16 2020 at 03:57):

copy in whole?

Sumagna Das (Sep 16 2020 at 03:57):

and not iterate?

Sean (Sep 16 2020 at 03:58):

memcpy() from bip's array to vip's array, that many bytes

Sean (Sep 16 2020 at 03:58):

one line of code

Sumagna Das (Sep 16 2020 at 03:59):

no iterations?

Sean (Sep 16 2020 at 03:59):

it's worth trying

Sumagna Das (Sep 16 2020 at 04:03):

wait a minute then

Sumagna Das (Sep 16 2020 at 04:03):

its building

Sean (Sep 16 2020 at 04:04):

there's a comment making some claim about alignment, but I think that was someone copying code from somewhere else without fully understanding the code they were copying.

and it got used again in some place (probably dsp) and again in another (ebm) and you've copied it again .. but the underlying premise that it needs to be iterated doesn't look valid to me because bip is just a big array.
arrays can be copied to arrays without iteration.

Sumagna Das (Sep 16 2020 at 04:04):

actually yea

Sumagna Das (Sep 16 2020 at 04:05):

i copied but i also doubted why the array was being copied while iterating

Sean (Sep 16 2020 at 04:06):

there's nothing wrong with iterating

Sean (Sep 16 2020 at 04:06):

other than it's just unnecessary ;)

Sean (Sep 16 2020 at 04:06):

let memcpy iterate

Sumagna Das (Sep 16 2020 at 04:06):

yea but it got messy

Sumagna Das (Sep 16 2020 at 04:14):

is this correct -> memcpy(vip->buf, cp, vip->xdim*vip->ydim*vip->zdim);?

Sean (Sep 16 2020 at 04:18):

looks like about what I had in mind

Sumagna Das (Sep 16 2020 at 04:19):

nothing appears in the graphic window

Sumagna Das (Sep 16 2020 at 04:19):

still

Sean (Sep 16 2020 at 04:19):

well that's not to say there aren't other issues

Sumagna Das (Sep 16 2020 at 04:20):

thats the main issue right now

Sean (Sep 16 2020 at 04:20):

well that's a big assumption :)

Sean (Sep 16 2020 at 04:21):

it's the issue you can see is an issue (or can't see rather, hah)

Sumagna Das (Sep 16 2020 at 04:21):

Sean said:

well that's a big assumption :smile:

yea

Sumagna Das (Sep 16 2020 at 04:23):

everything is going correctly but its still not showing up

Sean (Sep 16 2020 at 04:23):

okay, so first there are at least two issues. even if our memcpy is no good for some reason, it should at last just be messing up cell values, not affecting what is displayed

Sean (Sep 16 2020 at 04:24):

when you draw something, that ends up calling into the *_plot() function

Sumagna Das (Sep 16 2020 at 04:24):

ooh

Sean (Sep 16 2020 at 04:24):

so maybe start debugging from there

Sean (Sep 16 2020 at 04:26):

uh oh, and we do have at least another problem I think

Sean (Sep 16 2020 at 04:26):

just reading the file, I see this comment:

Sean (Sep 16 2020 at 04:26):

/*
 * Regular bit addressing is used:  (0..W-1, 0..N-1),
 * but the bitmap is stored with two cells of zeros all around,
 * so permissible subscripts run (-2..W+1, -2..N+1).
 * This eliminates special-case code for the boundary conditions.
 */

Sumagna Das (Sep 16 2020 at 04:27):

which file?

Sean (Sep 16 2020 at 04:27):

that's in vol.c

Sumagna Das (Sep 16 2020 at 04:27):

thats why it seems familiar

Sumagna Das (Sep 16 2020 at 04:28):

i saw this but didnt think it was going to matter

Sean (Sep 16 2020 at 04:28):

if it's a true statement, that changes everything

Sumagna Das (Sep 16 2020 at 04:28):

why?

Sean (Sep 16 2020 at 04:29):

think about it... a vol is a stack of bitmap images.. if every image is being wrapped in zeros, that completely changes the bytes and copying patterns needed

Sean (Sep 16 2020 at 04:29):

so consider an example:

Sean (Sep 16 2020 at 04:31):

say our vol is a 2x2x3:

01
23

45
67

89
01

Sean (Sep 16 2020 at 04:31):

three little 2x2 images

Sean (Sep 16 2020 at 04:31):

those byte values in a bw file would be: "012345678901"

Sumagna Das (Sep 16 2020 at 04:33):

no newlines

Sean (Sep 16 2020 at 04:33):

at least, if we consider these byte indices as array indicies, that's what it'd look like. bw files are actually bottom to top, so it'd actually be "230167450189"

Sean (Sep 16 2020 at 04:33):

newlines??

Sean (Sep 16 2020 at 04:33):

what newlines???

Sumagna Das (Sep 16 2020 at 04:34):

i mean its not taking them as individual lines

Sumagna Das (Sep 16 2020 at 04:34):

its taking the file as a whole

Sean (Sep 16 2020 at 04:34):

I think you're completely missing what I'm talking about if you're thinking about newlines :)

Sean (Sep 16 2020 at 04:34):

we're talking about vol data, bw data

Sean (Sep 16 2020 at 04:35):

they are unsigned 8-bit bytes

Sean (Sep 16 2020 at 04:35):

binary data

Sean (Sep 16 2020 at 04:35):

raw 0-255 values

Sumagna Das (Sep 16 2020 at 04:35):

that was a random thought. sorry for confusing.

Sean (Sep 16 2020 at 04:35):

what I "drew" was simply a visual depiction of bw values to help understand the ordering and quantity of bytes

Sean (Sep 16 2020 at 04:36):

so say you have a 2x2 bw image

Sean (Sep 16 2020 at 04:36):

it's looks sort of like this from an indexing perspective:

01
23

Sean (Sep 16 2020 at 04:37):

let say those numbers are the actual image values, just for simplicity

Sean (Sep 16 2020 at 04:37):

so on disk and in memory, the bytes will be "2301"

Sean (Sep 16 2020 at 04:37):

does that make sense?

Sumagna Das (Sep 16 2020 at 04:38):

yea

Sean (Sep 16 2020 at 04:38):

I can literally put that into code: unsigned char my_bw_image[4] = {2, 3, 0, 1}; ... that's a bw file in code

Sean (Sep 16 2020 at 04:40):

so say I have two more bw images like I showed above also. and we join them into one .bw data file using copy or cat or whatever

Sean (Sep 16 2020 at 04:40):

the bytes are simply "230167450189" for those three layers. understand that?

Sumagna Das (Sep 16 2020 at 04:40):

yea

Sean (Sep 16 2020 at 04:42):

so then if I read that bw data into a binunif array called bip, what's the value of bip.u.uint8[6] ?

Sumagna Das (Sep 16 2020 at 04:42):

4?

Sean (Sep 16 2020 at 04:42):

good

Sean (Sep 16 2020 at 04:43):

yes

Sean (Sep 16 2020 at 04:43):

okay, so now here's the stinker

Sean (Sep 16 2020 at 04:43):

that comment is saying that while the bip might be "230167450189" ... it's going to pad each of the images with zeros

Sumagna Das (Sep 16 2020 at 04:44):

like "023010...."

Sean (Sep 16 2020 at 04:44):

so instead of "2301" for the first layer, those bytes will be...

000000
000000
000100
002300
000000
000000

Sumagna Das (Sep 16 2020 at 04:45):

oooh

Sean (Sep 16 2020 at 04:45):

well, I'm assuming it's padding vertically, it may only pad horizontally like:

000100
002300

Sean (Sep 16 2020 at 04:46):

either way.. that changes the layer data in the vol to probably being "000000000000002300000100000000000000" for the first layer and so on for the other two

Sumagna Das (Sep 16 2020 at 04:46):

so what should be done?

Sean (Sep 16 2020 at 04:48):

well we have to confirm what is going on in the file read function like how you'd started and understand what it's doing (especially now that we know there may be padding being added). just keep in mind that the comment could also be totally wrong or misleading and there may be no padding, but it totally changes things if there is so we have to look and find out.

Sumagna Das (Sep 16 2020 at 04:48):

yea

Sean (Sep 16 2020 at 04:48):

we know where the data is ending up, vip->buf

Sean (Sep 16 2020 at 04:48):

so just have to look at anything being passed vip or a pointer to vip

Sumagna Das (Sep 16 2020 at 04:49):

yea but i saw there was already an unsigned char ** in vip called map

Sumagna Das (Sep 16 2020 at 04:49):

where the data was being stored for file

Sean (Sep 16 2020 at 04:50):

so why'd you say vip->buf then? :)

Sumagna Das (Sep 16 2020 at 04:50):

i saw that right now

Sean (Sep 16 2020 at 04:50):

there is no vip->buf

Sumagna Das (Sep 16 2020 at 04:51):

there is one

Sean (Sep 16 2020 at 04:51):

there's not in my code

Sumagna Das (Sep 16 2020 at 04:51):

Sumagna Das said:

yea but i saw there was already an unsigned char ** in vip called map

i am just saying can i store in this one?

Sean (Sep 16 2020 at 04:52):

saying or asking? :)

Sumagna Das (Sep 16 2020 at 04:53):

asking

Sean (Sep 16 2020 at 04:53):

you are making these decisions

Sean (Sep 16 2020 at 04:53):

with these changes.

Sean (Sep 16 2020 at 04:54):

we know we need to copy the bw data into the vol, at least that was an assumption we decided

Sean (Sep 16 2020 at 04:55):

so yeah, if the file reader is storing all the bw data into vip->map, then I'm not sure why we'd possibly want to store into anything else

Sean (Sep 16 2020 at 04:56):

if you did store into something else, that would explain why nothing plots. i mean if you added a vip->buf and there's a vip->map and the vol_plot() function is looking at vip->map and you copied into vip->buf... of course it's not going to find it.

Sumagna Das (Sep 16 2020 at 04:57):

that was my fault

Sean (Sep 16 2020 at 04:58):

so yeah, store into vip->map using a method similar if not identical to vol_from_file, but instead of fread() to copy bytes from file to memory, it'll be memcpy() to copy bytes from binunif to memory

Sumagna Das (Sep 16 2020 at 04:59):

right

Sean (Sep 16 2020 at 04:59):

use the same macros because they're the ones responsible for padding

Sean (Sep 16 2020 at 05:00):

yeah, here's the key:

    /* Get bit map from .bw(5) file */
    nbytes = (xdim+VOL_XWIDEN*2)*
        (ydim+VOL_YWIDEN*2)*
        (zdim+VOL_ZWIDEN*2);
    *map = (unsigned char *)bu_calloc(1, nbytes, "vol_import4 bitmap");

Sean (Sep 16 2020 at 05:00):

see nbytes? that's not xdim * ydim * zdim...

Sumagna Das (Sep 16 2020 at 05:00):

i have the nbytes set up

Sean (Sep 16 2020 at 05:01):

but do you understand now what that's doing?

Sumagna Das (Sep 16 2020 at 05:01):

now i understand what the macros were for

Sean (Sep 16 2020 at 05:02):

vol_from_file() allocated an array (zero-initialized via calloc) with (xdim+4) * (ydim+4) * (zdim+4)

Sean (Sep 16 2020 at 05:04):

the macros are making it skip into the array at the right position given it copied

01
23

into memory that looks like "000000000000002300000100000000000000", so the macros help jump index correctly into it

Sean (Sep 16 2020 at 05:05):

alright, my turn to rest. good luck and catch up with you in a few hours ;)

Sumagna Das (Sep 16 2020 at 05:58):

woohoo!! Screenshot-from-2020-09-16-11-27-29.png

Sumagna Das (Sep 16 2020 at 05:58):

created a vol from binary object

Sumagna Das (Sep 16 2020 at 06:06):

(deleted)

Sumagna Das (Sep 16 2020 at 07:06):

hey @Sean i submitted the patch containing all the changes

Sumagna Das (Sep 16 2020 at 07:06):

see if it is okay or not

Sumagna Das (Sep 16 2020 at 07:07):

if any changes are needed, tell me

Sumagna Das (Sep 16 2020 at 18:57):

did you check my patch?

Sean (Sep 16 2020 at 20:44):

@Sumagna Das I did check your patch and it looks good to me. that said, I haven't finished testing it but should be done soon.

Sumagna Das (Sep 16 2020 at 21:58):

did it work?

Sumagna Das (Sep 17 2020 at 15:29):

@Sean What's next?

Sean (Sep 17 2020 at 18:19):

@Sumagna Das so next is back in the plugin, no?

Sumagna Das (Sep 17 2020 at 18:19):

did it work?

Sean (Sep 17 2020 at 18:19):

oh, actually next would be to update the wdb interface

Sumagna Das (Sep 17 2020 at 18:19):

to include the obj changes?

Sean (Sep 17 2020 at 18:20):

I still haven't finished testing it. A bug came up in testing (unrelated to your changes) that I had to debug and fix. just finished up with that.

Sumagna Das (Sep 17 2020 at 18:20):

Ok

Sean (Sep 17 2020 at 18:20):

I should have it checked out here in a few hours while you're sleeping ;)

Sean (Sep 17 2020 at 18:21):

my reading of the code looked like it was good

Sean (Sep 17 2020 at 18:21):

I didn't catch any blatant issues

Sumagna Das (Sep 17 2020 at 18:21):

comments might be less though

Sumagna Das (Sep 17 2020 at 18:23):

Sean said:

oh, actually next would be to update the wdb interface

what should be updated?

Sumagna Das (Sep 17 2020 at 19:41):

what do i have to update though?

Sumagna Das (Sep 18 2020 at 20:16):

@Sean what should i update in the wdb interface?

Sean (Sep 18 2020 at 20:28):

@Sumagna Das you maybe already did, but the mk_vol() function saved the name into the ->file field of the vol, and I believe you renamed it? Assuming you did, the function could use an additional parameter to indicate whether that is auto, file, or object

Sumagna Das (Sep 18 2020 at 21:19):

Yea I changed that because it was causing compilation errors

Sumagna Das (Sep 18 2020 at 21:21):

I can add a parameter like datasrc corresponding to the actual variable in rt_vol_internal

Sumagna Das (Sep 19 2020 at 19:26):

Sean said:

Sumagna Das you maybe already did, but the mk_vol() function saved the name into the ->file field of the vol, and I believe you renamed it? Assuming you did, the function could use an additional parameter to indicate whether that is auto, file, or object

what will the datasrc parameter do anyways?

Sean (Sep 19 2020 at 20:51):

the same thing it does during the 'in' command prompting, it's the field you added to the struct that says whether the data is in an object or a file or auto

Sumagna Das (Sep 20 2020 at 06:43):

That means it will tell the function whether a binary object has been passed or a file name has been passed.

Sumagna Das (Sep 20 2020 at 20:29):

@Sean should i change the const char * parameter for the file name to a void * and it will be used according to the (new)datasrc parameter? what do you think?

Sumagna Das (Sep 21 2020 at 08:16):

I actually meant to say that if the datasrc tells the function that it has been passed an object then the void * parameter will be casted to rt_binunif(or whatever the name of the the struct was) and if it tells the function that a file is going to be read then the generic pointer will be casted to char *

Sean (Sep 22 2020 at 04:01):

No, it's still just a name - it's not passed an actual binunif @Sumagna Das .. callers first invoke mk_binunif() and then mk_vol() passing the name of the binunif created and auto or obj for the type.

Sumagna Das (Sep 22 2020 at 04:50):

so char * name will stay the same (i am talking about this parameter because i dont want to add another new parameter except for the datasrc parameter)

Sean (Sep 22 2020 at 04:51):

correct, it's just the object or file name

Sean (Sep 22 2020 at 04:51):

then the source parameter can indicate which

Sumagna Das (Sep 22 2020 at 04:52):

so i only have to add the datasrc parameter

Sean (Sep 22 2020 at 04:52):

correct

Sumagna Das (Sep 22 2020 at 04:52):

Sean said:

then the source parameter can indicate which

yea

Sumagna Das (Sep 22 2020 at 04:53):

Sean said:

No, it's still just a name - it's not passed an actual binunif Sumagna Das .. callers first invoke mk_binunif() and then mk_vol() passing the name of the binunif created and auto or obj for the type.

auto for the type is next after changing the mk_vol (if i can)

Sean (Sep 22 2020 at 04:56):

ah, I see that neither mk_dsp() or mk_ebm() were updated. they both appear to default to 'auto'.

Sean (Sep 22 2020 at 05:02):

actually, my recollection that they had an AUTO mode is apparently flawed also.

Sean (Sep 22 2020 at 05:06):

if I had to guess, I might guess that mk_dsp() and mk_ebm() are currently both broken because they don't set it and the code appears to treat unset/0 as an error since it's expecting 'o' or 'f'

Sean (Sep 22 2020 at 05:07):

would be nice to extend our fuzzer to call into the mk_() API....

Sumagna Das (Sep 22 2020 at 05:07):

fuzzer?

Sumagna Das (Sep 22 2020 at 18:34):

@Sean will wdb_export handle the vol's input source according to vol->datasrc?

Sean (Sep 22 2020 at 20:47):

@Sumagna Das wdb_export ends up calling ft_export which ends up calling rt_vol_export5() that you probably updated... but that would be the place to check whether you're serializing the datasrc field in the export5 function and deserializing it in the import5 function.

Sumagna Das (Sep 23 2020 at 03:19):

Sean said:

Sumagna Das wdb_export ends up calling ft_export which ends up calling rt_vol_export5() that you probably updated... but that would be the place to check whether you're serializing the datasrc field in the export5 function and deserializing it in the import5 function.

i updated the function which both rt_vol_import5 and rt_vol_export5 functions call so it might be less messy.

Sean (Sep 23 2020 at 06:18):

looks like your patch file is missing some changes @Sumagna Das .... include dir changes weren't included

Sumagna Das (Sep 23 2020 at 06:18):

seriously?

Sumagna Das (Sep 23 2020 at 06:18):

i might have to update the patch then

Sean (Sep 23 2020 at 13:44):

definitely

Sumagna Das (Sep 23 2020 at 13:44):

updated it already

Sean (Sep 23 2020 at 13:44):

I see it now

Sumagna Das (Sep 23 2020 at 13:45):

cleaned it up a bit also

Sean (Sep 23 2020 at 13:45):

So indentation we use is atypical these days (used to be more common)... it's 4 char indents BUT 8 char tab stops

Sumagna Das (Sep 23 2020 at 13:46):

so 8 spaces for tab?

Sean (Sep 23 2020 at 13:46):

what's your editor?

Sumagna Das (Sep 23 2020 at 13:46):

atom

Sean (Sep 23 2020 at 13:53):

I'm not as familiar with atom, but yeah I think you want to set 8 spaces for tab, and I think hard tabs, but then somewhere you want to set 4 char indentation

Sean (Sep 23 2020 at 13:55):

looks like the auto mode may work too

Sumagna Das (Sep 23 2020 at 13:55):

for mk_vol?

Sean (Sep 23 2020 at 13:57):

for atom

Sean (Sep 23 2020 at 13:57):

atom has an auto tabs indentation mode where it detects what the file is using

Sumagna Das (Sep 23 2020 at 13:58):

(deleted)

Sumagna Das (Sep 23 2020 at 13:58):

ohh

Sean (Sep 23 2020 at 13:58):

I was too... I don't understand the mk_vol question

Sean (Sep 23 2020 at 13:59):

this is for any code in brl-cad

Sumagna Das (Sep 23 2020 at 13:59):

Sean said:

looks like the auto mode may work too

i thought you were talking about auto in mk_vol

Sean (Sep 23 2020 at 14:00):

check out the "Indentation whitespace" section in HACKING for a visual diagram of the indent rules

Sumagna Das (Sep 23 2020 at 14:00):

ok

Sumagna Das (Sep 23 2020 at 14:00):

will do that

Sean (Sep 23 2020 at 14:00):

right, and that's why I said auto refers to atom indent settings

Sean (Sep 23 2020 at 14:00):

there's an auto tab setting

Sean (Sep 23 2020 at 14:00):

you might have to just try some edits to see, but first step will be to make sure you understand what it should be...

Sumagna Das (Sep 23 2020 at 14:01):

yea

Sean (Sep 23 2020 at 14:02):

I'd be a little surprised if atom can't handle the hybrid mode by default with the right settings, but this package may implement it: https://atom.io/packages/atom-smart-tabs

Sumagna Das (Sep 23 2020 at 14:02):

Sean said:

I'd be a little surprised if atom can't handle the hybrid mode by default with the right settings, but this package may implement it: https://atom.io/packages/atom-smart-tabs

thanks for that.

Sean (Sep 23 2020 at 14:03):

actually, never mind -- that's not quite what is needed

Sumagna Das (Sep 23 2020 at 14:03):

i will check it later today

Sean (Sep 23 2020 at 14:03):

that's tab indentation with alignment .. we use hybrid indentation chars

Sean (Sep 23 2020 at 14:06):

uh oh, looks like atom doesn't support it

Sean (Sep 23 2020 at 14:06):

https://discuss.atom.io/t/distiguish-between-tab-size-and-indentation-size/40197

Sumagna Das (Sep 23 2020 at 14:07):

ohh

Sean (Sep 23 2020 at 14:07):

what the moderator suggested appears to be a workaround, but a bit involved

Sean (Sep 23 2020 at 14:08):

i.e., insert spaces with tab width set to 4 but then replace all sequences of 8 chars with a tab on save

Sean (Sep 23 2020 at 14:12):

yep, this looks like the issue, auto-closed: https://github.com/atom/atom/issues/12931

Sean (Sep 23 2020 at 14:57):

I opened a new issue: https://github.com/atom/atom/issues/21371

Sumagna Das (Sep 23 2020 at 14:58):

Sean said:

I opened a new issue: https://github.com/atom/atom/issues/21371

thanks for that

Sean (Sep 23 2020 at 14:58):

so yeah, @Sumagna Das .. you're kind of in a hard place using Atom. It doesn't support the style we use so you're going to either have to deal with it manually or use another editor :(

Sean (Sep 23 2020 at 14:59):

we are planning on shifting to a different indentation style, but that's not happening yet/soon

Sumagna Das (Sep 23 2020 at 14:59):

i think i will deal with it manually right now

Sumagna Das (Sep 24 2020 at 18:39):

@Sean so should i set 4 for tab length in atom for now?

Sumagna Das (Sep 24 2020 at 18:39):

or 8

Sumagna Das (Sep 24 2020 at 19:08):

need some help with mk_binunif

Sean (Sep 25 2020 at 06:35):

@Sumagna Das I'd probably think it easiest to set it to 8 (with tabs, not spaces) and just manually fix

Sumagna Das (Sep 25 2020 at 07:48):

Ok

Sumagna Das (Sep 27 2020 at 19:14):

does the patch work?

Sumagna Das (Sep 30 2020 at 18:27):

hey @Sean does the patch work?

Sumagna Das (Sep 30 2020 at 18:27):

if yes then i will try to continue on it

Sumagna Das (Oct 01 2020 at 07:15):

hey @Sean , i am switching to vscode for now.

Sumagna Das (Oct 01 2020 at 07:16):

so are the indentation and tab space settings different in there?

Sumagna Das (Oct 01 2020 at 07:16):

i used it many days ago but didnt know much about editor so if you know please tell me

Sean (Oct 02 2020 at 20:02):

Hahaha, @Sumagna Das VScode has the same problem. :)

Sumagna Das (Oct 02 2020 at 20:03):

:lol:

Sean (Oct 02 2020 at 20:03):

It's Visual Studio (the Professional version) that supports mixed-mode. Looks like for VSCode that some of the devs want to implement it and others want to encourage projects to choose spaces vs tabs.

Sumagna Das (Oct 02 2020 at 20:04):

i am one of those devs

Sean (Oct 02 2020 at 20:04):

Don't worry about it. Just insert spaces for now. They're the easiest to fix.

Sumagna Das (Oct 02 2020 at 20:04):

i switched to vscode for some perfomance and extension issues

Sean (Oct 02 2020 at 20:05):

We're planning on switching, but then I thought we'd decided on one thing and others thought we'd decided on something else, and it was deferred

Sumagna Das (Oct 02 2020 at 20:05):

Oh

Sumagna Das (Oct 02 2020 at 20:06):

and what is the status of the Github migration

Sean (Oct 02 2020 at 20:06):

and then deferred again for gsoc, and then release, and the can keeps getting kicked as once we do switch it, it will make most patches fail to apply cleanly (or at least a significant number of them) ... so there's some desire to close the patches out first.

Sean (Oct 02 2020 at 20:07):

there's a checklist on the wiki. I had a few remaining issues that needed to be manually checked by someone other than starseeker. You helped with a couple of them, so some can be checked off.

Sumagna Das (Oct 02 2020 at 20:08):

i am checking that list but seems like nothing has changed or it hasnt progressed much because of the releases

Sean (Oct 02 2020 at 20:08):

yes, nothing progressed while release was happening

Sean (Oct 02 2020 at 20:09):

it's only been in the past week, I finished one of them and need to annotate the ones you did

Sean (Oct 02 2020 at 20:09):

(or you can)

Sumagna Das (Oct 02 2020 at 20:11):

Sean said:

(or you can)

i dont know about that

Sean (Oct 02 2020 at 20:27):

no worries

Sumagna Das (Oct 07 2020 at 19:31):

@Sean did you check the vol patch if it works? i will proceed only after fixing all the bugs in there (if there are any)

Sumagna Das (Oct 23 2020 at 12:54):

Sorry @Sean that I couldn't update the patch as I was busy with my studies. Will resume working on it after setting up my editor for BRL-CAD.

Sumagna Das (Oct 24 2020 at 14:36):

@Sean Did you see any bugs with the vol patch or anything more i need to add to the patch? I don't think i missed anything.

Last updated: Oct 09 2024 at 00:44 UTC