@Sean the dm-fb-merge branch is working - I was able to test it on Linux this morning, and I'll check Windows later today. Have no way to try graphically on the Mac (the github CI runners aren't set up to build X11 graphically, and I can't launch remotely from them to confirm GUI behavior in any case as far as I can tell...) so not sure what's happening there :-(
@Sean dm-fb-merge opengl display works on Linux, Windows, GhostBSD and OpenBSD here.
Woo-hoo! Still need to figure out four libged commands, but other than that have Tcl/Tk completely out of the core library stack below libged in the dm/fb branch.
saw the commit, that's awesome
think those the ones behind 'screenshot' too, yes?
er, 'screengrab'
screengrab is focused on the dm rather than the fb, so it might actually still work
or maybe not, might be going straight to the dm interface
that's cool
four of them were reading into a fbserv_obj for some reason - presumably to pull the rendered fb out of MGED/Archer, although I'm not sure of that yet. Not quite sure whether they should be talking to the dm to have it broker the right data flows, or make them plugin commands for libged supplied by libtclcad - needs more study.
they shouldn't exist in that form, imho
Heh - the ged struct entry they were using had a "this shouldn't be here" comment
I think I wrote that 10 years ago
Will have to see if there's a better way to express the functionality they were trying to encapsulate - could be the screengrab logic might actually do the trick...
fb2pix should probably just be part of screengrab, the overlay command, or draw/erase
pix2fb similarly with overlay or draw/erase
/me nods - makes sense
png2fb ditto
icv should be hooked behind that so we don't have to care about image format, imho
fbclear becomes overlay subcommand or erase switch
yeah
just have to decide how to handle the render buffer
with dm and fb merged, probably everything belongs under draw/erase
even overlay probably should merge under draw
"draw file.plot3" ?
we should design it so it's deterministic, but yeah, something similar
draw -overlay file.plot3
Ah, right - the difference between "on top" of the scene and "in" the scene
draw -overlay [file] vs draw [object], unless it's a fully qualified uri that can disambiguate like draw file:/path/to/file.plot3
/me reflects that as good as that felt, it actually didn't answer the immediate need, which is to get a proper Tk dm/fb backend up to replace the X11 fallback with 8.6... think I have some decent progress there, but need to revisit the original work that mated up the fb rendering with the Tk_Photo environment. Glad we saved that, it's going to come in handy now.
that's one that would be good to spend a few minutes designing for at least the cases we have now: draw geometry into the scene, draw images fixed to the viewport, draw plot/data into the scene, draw images into the scene, and (probably) draw plot/data into the viewport
/me nods - agreed. I'd also like to think about how we're handling multiple display managers (e.g. quad view) and how to address them.
that only leaves one outlier (draw geometry fixed to viewport), but "overlay" vs default-into-scene still might be the best way to disambiguate
so the 3d "scene" is universal, and only overlays are unique to specific viewports/dm instances?
maybe: draw [-view #] [-overlay [file]] [object]
or maybe better stated, the "dm instances" are different cameras into the same scene?
right, and by default you draw into all of them but certainly could draw into just one of them
Do we want solid objects to be up in one dm but not another? Just off the cuff I'm kinda liking the simplicity of the solid list being universal, with only the view object lists being unique to the camera views. May be good reasons to have the flexibility though.
(wondering what the interpretation of "draw -view 1 object" would be...)
Not by default and not something we presently support (I think) but there are lots of common use cases for custom view sets. for example, maybe show camera/view objects in the 3d scene in only some views (e.g., to show where a camera is in a 3rd "rendered" viewport) or to display reference data/images into the 3d scene, etc
but yeah, emphasis on the not by default -- should be you draw something, it displays in all the viewports
draw -view 1 would just draw something only into that viewport, default would be akin to "-view all" or "-view *" or something
@starseeker just a caution -- whenever you see something that seems to be only libtclcad, I've learned that is usually something Bob inserted for some Archer-built tool. Those four commands sound like something possibly in use by SlideMaker or EGM.
@Sean That was my guess as well. However, anything Archer based will also have libtclcad available, so if need be they can be supplied at that level. Once it starts getting closer to going live I'm planning to do more testing with the Archer level pieces (and we'll also have to coordinate with the other tools that have hooked libdm/fb into Java and Qt...)
At least in theory it should be possible to do that more cleanly with the new setup by making purpose-build dm plugins - I'm planning at some point to try a better Qt dm/fb backing for qged to test that out.
@starseeker understood that they're in libtclcad (or can be), that's not the issue.
the danger is leaving them disabled. they will be runtime failures not likely to be caught by future testing.
at least not without specific intent to find their usage. like a testing checklist.
fair enough, but unless we can contrive some tests for them I can't prove they work even in trunk, much less in the branch...
especially since they could even be specific to a single interface or tool, it'd be good to manage the risk in some manner.
unless you can prove they don't work and weren't broken recently by other changes, the converse is just as valid a position...
Not sure we've got any sort of testing at all for libtclcad beyond the regression tests themselves, for that matter...
if they're working in slidemaker, for example
I'll see if Bob can take a look quick - if not I'll have to come up with some way to exercise them
the lack of testing and awareness of whether it works or not is why the burden is raised on change .. this is going to be a recurring theme with bob out
it's going to tie our hands somewhat or we're going to have to pay down the debt as changes are made
IMHO we can't afford to tie our hands
it would be nice to find a uniform way to test the things in libtclcad that are not in libged. if they're in libged though some alternative feature, it's less of a concern.
I'd agree that we don't want to, that we shouldn't -- but not that we can't. Unknowingly but likely breaking tools that are functioning today would be irresponsible too. We can't do that to users.
Again just my opinion, but we urgently need some significant foundational changes to get us moving forward. Agree we don't want to break existing tools, but stasis is potentially fatal as a project.
I'll try to put my money where my mouth (or fingers technically) are by doing what I can to pay down the debt
Dude, you're preaching to the choir. I know that, you know that.
I'm not sure if you suggesting doing nothing about this is acceptable for the sake of change.
I'm saying I think that is not just unacceptable too but that it would be irresponsible to just yank something and hope for the best or expect it to get dealt with by [insert future situation].
that it should be managed in some manner. documented and followed up with when it comes time to testing could be a minimum, providing an alternative interface could be an option too.
And it's not just avoiding changing things because they might be risk. It's that those commands are potentially tied to a user-visible feature and knowingly breaking them without an alternative or testing or some mitigation would be a violation of our own change policy.
Ah. No, I'm not suggesting ignoring it is acceptable when the breakage is definitely known.
I guess my bias is to proceed forward when we can't readily prove there is breakage. (We know there is in this case, so I agree it needs to be dealt with before anything goes live.)
"ignorantia juris non excusat"
if there is breakage, whether we know about it or not, then the change policy has been violated and user trust is eroded (or completely lost in some instances) when it is encountered.
that doesn't mean we can't/shouldn't change things -- we can and do all the freaking time. that's exactly why there is a policy written on how we should go about doing it in a manner that is defensible and should minimize loss of user trust.
I'm also not suggesting against proceeding forward just because you don't know -- it was simply to take some action to at least document the situation and follow up. A policy of only acting on proven issues encourages simply not looking, not testing, not asking, not communicating. Not healthy development.
Fair enough. What would be good for this case? (Assuming I can't run them to earth quickly) - TODO file a the top level in the branch?
I think a TODO file in the branch for testing / search against tools as an action for the next release.
If there's not other way to save the fb or write an image into it, then it would be good to figure out some way in libged to do that cleanly in the new fb+db rework state.
@Sean If I roll back the changes in this branch to be just the dm/fb merger and the plugin loading, without the fb_obj relocation that's causing issues with the libged commands, would there be a reasonable chance to merge this back in? I didn't radically restructure the dm/fb interactions, and I'm concerned that this is going to be difficult to maintain as a branch for a long period give how much code moved around in the dm/fb libs...
@starseeker what would that mean in terms of overall changes? what's the benefit to trunk? the dm+fb merger by itself isn't very compelling by itself as that's inherently undoing modularity. that downside was being offset by the improvement to libged's modularity. if we don't get the libged improvement, won't trunk be worse off? I'm not familiar with the extent of the plugin loading changes, but I imagine that effect has to be minor as we don't rely on it for anything yet.
merge concern noted, though -- what are our options? is it not possible to preserve/rewrite those four commands so the whole branch can be merged?
Somewhat to my surprise, it looks like I was able to get the four commands (or a more generic version in png2fb's case) working.
@Sean I've not hooked it into overlay yet (right now I've just stubbed in icv2fb and fb2icv commands) but so far it looks like I might be able to completely replace fb2pix, pix2fb and png2fb.
How should the usability of this work? Should there be an "--fb" option to overlay to specify that the image is going into the framebuffer rather than the dm? Should the fb->image feature be an option to saveview?
Er - screengrab, not saveview...
roughed in -F options to overlay and screengrab in the branch. Haven't gotten any of the offset logic enabled or tested yet, but getting close
@starseeker That's great!
offset logic is pretty much working now - trying to convince screengrab to return subsets of its image
did the old commands do that?
starseeker said:
How should the usability of this work? Should there be an "--fb" option to overlay to specify that the image is going into the framebuffer rather than the dm? Should the fb->image feature be an option to saveview?
I think I saw you went with -F to imply the main / default?
That should be consistent enough with other use where -F specifies a framebuffer -- might want to make it "-F {fb_specification}" to be backwards compatible, something like "-F default"
should talk to any FB ideally, so things like -F 4 still work
Don't think saveview comes into play with this as that's talking about the camera / viewer parameters of the view, not the image / viewee aspects
The old commands were kind of an odd mishmash . With the exception of some png2fb specific options which don't seem to have been used anywhere (probably added for testing, if I had to guess) I'm trying to use libicv's image features to support rectangle subsets of the fb - that seems to have been the overall intent, but I don't think it was fully realized.
Right now I'm just using -F to distinguish between capturing the framebuffer and capturing the scene view - I don't have the ability to specify fbserv objects, and I don't think the old commands did either. I'm using the display manager pointer rather than talking over the fb network protocol, so the code's not well set up for arbitrary fb capture. If I should use an option other than -F to avoid that expectation that shouldn't be an issue - suggestions welcome.
Did dm-fb-merge build OK on OSX?
/me is trying another Windows build now...
starseeker said:
The old commands were kind of an odd mishmash . With the exception of some png2fb specific options which don't seem to have been used anywhere (probably added for testing, if I had to guess) I'm trying to use libicv's image features to support rectangle subsets of the fb - that seems to have been the overall intent, but I don't think it was fully realized.
So... I think you're saying "no"?
If the old code didn't I honestly wouldn't bother with extracting subsets even if it seems easy enough. Bob was a fan of doing that for some reason and it's just such an antipattern...
It would partially do subsets - you could change the x and y starting offsets to get portions of an image, but you couldn't specify the length of what you wanted in each direction from that point. I think there may be one or two places in Archer and other codes that use that feature.
Take a look at the screengrab and overlay commands in the branch - that's got my current feature set.
(and yes, I know I need to add/update man pages - waiting until we've got the commands in a stable state ;-)
I could see yanking all of the image subset code in mged/archer and downstream tools. That's like the feature he implemented to just rt-render a subset of the view that has been band-selected.
In all my years, I can't say I've ever wanted or needed to do that or heard of anyone else (besides Bob) wanting to do that. If it didn't complicate the usability, it wouldn't matter, but that's a feature we could probably strip for the sake of simplicity throughout (including downstream apps).
@Sean Well, for immediate purposes we could probably just not add it to screengrab/overlay - the other 4 commands aren't exposed directly to users to begin with. I'd rather do any more substantial rippage post-merge...
starseeker said:
Take a look at the screengrab and overlay commands in the branch - that's got my current feature set.
I did, that's why I've been giving comments on this and the -F option ... I saw and, hence, the attempt at having a design discussion early / contemporaneously.
I'm good with yanking the subset options - my only potential use case was band-selecting from a GUI, and I agree I've never wanted to do that in MGED proper - if I need an image subset I usually snap the whole image and bring it up in another tool to grab the piece I want.
Exactly why I mention it. That could all go from userland. Might leave the feature in rt as a debugging/advanced option, but definitely not menu options and propagation to libged commands. That complexity, however minor, doesn't appear to be terribly helpful.
By the way, branch build is mostly clean, but did hit a link error:
[ 73%] Linking C shared library ../../../libexec/dm/libdm-X.dylib
Undefined symbols for architecture x86_64:
"_vectorThreshold", referenced from:
_X_drawVList in dm-X.c.o
ld: symbol(s) not found for architecture x86_64
Um. That's legit. Interesting it didn't show up previously...
I think r76025 should deal with the X_drawVList issue.
vectorThreshold is icky... unless someone is actually manipulating that somewhere my vote would be to #define it and eliminate the global altogether...
yeah, doesn't look like anybody's touching it at all.
I hadn't compiled this branch prior. Looks like you yanked it when tcl.c went away.
Yep, got moved into libtclcad, but dm-X and dm-tk were referencing it. Surprised Linux build didn't fail. Anyway, dealt with now.
nod Just FYI, though, that TclLinkVar call made it available to Tcl scripts. It's conceivably called by a bit of Tcl somewhere, but don't see it in any of our Tcl. Just something to keep an eye out for later post-release. Fix will just be to delete the caller code, so it's not a problem.
@Sean sounds good.
Windows and Linux seem (knock on wood) to be behaving. I'll give OpenBSD a shot next.
@Sean did you get a chance to try OSX with the fix?
compilation succeeded with the fix
Awesome. MGED and Archer should work - if they do, what else would you like to see prior to a trunk merge?
@Daniel Rossberg How disruptive do you think the work in the dm-fb-merge branch will prove to the C++ code? I'm working towards merging that as the final step in pushing Tcl/Tk up out of our core libs.
A first test was successful. There was no change necessary on the C++ core interface source code.
However, dm-fb-merge's CMake is complaining about a missing fontconfig, which I do have installed. The build works nevertheless. And, I saw that lz4 is gone.
What was the specific fontconfig error? That ought to work, at least on Linux...
I folded the key bits we use from lz4 into librt, so that's correct.
CMake Warning at src/other/tk/CMakeLists.txt:259 (find_package):
By not providing "FindFontconfig.cmake" in CMAKE_MODULE_PATH this project
has asked CMake to find a package configuration file provided by
"Fontconfig", but CMake did not find one.
Could not find a package configuration file provided by "Fontconfig" with
any of the following names:
FontconfigConfig.cmake
fontconfig-config.cmake
Add the installation prefix of "Fontconfig" to CMAKE_PREFIX_PATH or set
"Fontconfig_DIR" to a directory containing one of the above files. If
"Fontconfig" provides a separate development package or SDK, be sure it has
been installed.
(Debian buster, kernel 4.19.0-9-amd64, gcc (Debian 8.3.0-6) 8.3.0)
Ah. What version of CMake? That error means FindFontconfig isn't available by default in that version.
@Daniel Rossberg I put back the misc/CMake copy
starseeker said:
Ah. What version of CMake? That error means FindFontconfig isn't available by default in that version.
In Debian buster the FindFontconfig.cmake is included in KDE development packages, but in Debian sid in CMake data and extra too.
Figures. "Modern" CMake seems to be trying to move more towards *Config.cmake based approaches where those files are provided by the packages, but I'm not up on that approach and if the upstream package doesn't provide it we're back to Find modules anyway...
@Sean Did MGED/Archer launch OK on the Mac with that build?
starseeker said:
What was the specific fontconfig error? That ought to work, at least on Linux...
I folded the key bits we use from lz4 into librt, so that's correct.
Eek, why?? That's a very active project that pushes out updates regularly. It would be nice to just update their sources or utilize a system lz4. That was working I thought...
starseeker said:
Sean Did MGED/Archer launch OK on the Mac with that build?
It's not been compiling for a few days. I thought you were in the middle of something, so I didn't mention it, but it dies with duplicate symbols when linking fftc.
(rather, trunk and bioh both fail with that error)
[ 33%] Linking C executable ../../bin/fftc
duplicate symbol '___sincos' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_signbitl' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isnanl' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isnormall' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isinfl' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isfinitel' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___sincospi' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_signbitf' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___sincosf' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isnanf' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isnormalf' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___sincospif' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isinff' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isfinitef' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_signbitd' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isnand' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isnormald' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isinfd' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___inline_isfinited' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
duplicate symbol '___sputc' in:
CMakeFiles/fftc.dir/fftc.c.o
CMakeFiles/fftc.dir/splitditc.c.o
ld: 20 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [bin/fftc] Error 1
make[1]: *** [src/libfft/CMakeFiles/fftc.dir/all] Error 2
Here's the actual compile line, so it looks like compilation options were changed elsewhere as those are all math system math symbols that were somehow embedded (maybe related to import/export changes?):
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -std=c11 -D_POSIX_C_SOURCE=200809L -D_XOPEN_SOURCE=700 -pipe -fvisibility=hidden -fno-strict-aliasing -fno-common -fexceptions -m64 -ggdb -Qunused-arguments -fstack-protector-all -fno-omit-frame-pointer -pedantic -pedantic-errors -Wall -Wextra -Wundef -Wfloat-equal -Wshadow -Wbad-function-cast -Wc++-compat -Winline -Wno-long-long -Wno-variadic-macros -Wdocumentation -Wno-c11-extensions -Werror -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -mmacosx-version-min=10.14 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -m64 -ggdb CMakeFiles/fftc.dir/fftc.c.o CMakeFiles/fftc.dir/splitditc.c.o -o ../../bin/fftc -lm
Good call - libfft would have been getting BRLCADBUILD and HAVE_CONFIG_H from src/CMakeLists.txt. Interesting Mac seems to be the only platform that hit that particular issue... r76142 should restore the definitions.
In trunk that is - I'll merge to dm-fb-merge next.
There we go. r76143 in dm-fb-merge
@Sean FWIW, trunk just built successfully on the github OSX runner.
dm-fb-merge branch as well, although unfortunately the runner environment doesn't end up enabling any of the graphics...
@Sean did dm-fb-merge build OK for you?
It builds and runs if I enable all, but default build fails with X11 symbol errors (like a header inclusion is missing). Trunk doesn't have the error, so it's some change.
error was on dm-ogl.c iirc
Blast. I really wish Apple would make some sort of remote test platform available for devs...
/me waddles through a plethora of training
/me nods - always lots of fun
@starseeker I dug a lot deeper into the failure, did tons of comparisons with trunk, and looking at the specific cause of the failure -- it's related to the detection of tcl/tk on the branch. We might have to revisit the tcl logic on trunk, but it doesn't look like a show-stopper now that I found the cause. I think it's safe enough to merge.
@Sean OK, thanks for digging into it - sorry it proved to be such a hassle. I'm wondering if I should try to get ahold of a second-hand Mac somewhere for testing purposes - I seem to be hitting a lot of failure that only manifest there...
That could probably be said of any platform we don't compile on daily. I can count at least a dozen times I made some super safe innocuous change that I thought was perfectly okay only to find it broke the [xxx] build.
I think we can just keep pushing more CI, take steps to not break intentionally, investigate and learn when it does break, and move on.
phew - merged. (hah - happened to be commit 76200 - nice round number)
Last updated: Jan 09 2025 at 00:46 UTC