starseeker (Oct 01 2020 at 13:45): starseeker (Oct 01 2020 at 13:45):@Sean There are several ways I can go with a superbuild pattern, all of which have their tradeoffs:
I can treat BRL-CAD the same way I'm treating the other builds, listing final outputs explicitly for management by the superbuild during final install.
I can define a custom install script for the superbuild that reconfigures and recompiles the BRL-CAD subbuild for its final target location, avoiding any need for the superbuild to explicitly manage BRL-CAD outputs but requiring some recompilation of BRL-CAD for the install step.
I can do the custom install script for all subbuilds, not just BRL-CAD, in lieu of the custom ExternalProject extensions and output listing I'm doing currently. This is probably the simplest option from the superbuild's standpoint, but we would likely pay a substantial recompilation penalty for the install step as we'd be rebuilding most or all of the src/other projects.
My inclination is to try number 2, but at this juncture a second opinion is probably in order...
Sean (Oct 02 2020 at 20:30):@starseeker I honestly didn't understand any of those three options. lol.
Sean (Oct 02 2020 at 20:30):It was my understanding that cmake has proper sub-build support so one shouldn't need to explicitly list or script things.
Sean (Oct 02 2020 at 20:31):what're others doing?
Sean (Oct 02 2020 at 20:32):I'm also not sure what's meant by recompiling for final location. Is that's referring to embedded rpaths, those are also adjustable post-compilation (not sure how strong cmake's support is, but the tooling is pretty straightforward).
Sean (Oct 02 2020 at 20:33):I think I've seem some projects do that on install, just re-run install_tool and ldconfig and such.
Sean (Oct 02 2020 at 20:37):I think we don't want to be rebuilding anything in src/other -- they do their thing, they're installed, and they're done. BRL-CAD's build (or BRL-CAD's superbuild) just detects or is explicitly configured where they exist and uses them in their locations.
So I guess in the superbuild context, src/other is make+install stage followed by a make stage for BRL-CAD and, optionally, install.
Sean (Oct 02 2020 at 20:46):Still would be good to see what others did. That's just my naive thoughts based on how ar and ld behave, what they need. I also assume you're sorting through how to run pre-install (including 3rd party deps) and that's certainly appealing too.
starseeker (Oct 03 2020 at 00:41):Ah, sorry - assumed too much context.
I'm attempting to create a superbuild that will use ExternalProject_Add for the 3rd party builds where that makes sense, but still preserve the current property where we can create a relocatable bundle (including those 3rd party deps.)
When using ExternalProject_Add, to ensure proper ordering (3rd party builds complete, then build BRL-CAD using their outputs) it is necessary that BRL-CAD itself be built with ExternalProject_Add. No other available CMake mechanism provides the necessary ordering guarantees, since the 3rd party builds must be completed and installed before the BRL-CAD build makes any attempt to compile its source files. (My first attempt, which didn't do that, got very close to success but finally ran aground on that limitation.)
If we perform BRL-CAD's "main" build with ExternalProject_Add, the superbuild must take over the responsibility of assembling the final package that will form the BRL-CAD binary (incorporating the 3rd party deps and BRL-CAD into a single install package.) My three options were various approaches that could be taken to work towards achieving that result.
starseeker (Oct 03 2020 at 03:16):I'm pretty sure what others are doing is unlikely to be as aggressive as what I'm trying to do, in the sense that I'm trying to preserve the present property of our build that the build directory compilation output is fully functional without requiring write access to the final install location. ExternalProject_Add complicates that quite a lot - I've solved some of the problems, but
To allow the creation of a working setup without requiring the user have permissions to write to the final install location, the superbuild sets the "install" location for all subprojects to its own build directory. Once all of the ExternalProject_Add "install" steps are complete, the superbuild's "build" step can be considered complete and it can then assemble the outputs of all the various subbuilds into the final package.
For most of the projects I have a series of extensions to ExternalProject that list and process their expected output files to achieve this final assembly - essentially duplicating the RPath magic CMake uses for this process but applying it to the outputs of the ExternalProject_Add builds. This appears to work, but it's a tall order for BRL-CAD - our output changes too much and too rapidly to want to manually maintain such an output list. That leaves us with the tradeoffs mentioned before.
It's probably possible to teach our CMake logic to write out the necessary information to allow the superbuild to process our outputs directly, and that would have the advantage of being consistent with the way other projects are managed, but it's likely to be the most work to set up.
The "cheating" alternative is to just reconfigure the BRL-CAD subbuild to recompile as much as is needed to support the CPack requirements for creating the final bundle - essentially create a "make install" step for the superbuild that manipulates the BRL-CAD compilation as needed to achieve the desired results. That's easier to experiment with and less work, but even it if proves viable it will mean that an "install" step (whether make install or package creation) will also entail a significant amount of recompilation of the BRL-CAD libs/programs.
starseeker (Oct 05 2020 at 12:47):One other possibility might be to invoke the ExternalProject builds from the configure stage of the main build, rather than as part of the build process. That would guarantee the correct ordering as far as the main build is concerned, and the main price would be a (potentially) very long configure process as the src/other builds were completed to have the correct find_package inputs ready to go...
Sean (Nov 07 2020 at 02:59):Hey @starseeker just noticed the recent commits and a couple things.. first, have a vague recollection that sdai header names and paths are mandated a particular way for implementations to be conformant. Might be breaking things hard there. Other issue, those changes are pushing us farther from upstream, which has a slew of fixes. I was hoping to update post migration.. and that’d require undoing/redoing everything.
Sean (Nov 07 2020 at 03:00):Basically, SDAI isn’t just a bunch of arbitrary design decisions. That interface is part of STEP spec iirc.
starseeker (Nov 07 2020 at 04:09):I made a quick try at using the upstream, and for whatever reason it didn't work. Debugging why has a high probability of being a laborious, time-consuming process, as they've changed a number of things about how the C++ files are generated.
starseeker (Nov 07 2020 at 04:11):The whole setup badly needs simplification - it's still using a variation on my original CMake build logic, which ended up being way too heavy for the purpose of the libraries in retrospect. Unfortunately, I don't have any sense at this point of how involved that would be while avoiding breaking the python pieces.
starseeker (Nov 07 2020 at 04:12):At this point I'm just trying to zero in on why the Ninja Windows build specifically is failing (while the Visual Studio build on the same machine, same compiler does not.) I'm getting (I think) some form of memory corruption trying to read a step file in step-g, but only with the Ninja build.
starseeker (Nov 07 2020 at 04:13):exp2cxx has a separate problem on the Mac, which I've not been able to diagnose at this point, in either case I'm trying to eliminate all the extraneous complexity I can.
starseeker (Nov 07 2020 at 04:13):If I need to scrap what I'm doing now once I actually figure out what the problem is that's fine...
starseeker (Nov 07 2020 at 04:15):I suspect I'm not understanding some of the motivations for the SDAI interface design properly - so far my experiences working with it have been pretty negative.
starseeker (Nov 07 2020 at 04:19):If I had the time (which I acknowledge I don't) I'd probably want to go back to the original plan - grab the express bits, and write a new convertor logic ignoring SDAI that focuses just on what we need for converters.
starseeker (Nov 09 2020 at 23:32):Growl... best guess at this point is the build flags from the MSBUILD setup happen to produce a working converter, the Ninja generator is generating something different, and the STEPCODE C++ isn't tolerating it...
starseeker (Nov 09 2020 at 23:32):Main flaw in that theory is the all-in-one build does work with Ninja...
Sean (Nov 10 2020 at 14:58):@starseeker If you want, I can try to track down more specifically whether sdai subdirs violate or are okay with the spec -- that's a case where I'd have to actually go back to the spec and read up on it. My recollection could be wrong, but I seem to recall it being spelled out in detail for conformance testing of implementations. Happy to look, it just might take a while to locate and read up on it. Been a while since I dug through the docs.
Sean (Nov 10 2020 at 14:58):I think that was also a reason we'd have to do some dynamic work to have support for multiple AP's in a single binary.
starseeker (Nov 10 2020 at 14:59):No problem, and not terribly urgent - upstream doesn't work, the refactor didn't fix what I'm fighting on the github runners, and even if we waded through it all a correct answer would result in no functional improvements :-/
Sean (Nov 10 2020 at 15:00):I was going to say, that all seems orthogonal to the problem you described with ninja.
Sean (Nov 10 2020 at 15:00):if a ninja build is different, that should be directly observable -- what's being compiled or linked differently?
starseeker (Nov 10 2020 at 15:00):It is - I knew it probably would be, but I was trying to minimize the amount of clutter I had to wade through. It served its purpose, as far as that goes...
Sean (Nov 10 2020 at 15:01):fwiw, I don't think you were on the discussion thread, but AP214 support came up just a couple days ago. matt suggested an scr so we can prioritize it.
starseeker (Nov 10 2020 at 15:01):I did finally figure out how to get a look at the comparative build options - msbuild is not friendly, when it comes to that. It looks like the root issue was that I wasn't passing CMAKE_BUILD_TYPE to the subbuilds, so they were doing the default thing.
starseeker (Nov 10 2020 at 15:02):(very much hoping that's it, and some preliminary tests succeeded - that's why I reverted)
Sean (Nov 10 2020 at 15:03):I'm surprised that even worked at all... Debug builds aren't typically considered link-compatible with Release builds on Windows.
Sean (Nov 10 2020 at 15:03):-Mt vs non-Mt can also make them incompatible iirc
starseeker (Nov 10 2020 at 15:04):I think it was probably the std::string exposure that actually failed - most of the other libs probably accidentally worked, but I was getting warnings about std::string needing some kind of DLL exporting instructions from the step code. Was really hoping I wouldn't have to dive that deep... that's a very thorny rabbit hole
starseeker (Nov 10 2020 at 15:05):I think I'm getting it sorted now - in retrospect it's not surprising we're seeing these sorts of issues, since direct add_subdirectory integration into the build gives us a lot of settings for free...
Sean (Nov 10 2020 at 15:10):aha! yes, that'd be an issue with debug and release -- where is std::string exposed in the API?
starseeker (Nov 10 2020 at 15:11):In some of the classes - hang on, I'll try to pull some of the logs from github
Sean (Nov 10 2020 at 15:12):it's typically not recommended to expose STL across DLL boundaries, because it will compile but will result in run-time crashes as the debug vs non-debug versions of c++ template instantiation are different (and in production release will result in runtime crashes where it might locally work just fine).
Sean (Nov 10 2020 at 15:12):that's an old time msvc issue.
Sean (Nov 10 2020 at 15:13):as far as I know/knew though, STEPcode didn't/doesn't expose STL across a public library boundary
starseeker (Nov 10 2020 at 15:17):Here's an example:
2020-11-09T17:43:18.9047539Z D:\a\cadcitest\cadcitest\src\other\ext\stepcode\include\stepcode\utils/errordesc.h(58,29): warning C4251: 'ErrorDescriptor::_userMsg': class 'std::basic_string<char,std::char_traits<char>,std::allocator<char>>' needs to have dll-interface to be used by clients of class 'ErrorDescriptor' [D:\a\cadcitest\cadcitest\build\src\other\ext\STEPCODE_BLD-prefix\src\STEPCODE_BLD-build\src\dai\stepdai.vcxproj] [D:\a\cadcitest\cadcitest\build\src\other\ext\STEPCODE_BLD.vcxproj]
2020-11-09T17:43:18.9050908Z C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.27.29110\include\xstring(4564): message : see declaration of 'std::basic_string<char,std::char_traits<char>,std::allocator<char>>' [D:\a\cadcitest\cadcitest\build\src\other\ext\STEPCODE_BLD-prefix\src\STEPCODE_BLD-build\src\dai\stepdai.vcxproj]
Sean (Nov 10 2020 at 15:32):ahh, I see.
Sean (Nov 10 2020 at 15:35):So @starseeker this is a deeper issue (two issues actually)
Sean (Nov 10 2020 at 15:37):first, someone apparently exposed STL in an interface there. that's typically discouraged for truly portable library code, but isn't uncommon and is perfectly fine under static (i.e., non-dll) compilation.
Sean (Nov 10 2020 at 15:38):I seem to recall STEPcode originally having an SCLstring class to specifically avoid this issue. That may be something we pulled from upstream or did to ourselves.
Sean (Nov 10 2020 at 15:39):That said, it can be made to work in a DLL (which is undoubtedly how it's working now on trunk) where we just assume we're compiling and linking with the same compiler and same options.
Sean (Nov 10 2020 at 15:39):We're obviously using the same compiler, so the error is definitely coming from different compilation options being used.
Sean (Nov 10 2020 at 15:41):So fix will be to either make the compilation options consistent which should prevent it from throwing the error about not having a dll-interface, or to re-wrap the STL types behind a class (ugh).
Sean (Nov 10 2020 at 15:43):this is an ancient c++ issue, fwiw. here's some background on the issue as it pertains to msvc, but it's not specific to msvc: https://stackoverflow.com/questions/5661738/how-can-i-use-standard-library-stl-classes-in-my-dll-interface-or-abi
Sean (Nov 10 2020 at 15:48):also worth mentioning that microsoft just open sourced their libc++ just last year and there's a lot of callouts made about c++ stl compatibility, so what was once very bleak and a "don't ever do that" isn't as much a concern going forward fortunately.
starseeker (Nov 10 2020 at 16:10):/me nods - yeah, it's looking like fixing the build type (which aligns the options properly) is getting buy it. We originally switched to std::string to get off of the SCL string class, which IIRC had some issues, but it looks like we didn't end up PImpling it well enough.
starseeker (Nov 13 2020 at 21:29):@Sean did you get a chance to try the extbuild branch on the Mac?
Sean (Nov 17 2020 at 19:45):yes, sorry, I thought I'd conveyed that .. it fails in a couple places. the tclsh8.6 binary is crashing (image not found error, so something is wrong in the linking/pathing) and it throws a compile-time error in itk3 (looks like trunk issue using system Framework headers)
Sean (Nov 17 2020 at 19:45):/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -DBUILD_itk -DHAVE_DLFCN_H=1 -DHAVE_ERRNO_H=1 -DHAVE_FLOAT_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_LIMITS_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STDINT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRINGS_H=1 -DHAVE_STRING_H=1 -DHAVE_SYS_PARAM_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_WAIT_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DTCL_THREADS=1 -DUSE_ITCL_STUBS -DUSE_TCL_STUBS -DUSE_TK_STUBS -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tcl.framework/Headers -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tk.framework/Headers -I/Users/morrison/brlcad.extbuild/.build/include -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic -I/Users/morrison/brlcad.extbuild/.build/src/other/ext/ITK_BLD-prefix/src/ITK_BLD-build/include -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/unix -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../itcl3/generic -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -mmacosx-version-min=10.14 -o CMakeFiles/itkstub.dir/generic/itkStubLib.c.o -c /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c:31:
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itk.h:79:
In file included from /Users/morrison/brlcad.extbuild/.build/include/itclInt.h:50:
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclInt.h:2765:14: error:
unknown type name 'Tcl_NRPostProc'; did you mean 'Tcl_CloseProc'?
MODULE_SCOPE Tcl_NRPostProc TclNRForIterCallback;
^~~~~~~~~~~~~~
Tcl_CloseProc
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tcl.framework/Headers/tcl.h:652:15: note:
'Tcl_CloseProc' declared here
typedef void (Tcl_CloseProc) _ANSI_ARGS_((ClientData data));
^
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c:31:
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itk.h:79:
In file included from /Users/morrison/brlcad.extbuild/.build/include/itclInt.h:50:
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclInt.h:2766:14: error:
unknown type name 'Tcl_NRPostProc'; did you mean 'Tcl_CloseProc'?
MODULE_SCOPE Tcl_NRPostProc TclNRCoroutineActivateCallback;
^~~~~~~~~~~~~~
Tcl_CloseProc
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tcl.framework/Headers/tcl.h:652:15: note:
'Tcl_CloseProc' declared here
typedef void (Tcl_CloseProc) _ANSI_ARGS_((ClientData data));
^
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c:31:
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itk.h:79:
In file included from /Users/morrison/brlcad.extbuild/.build/include/itclInt.h:50:
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclInt.h:2768:14: error:
unknown type name 'Tcl_NRPostProc'; did you mean 'Tcl_CloseProc'?
MODULE_SCOPE Tcl_NRPostProc TclNRTailcallEval;
^~~~~~~~~~~~~~
Tcl_CloseProc
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tcl.framework/Headers/tcl.h:652:15: note:
'Tcl_CloseProc' declared here
typedef void (Tcl_CloseProc) _ANSI_ARGS_((ClientData data));
^
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c:31:
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itk.h:79:
In file included from /Users/morrison/brlcad.extbuild/.build/include/itclInt.h:50:
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclInt.h:2774:14: error:
unknown type name 'Tcl_NRPostProc'; did you mean 'Tcl_CloseProc'?
MODULE_SCOPE Tcl_NRPostProc TclNRReleaseValues;
^~~~~~~~~~~~~~
Tcl_CloseProc
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tcl.framework/Headers/tcl.h:652:15: note:
'Tcl_CloseProc' declared here
typedef void (Tcl_CloseProc) _ANSI_ARGS_((ClientData data));
^
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c:31:
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itk.h:79:
In file included from /Users/morrison/brlcad.extbuild/.build/include/itclInt.h:50:
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclInt.h:2886:14: error:
unknown type name 'Tcl_NRPostProc'; did you mean 'Tcl_CloseProc'?
MODULE_SCOPE Tcl_NRPostProc TclClearRootEnsemble;
^~~~~~~~~~~~~~
Tcl_CloseProc
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tcl.framework/Headers/tcl.h:652:15: note:
'Tcl_CloseProc' declared here
typedef void (Tcl_CloseProc) _ANSI_ARGS_((ClientData data));
^
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c:31:
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itk.h:79:
In file included from /Users/morrison/brlcad.extbuild/.build/include/itclInt.h:50:
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclInt.h:4888:5: error:
unknown type name 'Tcl_NRPostProc'; did you mean 'Tcl_CloseProc'?
Tcl_NRPostProc *procPtr;
^~~~~~~~~~~~~~
Tcl_CloseProc
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tcl.framework/Headers/tcl.h:652:15: note:
'Tcl_CloseProc' declared here
typedef void (Tcl_CloseProc) _ANSI_ARGS_((ClientData data));
^
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c:31:
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itk.h:79:
In file included from /Users/morrison/brlcad.extbuild/.build/include/itclInt.h:50:
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclInt.h:4928:
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclIntDecls.h:153:8: error:
unknown type name 'CONST86'
EXTERN CONST86 char * TclpGetUserHome(const char *name,
^
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclIntDecls.h:153:16: error:
expected identifier or '('
EXTERN CONST86 char * TclpGetUserHome(const char *name,
^
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclIntDecls.h:262:8: error:
unknown type name 'CONST86'
EXTERN CONST86 char * TclSetPreInitScript(const char *string);
^
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclIntDecls.h:262:16: error:
expected identifier or '('
EXTERN CONST86 char * TclSetPreInitScript(const char *string);
^
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclIntDecls.h:691:5: error:
unknown type name 'CONST86'
CONST86 char * (*tclpGetUserHome) (const char *name, Tcl_DString *bu...
^
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclIntDecls.h:691:13: error:
expected member name or ';' after declaration specifiers
CONST86 char * (*tclpGetUserHome) (const char *name, Tcl_DString *bu...
~~~~~~~ ^
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclIntDecls.h:691:12: error:
expected ';' at end of declaration list
CONST86 char * (*tclpGetUserHome) (const char *name, Tcl_DString *bu...
^
;
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclIntDecls.h:750:5: error:
unknown type name 'CONST86'
CONST86 char * (*tclSetPreInitScript) (const char *string); /* 101 */
^
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclIntDecls.h:750:13: error:
expected member name or ';' after declaration specifiers
CONST86 char * (*tclSetPreInitScript) (const char *string); /* 101 */
~~~~~~~ ^
/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic/tclIntDecls.h:750:12: error:
expected ';' at end of declaration list
CONST86 char * (*tclSetPreInitScript) (const char *string); /* 101 */
^
;
16 errors generated.
make[5]: *** [CMakeFiles/itkstub.dir/generic/itkStubLib.c.o] Error 1
make[4]: *** [CMakeFiles/itkstub.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [src/other/ext/ITK_BLD-prefix/src/ITK_BLD-stamp/ITK_BLD-build] Error 2
make[1]: *** [src/other/ext/CMakeFiles/ITK_BLD.dir/all] Error 2
You had mentioned earlier the possibility of my getting a login somewhere on an OSX box - is that practical? It's looking like I'm going to have to iterate on this to try and run it to ground - especially worried about the linking/path issue, which I know has nuttiness unique to OSX...)
Sean (Nov 17 2020 at 19:58):If I manually change the compile line to this, it works:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -DBUILD_itk -DHAVE_DLFCN_H=1 -DHAVE_ERRNO_H=1 -DHAVE_FLOAT_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_LIMITS_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STDINT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRINGS_H=1 -DHAVE_STRING_H=1 -DHAVE_SYS_PARAM_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_WAIT_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DTCL_THREADS=1 -DUSE_ITCL_STUBS -DUSE_TCL_STUBS -DUSE_TK_STUBS -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tcl.framework/Headers -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tk.framework/Headers -I/Users/morrison/brlcad.extbuild/.build/include -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic -I/Users/morrison/brlcad.extbuild/.build/src/other/ext/ITK_BLD-prefix/src/ITK_BLD-build/include -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/unix -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../itcl3/generic -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -I/usr/X11/include -mmacosx-version-min=10.14 -o CMakeFiles/itkstub.dir/generic/itkStubLib.c.o -c /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c
(namely, changed frameworks to isysroot and added the missing X11 include dirs
Sean (Nov 17 2020 at 20:00):let me know if you want me to test a different configuration -- I believe this is default
Sean (Nov 17 2020 at 20:00):(cmake ..)
starseeker (Nov 17 2020 at 20:22):Did we change something in trunk to fix that? It shouldn't be pulling in framework headers at all, really...
starseeker (Nov 17 2020 at 20:23):I see I adjusted the extbuild FindTCL on 11/3 but apparently that didn't do the trick.
Sean (Nov 17 2020 at 20:23):my last trunk build was enable all so I don't know the current status. I seem to recall you tweaking the FindTcl file.
Sean (Nov 17 2020 at 20:23):i'll try a trunk default build
starseeker (Nov 17 2020 at 20:47):@Sean If you get a chance, can you confirm https://github.com/starseeker/citest also fails on your machine?
Sean (Nov 17 2020 at 21:30):[ 93%] Built target stepeditor_stage
Scanning dependencies of target stepdai_stage
[ 94%] Generating lib/libstepdai.2.0.0.dylib
[ 95%] Generating lib/libstepdai.dylib
[ 96%] Generating lib/libstepdai.2.dylib
[ 96%] Built target stepdai_stage
Scanning dependencies of target exp2cxx_exe_stage
[ 97%] Generating bin/exp2cxx
[ 97%] Built target exp2cxx_exe_stage
Scanning dependencies of target exptest
[ 98%] Generating exp.stamp
dyld: Library not loaded: @rpath/libexppp.2.dylib
Referenced from: /Users/morrison/brlcad.trunk/cliff_test/.build/bin/exp2cxx
Reason: image not found
make[2]: *** [exp.stamp] Abort trap: 6
make[1]: *** [CMakeFiles/exptest.dir/all] Error 2
Interesting! Different message, but same failure point. Now the question is why...
starseeker (Nov 17 2020 at 22:57):For tcl, r77671 and 77674 were the changes I was thinking of - rather surprised we're still picking up a framework with those in place...
starseeker (Nov 18 2020 at 00:18):@Sean I adjusted the RPATH logic for Mac - can you try rebuilding? And if tclsh or btclsh don't work, can you check what otool says about their rpath settings?
starseeker (Nov 18 2020 at 03:55):Nevermind, I finally got a useful printout on the runner. Ick.
starseeker (Nov 18 2020 at 13:21):My, sourceforge is cranky today... lots of "svn: E000013: Can't open file '/svn/p/brlcad/code/db/txn-current-lock': Permission denied" errors...
starseeker (Nov 18 2020 at 15:57):@Sean If you can try extbuild r77744 and maybe get a debugger on one of the crashing btclsh runs I'd appreciate it - I'm still getting an abort trap but now I'm not seeing any immediately obvious issues with the rpaths...
starseeker (Nov 18 2020 at 15:58):Actually, wait - I'm still testing asc2g, let me first check btclsh directly
starseeker (Nov 18 2020 at 16:03):Grr. Yeah, Abort trap. What on earth...
starseeker (Nov 18 2020 at 19:31):Anybody know any good tricks to get gdb (or is it lldb with clang?) to spit out a backtrace on a github runner when a scripted program crashes?
starseeker (Nov 19 2020 at 01:10):Interesting. We didn't actually need to link libitcl, and Linux figured that out automatically - in the process, it ignored what would have been a lib lookup problem. OSX went ahead and tried to find it (and failed) which is apparently why btclsh wasn't working.
starseeker (Nov 19 2020 at 01:10):Now just need to figure out why rpath setting is failing on OSX during package build...
starseeker (Nov 19 2020 at 04:14):/me is going crazy... I wish Apple would make dev images available for debugging purposes or something
starseeker (Nov 19 2020 at 14:14):@Sean If you do a vanilla autotools build of tclsh on the Mac, does otool -l report that LC_RPATH is set in the executable?
Sean (Nov 20 2020 at 07:37):still need me to test extbuild?
Sean (Nov 20 2020 at 08:26):also, since 77752 doesn't really seem to do what's needed, should it stay in there? the logic in there looks potentially problematic as I think I saw a separate Darwin case, so that's potentially a no-op or undefined behavior logic.
starseeker (Nov 20 2020 at 13:57):No, don't need to test right now (thanks though) - trying your idea to use install_name_tool
starseeker (Nov 20 2020 at 13:58):Agreed - r77752 should be obe and removed in the next few commits
starseeker (Nov 20 2020 at 15:43):@Sean I'm going somewhat cross-eyed. With r77757 I'm seeing the following: https://brlcad.org/~starseeker/osx_runner.log
Per otool -l, the install_name_tool command is running but not changing the path:
Before:
2020-11-20T15:13:22.8653280Z OLD_RPATH: /Users/runner/work/cadcitest/cadcitest/build/lib
2020-11-20T15:13:22.8654490Z NEW_RPATH: /usr/brlcad/rel-7.32.1/bin;@loader_path/../bin
2020-11-20T15:13:22.8679810Z -rwxr-xr-x 1 runner staff 49688 Nov 20 14:43 /Users/runner/work/cadcitest/cadcitest/build/_CPack_Packages/Darwin/TBZ2/BRL-CAD_7.32.1_Darwin_x86_64/bin/tclsh8.6
2020-11-20T15:13:22.8833280Z OTOOL_OUT: /Users/runner/work/cadcitest/cadcitest/build/_CPack_Packages/Darwin/TBZ2/BRL-CAD_7.32.1_Darwin_x86_64/bin/tclsh8.6:
2020-11-20T15:13:22.8834170Z Mach header
2020-11-20T15:13:22.8935420Z Load command 19
2020-11-20T15:13:22.8935770Z cmd LC_RPATH
2020-11-20T15:13:22.8936100Z cmdsize 64
2020-11-20T15:13:22.8944310Z path /Users/runner/work/cadcitest/cadcitest/build/lib (offset 12)
Command:
2020-11-20T15:13:22.8946440Z install_name_tool -change /Users/runner/work/cadcitest/cadcitest/build/lib /usr/brlcad/rel-7.32.1/bin;@loader_path/../bin /Users/runner/work/cadcitest/cadcitest/build/_CPack_Packages/Darwin/TBZ2/BRL-CAD_7.32.1_Darwin_x86_64/bin/tclsh8.6
After:
2020-11-20T15:13:22.9239390Z Load command 19
2020-11-20T15:13:22.9240000Z cmd LC_RPATH
2020-11-20T15:13:22.9240630Z cmdsize 64
2020-11-20T15:13:22.9241520Z path /Users/runner/work/cadcitest/cadcitest/build/lib (offset 12)
Can you see anything wrong with the install_name_tool command?
starseeker (Nov 20 2020 at 15:45):I'll see if I can capture a binary package off of the runner to allow for direct inspection...
starseeker (Nov 20 2020 at 16:47):brlcad.org/~starseeker/BRLCAD_OSX.zip
starseeker (Nov 20 2020 at 17:53):OK, nevermind -change doesn't work, but successive -delete_rpath and -add_rpath calls do.
starseeker (Nov 20 2020 at 17:55):Rebuilding with full config enabled - if that works, will be ready for build testing on a platform with graphics capability. (Should know in a few hours if the full build succeeds... - no bets either way at this point.)
starseeker (Nov 20 2020 at 20:00):/me is reminded of why he disliked dealing with the original Tcl build so much...
starseeker (Nov 20 2020 at 20:00):Setting LDFLAGS to our build directory's lib directory messes with Tcl's configure.
starseeker (Nov 20 2020 at 20:33):Or, more correctly, configure messes with our LDFLAGS setting. Grr.
starseeker (Nov 21 2020 at 16:38):@Sean It's probably worth trying a Mac build now (extbuild branch).
starseeker (Nov 22 2020 at 22:11):If it works for you I'd like to consider merging back to trunk...
Sean (Nov 23 2020 at 15:34):at a glance, I would have questioned whether semi-colon delimited is valid or not. since you can add multiple rpaths, I would kind of expect that to be not valid syntax unless something explicitly said so
Sean (Nov 23 2020 at 15:35):I could also see that semicolon making the install_name_tool command invalid (missing arg), hence no result
Sean (Nov 23 2020 at 15:35):I'll give the extbuild a compile today
Sean (Nov 23 2020 at 15:37):that's major surgery, it really needs multiple configurations tested before being merged back to trunk -- I really can't have trunk be unstable on me right now
Sean (Nov 23 2020 at 15:39):by the way, is 77803 cheating? ... you're supposed to fix one if you added one, not bump the number :)
Sean (Nov 23 2020 at 17:16):Here's log files as of 77804, fresh build dir:
cmake.log
build.log
starseeker (Nov 23 2020 at 21:15):Yes, it's cheating, but I couldn't afford the distraction of chasing down a platform symbol in the middle of other testing - jaw is locked on getting this done before I lose it.
starseeker (Nov 23 2020 at 21:15):Blast it - we're STILL getting the framework headers included???
starseeker (Nov 23 2020 at 21:17):Alright, I'll leave trunk alone for now.
starseeker (Nov 23 2020 at 21:19):Oh... it looks like its trying to build against the system Tcl/Tk. That's not good...
starseeker (Nov 23 2020 at 21:21):At least, if it's trying to build against the framework. The Tk graphics test should have caught that and failed it... not too surprised that bit's not working I guess, since it's the one piece I can't test here at all.
starseeker (Nov 23 2020 at 21:25):/me shakes head... I'll try to clean up what I've got and make sure someone can pick it up later. In one sense it probably wouldn't be a bad thing for me to leave it and return to it later with fresh eyes.
starseeker (Nov 23 2020 at 21:28):I think at this point I've demonstrated the fundamental practicality of the mechanism itself - it's just cleaning up all the details of the various platforms.
Sean (Nov 24 2020 at 05:25):starseeker said:
Yes, it's cheating, but I couldn't afford the distraction of chasing down a platform symbol in the middle of other testing - jaw is locked on getting this done before I lose it.
We should delete the test if you're not going to adhere to it, or if you just see it as a distraction. That's the only / entire point of the test. Bumping the number is up there with turning off a particular warning because they fail strict build. Of course, there's a difference if increases are coming from 3rd party being updated vs code we touch/write/modify.
Sean (Nov 24 2020 at 05:29):I also can't say I support intentionally degrading or relaxing our code quality measures to reach any goal. Maybe a pause to come back with fresh energy later would be good. Otherwise, let me know if there's something else I can test on Mac to figure out what's wrong there with the framework logic.
starseeker (Nov 24 2020 at 14:47):I'll try to get the number back down before we merge to trunk - I needed to canvas the regression test behavior across multiple platforms looking for design issues, and it was easier to adjust the number in the branch for that purpose than to manually change it on each platform.
starseeker (Nov 24 2020 at 14:54):I'll be pausing on this anyway - the urgency was the need for Thread Building Blocks to address the parallel issues, and from what you were saying the stability of trunk is more critical to that goal unless/until we are actually ready to introduce the new dependency.
I think the fundamental issues have largely been solved - I know now how to use the imported target mechanism and custom targets to integrate the ExternalProject builds into our main build, and demonstrated that in principle it can work across multiple platforms. I'll try to cherrypick a few build simplifications back into trunk, but otherwise I'll leave it unless/until we need to make the shift.
Sean (Nov 24 2020 at 15:07):starseeker said:
I'll try to get the number back down before we merge to trunk - I needed to canvas the regression test behavior across multiple platforms looking for design issues, and it was easier to adjust the number in the branch for that purpose than to manually change it on each platform.
But that begs the question why it's increasing at all. If platform-specific code is being introduced as a pattern of problem solving, that's also no good imho, no matter the reason. If it's coming from other people's code, so be it (but then that begs a question why regression is seeing it).
starseeker (Nov 24 2020 at 15:08):In that particular case it was because the new 3rd party logic has to accommodate the naming patterns of files produced by 3rd party build systems, and those names are highly platform specific. OpenBSD is particularly entertaining, because unless the build outputs match their particular naming pattern find_package doesn't see them.
Sean (Nov 24 2020 at 15:12):If you don't see how that applies here, coming to agreement / understanding is going to be really hard.
Sean (Nov 24 2020 at 15:13):anything with different behavior can be framed as being platform specific. there wouldn't be a difference otherwise.
Sean (Nov 24 2020 at 15:14):it's like binaries using .exe on windows. sure that's platform specific, but the point is how to deal with that deviation.
Sean (Nov 24 2020 at 15:15):one can always frame that in a platform perspective and type up code that does ifdef _WIN32, do_something(".exe") .. but that is the pattern that costs more in the long-term, particularly to the person that didn't write that code working with other environments.
starseeker (Nov 24 2020 at 15:15):For me it was an order of operations priority. IFF this can work, it may be worth trying to figure out a better solution to that particular issue. Until last week, I wasn't even sure it could work at all. My take was that I need to prove that it could work before the effort to improve things like the platform file convention name handling were worth the investment.
starseeker (Nov 24 2020 at 15:18):I'm not immediately sure what a better solution would look like, to be honest - Windows, Apple, *BSD/Unix, and OpenBSD all have different naming conventions, and individual libraries also have their own naming quirks. I'm certainly open to a better way to generalize that, but I'm not really sure where to start...
Sean (Nov 24 2020 at 15:19):but you don't...
Sean (Nov 24 2020 at 15:22):You have to see you've used that reasoning before many times. Either something else steals priority attention, thus not returning to do any "better solution", or you literally run out of time and can't justify spending more effort on it, or you're just tired with it. All the while, it almost never happens.
Sean (Nov 24 2020 at 15:25):That's not a jab at you as much as it is the avalanche of competing priorities and values. There's always going to be something else. If writing "code complete" is not valued while it's being written, it's not likely to happen later as it costs even more then.
starseeker (Nov 24 2020 at 15:26):Well, for something like this and given my own capabilities and resources, it's the best I can do. The option is to not attempt these large changes.
Sean (Nov 24 2020 at 15:27):I guess I've just also seen you do things differently too (often grumbling, but when you do it turns out well)
Sean (Nov 24 2020 at 15:29):search is a great example. The code doesn't look like you short-circuited much of anything in there and it's been pretty robust to new development, hasn't felt brittle, has been exemplary of a really hard chunk of code made to work well.
starseeker (Nov 24 2020 at 15:30):It's also a lot simpler in terms of its interactions - it's almost completely isolated from any platform specific issues. It's world is the .g file and BRL-CAD APIs - I love it when we can live there, because it's far simpler.
starseeker (Nov 24 2020 at 15:30):The build system is a lot more like the compatibility layers we have to put in libbu, IMHO
starseeker (Nov 24 2020 at 15:31):I'm not a fan of platform specific stuff - indeed, the whole CMake conversion was an attempt to reduce platform specific build logic - but at some point, somewhere, those differences must be abstracted.
starseeker (Nov 24 2020 at 15:31):The question is where and how.
Sean (Nov 24 2020 at 15:32):the plugins is probably a good platform-specific example. it's had instability, but by large hasn't had platform-specific injection (unless I missed them). granted, some of that was already refactored into libbu.
Sean (Nov 24 2020 at 15:33):speaking of which, I assume you saw bills messages -- looks like gcv is broken
Sean (Nov 24 2020 at 15:33):or his build was incomplete
starseeker (Nov 24 2020 at 15:33):bu_dlopen and friends are the only reason we were able to get away with that
Sean (Nov 24 2020 at 15:33):I didn't scrutinize his logs to figure out which
starseeker (Nov 24 2020 at 15:33):It's broken - I think I know what it is
starseeker (Nov 24 2020 at 15:34):I'm on leave this week, but I'll see if I can fix it - short term fix is probably to revert 77777
Sean (Nov 24 2020 at 15:36):I just guess I really am not approving of giving CMake a pass on these issues because it's build system code.
Sean (Nov 24 2020 at 15:36):Code is code. Maybe we need the equivalent of libbu-for-cmake so platform-specific things aren't intermixed with build logic.
starseeker (Nov 24 2020 at 15:36):Hmm. I like that idea, actually
Sean (Nov 24 2020 at 15:37):I see the complexity and it takes me ages to figure out what's actually going on, so much spaghetti and side effects
starseeker (Nov 24 2020 at 15:39):Agreed - I don't like it myself, and I know as the author of most of it it's way easier for me to deal with. But that also means I'm not going to be the best one to come up with how to solve the problem, since I'm inherently less able to recognize it when it's there.
Sean (Nov 24 2020 at 15:39):I see things like the gcv conversion and it's concerning when the build logic goes from being a one-liner in a subdir to being 20 lines. Other places, it's logic repeated in a half dozen places that seem to all be doing the same thing, the same repetition of code blocks all including the same pattern.
starseeker (Nov 24 2020 at 15:43):If we want something better than my ad-hoc solutions to a lot of those problems, we should probably take a day or two to examine what the various BRLCAD_ADD* and BRLCAD_MANAGE_* functions are doing, what the plugins need/do that is different, and how best to organize it all.
Sean (Nov 24 2020 at 15:44):heh, i've not even encountered BRLCAD_MANAGE_ yet...
starseeker (Nov 24 2020 at 15:45):I think you have and you've forgotten/blocked the memory - you didn't care for it, IIRC
Sean (Nov 24 2020 at 15:46):I would believe that -- I don't remember it at all, but then I don't like what's implied by the name either... :)
starseeker (Nov 24 2020 at 15:46):It's the bit responsible for staging data files and the like in the build directory and setting up their install rules. Has to do one thing for platforms with symlinks and another for platforms without.
Sean (Nov 24 2020 at 15:46):BRLCAD_ADD is all fine/good stuff, reasonable familiar abstraction
Sean (Nov 24 2020 at 15:48):ahh. yeah, that sounds like something I wouldn't like implemented as some sort of "management" interface. it should be patterned after whatever it's doing, not a catch-all.
Sean (Nov 24 2020 at 15:49):in gcv's case, though -- was there a reason the plugin logic couldn't be put behind a function/macro? there was already one there and it was replaced I think?
starseeker (Nov 24 2020 at 15:50):Anyway, there's that, the BRLCAD_ADD bits and their attendant functions (of which there are a lot), the plugins, and the doc building, not to mention the various converter running functions for db/ and the tclIndex building stuff.
starseeker (Nov 24 2020 at 15:51):Um. Probably not - I was just duplicating the pattern established for libdm/libged, which put the plugins in the correct libexec subdirs. Undoubtedly all of that should be consolidated into a BRLCAD_PLUGIN function or something along those lines, but that begs the question of what functionality it should share with BRLCAD_ADDLIB
starseeker (Nov 24 2020 at 15:55):I backed out the gcv plugin change - hopefully that'll get bill working until I figure out what was making Windows unhappy.
starseeker (Nov 24 2020 at 15:56):I thought a bit about making a "PLUGIN" option to add to BRLCAD_ADDLIB, but I'm not sure if that's ontologically correct.
Sean (Nov 24 2020 at 15:56):I need to write a gcv plugin myself. For me a lot of this becomes more clear when I go through the motions of adding another and seeing what knowledge is being required of others to extend it.
starseeker (Nov 24 2020 at 16:00):The whole GCV setup is very add-hoc, or at least the gcv executable itself is - my recollection is that was mostly me trying to figure out how on earth to expose arbitrarily complex options for all the plugins on the command line. I don't recall being involved too much with the initial GCV plugin design/setup, so I don't know as much about it.
starseeker (Nov 24 2020 at 16:02):I am pretty sure though that the gcv plugins predate our support for the libexec directory.
Sean (Nov 24 2020 at 16:04):yeah, they do predate it. how the plugins are set up and managed isn't meant to dominate gcv. if anything, it just needs to be something simple for authors (hence importance of writing a plugin), which I think we recently failed so I intended to take a look at it all here soon.
starseeker (Nov 24 2020 at 16:05):/me nods. Whatever we decide on, we should update libdm and libged to do as well so we're consistent across the board.
Sean (Nov 24 2020 at 16:06):@Sumagna Das test was pretty much a failure on GCV's part... I spent a day debugging only to realize what we were trying to make it do wasn't possible without fundamentally changing some things (not a good sign for any plugin system).
Sean (Nov 24 2020 at 16:07):consistent would be great, but they do serve rather different purposes. ged and dm's plugins are a means to an end. for gcv, it's intentional architectural and conceptual design.
Sean (Nov 24 2020 at 16:08):they may (or may not) end up being the same as they have slightly different needs, different runtime profiles.
starseeker (Nov 24 2020 at 16:09):Ah, the mime type thing? I'd have to go back over the discussions, but my recollection is there were at least one or two fundamental conceptual issues that were unresolved (like how to handle multiple plugins supporting the same format, or plugins that could do multiple different things with the same input - i.e., a converter that can embed a png as an image in a .g fle or generate an extruded solid from that png file... which behavior does the user want, and how do they specify? Moreover, how do they learn that they can specify such options?)
Sean (Nov 24 2020 at 16:13):the mime type was a symptom -- but no, not it specifically. it's when facing a new dev with the task of implementing a simple plugin, are they likely independently successful without pulling in another dev or a debugger. in his case, the simple plugin was riddled with issues including mime types, other plugins seeming taking priority, inconsistent searching rules making it not obvious what his plugin is actually doing, is it even loading and failing or not loading, etc.
Sean (Nov 24 2020 at 16:13):point being there wasn't a solution that didn't involve modifying gcv and didn't involve a debugger.
starseeker (Nov 24 2020 at 16:16):Ah, gotcha.
Sean (Nov 24 2020 at 16:17):and to be clear, that's not a negative on you or anyone else. it's just a status check on the state of GCV.
starseeker (Nov 24 2020 at 16:17):Agreed. Have to say it doesn't surprise me, really - I found the problem pretty overwhelming the last time I poked at gcv.
Sean (Nov 24 2020 at 16:18):that's why I was saying I think I need to add a plugin, that will make it real and relatable to me, and should suss out the essential pain points without overarchitecting solutions.
starseeker (Nov 24 2020 at 16:19):The png case was actually an interesting one - we didn't consider the possibility of non-geometry to geometry conversions when the mimes were originally set up, so it makes me wonder how we can avoid similar unconscious assumptions
Sean (Nov 24 2020 at 16:19):I think part of the problem can be improved if we eliminate concepts of "auto". We have to get declarative working, so behavior is deterministic and self-discoverable.
starseeker (Nov 24 2020 at 16:20):So make the plugins be explicit about their formats? GDAL's gonna be a doozy...
Sean (Nov 24 2020 at 16:21):I considered it as creating geometry from images and images from geometry has been on my mind for gcv from the onset, but even still I didn't realize the implications of the mime type groupings at the time it was implemented.
starseeker (Nov 24 2020 at 16:21):Hmm. Does "gcv file.g image.png" become an alias for rt?
Sean (Nov 24 2020 at 16:22):gdal is a great example actually
Sean (Nov 24 2020 at 16:22):for example, I wouldn't just write out all potential gdal inputs ... they'd need to be tested and we turn on the ones we know work, and then we immediately explicitly are faced with what to do when two things read stl.
Sean (Nov 24 2020 at 16:24):that came up with gdal stealing png input and then consequently crashing GCV because of something not getting handled right. if png weren't explicitly enabled, that wouldn't have happened until someone tested it and made sure it was working.
Sumagna Das (Nov 24 2020 at 18:52):what happened to GCV exactly?
Sumagna Das (Nov 24 2020 at 18:58):I dont have time to read the whole conversation right now so can anyone give a brief account of it?
starseeker (Nov 24 2020 at 18:59):Just discussion about what design decisions are needed to correct the current difficulties of working with/adding to it
Sean (Nov 24 2020 at 19:09):yeah, sorry for the ping @Sumagna Das -- that's not something you need to read through. Summary is just that gcv needs some lovin' and your work making a plugin for it make that obvious.
Sumagna Das (Nov 24 2020 at 19:09):its ok...i just dont have time right now due to my final exams
Sean (Nov 24 2020 at 19:10):no problem, maybe I'll have it cleaned up by the time you're done with exams ;)
Sean (Nov 24 2020 at 19:10):good luck with your exams @Sumagna Das !
starseeker (Dec 02 2020 at 03:15):@Sean Don't update the platform symbol count yet - the reduction in trunk is targeted to get the count in extbuild below the current specified level in trunk.
starseeker (Dec 02 2020 at 03:16):(Did it in trunk to keep the merge simpler - let me know if you prefer to keep those changes out of trunk until the merge...)
starseeker (Dec 04 2020 at 03:57):@Sean If you've got spare machine cycles, it would be interesting to know if the extbuild's failure mode has changed on the Mac with the latest changes to the branch.
Sean (Dec 04 2020 at 03:58):I'll check it. My cpu fans finally died down just an hour ago (been doing renders for a particular tasker that have been keeping things .. hot)
starseeker (Dec 04 2020 at 04:00):urk. nevermind, then - I'll ping after I've gone another round with the github runners
Sean (Dec 04 2020 at 04:01):I'm saying it's fine, I can
starseeker (Dec 04 2020 at 04:01):Ah, OK - wasn't sure if you were wanting to cool things down a bit first ;-)
Sean (Dec 04 2020 at 04:11):it was a bit anticlimactic... had a single render take nearly 24 hours and it was barely better than the previous at around 30 min.
Sean (Dec 04 2020 at 04:12):I miscalculated how much I'd increased the workload, but figured I'd let it complete anyways
starseeker (Dec 04 2020 at 04:13):Blegh. Hate it when that happens...
starseeker (Dec 06 2020 at 18:26):Got extbuild distcheck-full pass on both ubuntu and GhostBSD.
Sean (Dec 07 2020 at 06:39):Cool! What's GhostBSD like?
starseeker (Dec 07 2020 at 13:35):It's one of the current iteration of the "GUI desktop" BSDs - tends to be very current (I think it was the first distro I hit with clang 10)
starseeker (Dec 07 2020 at 13:37):FreeBSD based (it's kinda my "FreeBSD stand-in" so I don't have to bother with getting Xorg, etc. set up on FreeBSD.)
starseeker (Dec 07 2020 at 13:40):As long as I remember to use gmake I can get a successful distcheck-full on it, so I keep it around as my "second platform" for release testing. bz manifests that threading issue, which means it can build but not run the tests, and OpenBSD has some sort of core dump with MGED that I've so far been unable to debug on that platform (I can't get a version of gdb or egdb to successfully load the dump...)
Sean (Dec 07 2020 at 16:30):Sounds interesting. I've not done much with any of the BSD desktop distros.
Sean (Dec 07 2020 at 16:31):@starseeker FYI, common.h provides GCC_PREREQ() and ICC_PREREQ() for detecting minimum versions in our code, doing the right thing for versions that pre-date the new gcc built-ins.
starseeker (Dec 07 2020 at 16:32):Ah, OK. I'll shift those over - thanks.
Sean (Dec 08 2020 at 08:26):I have a compile of the latest extbuild going now, should have a mac compile-test result for you in the morning.
Sean (Dec 08 2020 at 08:54):actually have it now, failed:
-- Looking for IceConnectionNumber in ICE - found
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/morrison/brlcad.extbuild/.build/src/other/ext/ITK_BLD-prefix/src/ITK_BLD-build
[ 24%] Performing build step for 'ITK_BLD'
Scanning dependencies of target itkstub
[ 11%] Building C object CMakeFiles/itkstub.dir/generic/itkStubLib.c.o
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c:31:
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itk.h:79:
/Users/morrison/Applications/include/itclInt.h:58:10: fatal error: 'common.h'
file not found
#include "common.h"
^~~~~~~~~~
1 error generated.
make[5]: *** [CMakeFiles/itkstub.dir/generic/itkStubLib.c.o] Error 1
make[4]: *** [CMakeFiles/itkstub.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [src/other/ext/ITK_BLD-prefix/src/ITK_BLD-stamp/ITK_BLD-build] Error 2
make[1]: *** [src/other/ext/CMakeFiles/ITK_BLD.dir/all] Error 2
make: *** [all] Error 2
More detail:
[ 11%] Building C object CMakeFiles/itkstub.dir/generic/itkStubLib.c.o
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -DBUILD_itk -DHAVE_DLFCN_H=1 -DHAVE_ERRNO_H=1 -DHAVE_FLOAT_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_LIMITS_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STDINT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRINGS_H=1 -DHAVE_STRING_H=1 -DHAVE_SYS_PARAM_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_WAIT_H=1 -DHAVE_UNISTD_H=1 -DSTDC_HEADERS=1 -DTCL_THREADS=1 -DUSE_ITCL_STUBS -DUSE_TCL_STUBS -DUSE_TK_STUBS -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tcl.framework/Headers -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tk.framework/Headers -I/Users/morrison/Applications/include -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic -I/Users/morrison/brlcad.extbuild/.build/src/other/ext/ITK_BLD-prefix/src/ITK_BLD-build/include -I/opt/X11/include -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/generic -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../tcl/unix -I/Users/morrison/brlcad.extbuild/src/other/ext/itk3/../itcl3/generic -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk -mmacosx-version-min=10.14 -o CMakeFiles/itkstub.dir/generic/itkStubLib.c.o -c /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itkStubLib.c:31:
In file included from /Users/morrison/brlcad.extbuild/src/other/ext/itk3/generic/itk.h:79:
/Users/morrison/Applications/include/itclInt.h:58:10: fatal error: 'common.h' file not found
fwiw, this was the cmd: rm -rf .build && mkdir .build && cd .build && cmake .. && make
starseeker (Dec 08 2020 at 14:23):Interesting. It's finding an Itcl installation in your Applications directory, which must be one of our installs since it's trying to include common.h. That must be an old one though, since neither trunk nor extbuild itcl3 has any common.h includes...
starseeker (Dec 08 2020 at 14:31):Did you do a "system" install of Itcl3 to avoid having to rebuild it for the cad builds? If so, I'd suggest trying a "vanilla" Itcl3 version without the common.h include as a system install and seeing if that helps.
Sean (Dec 08 2020 at 15:13):I can't dig into it today (deadline), but I'll take a look tomorrow. There is a full install of brlcad in ~/Applications from 2016 that's occasionally used for testing. There isn't an itcl3 just for itcl3's sake.
starseeker (Dec 08 2020 at 15:15):Ah, that makes sense. The trick will be how to ignore it then. No hurry - I've got a deadline too.
Sean (Dec 08 2020 at 15:36):The question is also how's it picking it up now while it wasn't before (afaik) and isn't on trunk.
Sean (Dec 08 2020 at 15:36):their configure script?
starseeker (Dec 08 2020 at 15:37):Trunk may be picking it up and we just aren't noticing - if itk is built with an include dir that includes common.h, we'd probably never see it.
starseeker (Dec 08 2020 at 15:37):IIRC there were a couple issues like that I ran into with the extbuild work when builds actually became fully independent...
starseeker (Dec 08 2020 at 15:37):FindITCL is probably locating it.
Sean (Dec 08 2020 at 15:39):I can check that tomorrow, but there's a slew of other stuff in ~/Applcations/include -- that's my own personal system installation directory, might as well be /usr/local
Sean (Dec 08 2020 at 15:41):it's not specified anywhere that I know of for it to find it -- the only possible linkage I can think of is that the compiler is coming from there but again, that's been true for years and across branches too. did I mention that was a 2016 install of BRL-CAD?
starseeker (Dec 08 2020 at 15:43):Yes - the difference would be in the itk build logic and (possibly) what the Find packages in extbuild are identifying. If the trunk itk build includes a common.h directory we'd never notice any issues, and the newer Find modules may be better at locating things on the system.
Last updated: Oct 09 2024 at 00:44 UTC