Raghav Sharma (Feb 24 2026 at 15:38): Raghav Sharma (Feb 24 2026 at 15:38):Hey folks
I've been playing around with BRL-CAD lately on my setup and started building something I thought you guys might find interesting.
I have put together a working local prototype of an MCP (Model Context Protocol) server for BRL-CAD. Right now, it uses a non-blocking Tcl socket bridge to pipe geometry commands into a live MGED session. I hooked it up to an LLM via LangGraph, so I can literally just type something like "make a 15mm cylinder and subtract it from that sphere," and the BRL-CAD GUI updates live with the boolean math. (Right now I've been playing with boolean operations and spheres, but could definitely index more tools with time).
I know one of the long-term goals for BRL-CAD is improving the UX and lowering the learning curve. Letting users interact with the CSG engine using natural language feels like a really fun way to tackle that. Also, I noticed that tools like FreeCAD and OpenSCAD already have active MCP integrations out there, but BRL-CAD doesn't seem to have one yet. It feels like a missed opportunity to get BRL-CAD plugged into the new AI agent ecosystem.
I'm really interested in applying for GSoC this year. I know an MCP integration isn't currently listed on the official problem statements, but would the mentors be open to considering a proposal around this? My goal for the summer would be to turn this Tcl socket prototype into a production-grade interface, perhaps integrated with the main GUI as a model-agnostic MCP interface with a BYOK system.
WhatsApp Video 2026-02-24 at 20.44.18.mp4
Sean (Feb 24 2026 at 20:27):Hi @Raghav Sharma and thanks for the outstanding introduction. Short answer is yes.
Longer answer is that is something that I’ve been actively working and thinking about, albeit with manual scaffolding to explore how practical and effective it can be. So yes you should definitely submit your ideas and we should continue to discuss. The potential is tremendous for UX but also as a means for discovery and productivity. The details of how and what skills/tools/features and data integrity will need to be developed as well.
Love custom ideas like this that align.
Raghav Sharma (Feb 24 2026 at 20:32):Aha I'm glad that's the case @Sean
I'd love to discuss the tools and processes with a mentor while I continue to build upon this scaffold and before I start drafting a proposal. Things like integrations with the mged CLI/GUI window, memory persistence and a priority list for tool integration. Would it be possible to perhaps have a call some time?
Raghav Sharma (Feb 27 2026 at 19:58):I've been building upon this in the meanwhile. I have added support for Sphere, Cylinder and Box creation along with basic boolean operations
I have also added a soft fallback to the user in case of missing parameters (If i ask the AI to make a cylinder of radius 5, without specifying the height or the location, it would give me a message asking me to specify the missing parameters before sending it to the socket bridge)
image.png
Raghav Sharma (Feb 27 2026 at 20:01):
I am also handling command formatting in the MCP layer itself so that there are no weird hallucinated commands going to the editor. (though I need to add some sort of try-catch validation here for invalid values coming from the LLM before they hit the editor, or to evaluate errors raised by the editor after the command goes through)
Sean (Mar 01 2026 at 22:23):@Raghav Sharma I would love to set up a call to talk in more depth. We have held open meetings with all contributors before and may do that again this year as well. It really though often boils down to motivation, having a plan, and determination. If you’re excited about what you’re doing others often will be excited also.
Sean (Mar 01 2026 at 22:25):For mcp integration, I wonder if you could make a tool robust enough to discover how it needs to issue commands, maybe by querying the help system or running commands like “in” interactively where it describes each.
Sean (Mar 01 2026 at 22:25):(As a means for mining hallucinations also)
Sean (Mar 01 2026 at 22:26):Having an error handler sounds like a great idea regardless so it can hopefully recover given expected situations.
Raghav Sharma (Mar 02 2026 at 00:07):Sean said:
Raghav Sharma I would love to set up a call to talk in more depth. We have held open meetings with all contributors before and may do that again this year as well. It really though often boils down to motivation, having a plan, and determination. If you’re excited about what you’re doing others often will be excited also.
@Sean That sounds great! I really am excited to build this and I am in the process of drafting the technical architecture for this. Please let me know whenever you'd be available for it, I'd love to hop on some time to discuss the details and make sure it aligns with BRLCAD's goals.
Raghav Sharma (Mar 02 2026 at 00:17):Sean said:
For mcp integration, I wonder if you could make a tool robust enough to discover how it needs to issue commands, maybe by querying the help system or running commands like “in” interactively where it describes each.
Ah yes I was thinking about this too.
I was initially thinking of building a standard MCP for this (ones the likes of slack or postgres currently use, with static defined tools) but something like that might not work with BRLCAD as it has a large number (around 400+) commands that would have to be indexed that way (each having different parameters too) which would be a pain to index manually and would absolutely blow up the context window of the LLMs
So I was in the process of redefining it as a Dynamic MCP with a discovery and error handling API to actively fetch context from terminal responses for tool selection, query formatting and errors being thrown.
MCPs with requirements like this rely on a ReAct (Reason + Act) sequence which is actually pretty achievable (with our Agent ingesting man pages on the fly to understand commands in runtime, or using ls to query existing elements in the database). I'll try to get a prototype of this approach before our call.
Raghav Sharma (Mar 02 2026 at 00:24):We could also do some visual self-correction stuff with VLMs (Visual LLM Models) too, with the Agent actually getting visual context. The agent could just run rt commands to output PNGs in different camera angles to visually verify things like boolean ops and then self-correct the coordinates if needed
Though it would be more of an experimental thing we could try after building the base MCP
Raghav Sharma (Mar 04 2026 at 16:05):
successfully built a prototype where the agent finds the commands it needs to execute via a resource sheet and can automatically query man pages for their usage
The basic flow is:
the agent first checks the query, goes through a bunch of predefined tools (that I had to define for some tougher syntactic or sequential operations like boolean ops)
Then if the agent does not find them in predefined tools, it loads up a resource (imagine a cheatsheet) with a list of available commands with 1 liner descriptions
if the agent sees something relevant, it opens up the man page for that command, learns how to execute it, and then runs it in the CLI
If there is any error, it goes back to the agent via the socket, the agent reads the error and then corrects the method to run the operation (Done in an incremental loop upto a set number of executions, in this case the limit is set to 5, just so we don't run into a recursive infinite loop)
(In this case, an error occured, with the LLM seeing that a sphere of the name sphere.s already exists, so it went back and changed the name and re-executed the command)
Raghav Sharma (Mar 04 2026 at 16:06):Its pretty rough around the edges right now, but I could certainly refine it across the GSoC developement period
Raghav Sharma (Mar 04 2026 at 16:33):
You can also ask about any commands
It will automatically query the help pages and tell you about it
Raghav Sharma (Mar 04 2026 at 17:01):
An example where the agent finds the necessary command, reads its documentation, and executes it autonomously while also answering a user question about the file location
Sean (Mar 09 2026 at 03:55):@Raghav Sharma Definitely showing some promise in a quick span of time. What would be good to articulate in your proposal is just how far would you plan on taking it? What sorts of user stories do you envision achieving? What about possible RAGification of the tutorials or curated examples of the mcp tools being defined so it has various patterns to leverage? Would it be able to be an instructive agent that helps teach as it works? What sort of and how much base knowledge would be good to have defined in advance? What about making it work offline usably well enough with a local model? Or via some online service, but without consuming credits? Just a few questions like that to think about and help scope what it would and would not attempt to achieve under GSoC. Obviously can't do everything but could probably get a lot done at the pace you're demonstrating. Think about how to make it deployable to users. Think about what tech stack changes are assumed. Think about multiple platforms. Think about maintainability/extensibility, etc.
Володимир (Mar 09 2026 at 11:23):hello! i cant install brlcad on arch linux, i cant find PKGBUILD
Raghav Sharma (Mar 21 2026 at 13:30):Hey @Sean ,
Thank you so much for the detailed feedback. The questions you raised around RAG, offline usability, user stories, and deployability have been incredibly helpful for shaping the scope of the proposal. I've taken all of it into account while drafting.
I've put together an initial proposal draft here:
https://docs.google.com/document/d/174vokenqUojAx9taqwMRZwkrVz4QGqr8Ma2u15KU5EY/edit?usp=sharing
Would really appreciate it if you could give it a read and let me know if the direction and scope feel aligned with BRL-CAD's goals, and if there's anything I should expand on or cut down before submission. Also while submitting it on the GSoC portal, should I put it as a medium or large project?
Sean (Mar 22 2026 at 14:59):@Raghav Sharma This looks good, but obviously heavily planned out by AI which isn't a problem per se but calls into question several aspects that need to be considered. One is the scope is very broad and extensive. While this may be possible, there is lots of baked in assumptions about how things are going to work. For example, I'm unclear on the uplevel tcl injections being feasible or even functional across all command modalities. That said, it's an implementation detail that didn't have to be mentioned and the actual result could be discovered. The risk is that the summer ends with a broad but shallow prototype. A stronger proposal should be described incrementally or narrow success criteria sharply. That doesn't mean simply proposing less, it's about the software engineering being intentional and having room to expect integration problems, bugs, development iterations, etc.
What I would like to see is a few examples in your own words on what types of interesting prompts that are BRL-CAD-specific are going to be the goal. You mention golden tools, but that's so general and vague that it leaves much to the imagination. Basically, show a few specific user stories of something they're attempting to achieve -- and not discuss the solution or the "how" but identify the "what" as a target set of examples. The paper goes into great detail on how but very little is specific to BRL-CAD and what it might mean for users and workflows. If this is integration in MGED, what is the specific nature of the integration (from a user perspective) -- i.e., where is this cursor agent going to reside? Another panel? A command? A menu tool?
The weakest use case is the ambitious but general “generate/manipulate/analyze complex 3D geometry from natural language” promise. That is where ambiguity, hallucination, partial failure, and semantic mismatch are high risk. Your proposal leans heavily on that vision without evidence or measurement activity baked in to ensure it'll be or can be made trustworthy within scope/time.
Architecturally, a big one is you're proposing introducing Python as a new requirement to BRL-CAD. That's a major change and needs to be discussed and justified. How's that integration going to look from a software engineering perspective? Are you going to address the necessary changes to "bext" to support it? You mention json parsing several times -- what's doing that parsing and how is that dependency managed? I'll note we have a couple options already, but you don't mention them.
Lastly, would be very helpful to hear your thoughts (not AIs) on MGED vs Archer vs Arbalest vs future Qt deployments. You can't and shouldn't target them all, but should have done enough research to be aware of them and how your work will transition into a bigger picture or be reusable as we continue to evolve our UX.
Raghav Sharma (Mar 22 2026 at 15:51):Hi @Sean
Yes some of the details were based on assumptions and some were a result of experimentations I had experienced while working on the Tcl socket based prototype. I didn't run into any problems while building it using Tcl uplevel injections, but I wasn't aware that there would be any functional limitations with modalities, maybe we could explore some other way to interface the MCP to it, perhaps a C++ socket, closer to the geometry engine
I'll also update the proposal to be a bit more user centric on the features.
Also if the "generate/manipulate/analyse complex 3D geometry" thing seems a little too ambitious for this summer, would a more analytics/ repetition/ database analysis tool (eg: finding objects with unassigned materials, or batch automation of some repetitive commands) be aligned with the organisation? It would be a lot less prone to hallucinate and less risky, though I'd need some user input from people using the software quite often to actually know what parts they'd need automated. I was also thinking of having transactional actions for this to reduce the risk of partial failure (Someone in GSoC 2021 had apparently implemented transactions and undo features in BRL-CAD but the wiki seemed to be down so I couldn't read about it in depth)
For the python thing, I wasn't planning on having the MCP coming strapped in along with the BRL-CAD codebase (as mentioned in the proposal). Instead I thought of it as a decoupled extension the user could optionally clone and integrate locally with their own agent and use with BRL-CAD. So the parsing stuff would actually be stuff managed between the agent and the MCP. Because Python handles the JSON, BRL-CAD doesn't need a new C-dependency.
For the MGED vs Archer vs Arbalest vs future Qt stuff I had read about them and the direction BRL-CAD is going in with MOOSE and Qt. For this summer I had initially planned for going with Tcl uplevel injections (compatible with MGED and Archer) as they were fairly straight forward places to test the core AI logic and database interactions. As the heavy lifting for the MCP is done in the decoupled Python extension for now (the LLM routing, tool discovery, error recovery etc) and that architecture would inherently stay future proof. When MOOSE does become mature though we'd probably need to switch away from the Tcl thing.
Now that I think about it, a C++ socket listener hooking in closer to the geometry engine would be a better idea than Tcl overall as it would be even more future proof. I'll do some reading into this, thanks a lot for the feedback again Sean
Raghav Sharma (Mar 23 2026 at 16:39):Hi @Sean
Just a follow up, I explored how commands could be piped through both MOOSE and the current MGED/archer exec pipelines and did some more reading into how I could use existing libraries and make this more future proof.
I am thinking of going for a C++ based socket for piping in commands so its more future proof for MOOSE (and has minimal friction to port once it matures)
I am thinking of having a C++ socket that receives strings coming in from the Python FastMCP server (flat commands like 'in sph1 sph 0 0 0 5' with no json). This could be caught by the C++ listener and then broken down by bu_argv_from_string from libbu to break it into argc/argv strings safely
This above part can be the same for both MGED/Archer and the upcoming arbalest as both of them have the ability to run argc/argv commands for backwards compatibility with the rest of the BRLCAD libraries. (for example if i am running a libged command, in MGED/Archer it could be using ged_exec and in MOOSE it could be using CommandString::Parse)
To avoid complex mutex locking and race conditions, the socket itself could be watched by the main thread's event loop. For Phase 1 (MGED/Archer), this means registering the socket file descriptor with Tcl_CreateFileHandler. For Arbalest/MOOSE later, some one can just swap that Tcl hook for a Qt event loop hook. (This synchronous approach feels a lot safer than attempting background thread execution, which would require heavy file-locking, though I'm not sure if there would be any limitations with any particular command modalities in this case as there was with Tcl uplevel injections so please let me know if there are).
The pipeline would be:
Event Loop Trigger -> Read Socket -> Parse (libbu) -> Execute
Also on a side note, I think that proposing a C++ socket connection that could actually enable the user to safely pipe commands over an exposed port could be very impactful even without an MCP directly attached in the particular proposal, as it could actually lead to users building many different web based or local extensions to BRL-CAD that could be able to execute commands safely. While building the MCP I kind of wished there was one, where I could just run a script and be able to send commands to the software via a port. Would it be a good idea to propose that socket connection itself as a proposal considering the future possibilities in extensions/plugins and MCPs it could unlock?
I remember neovim had a bunch of similar projects last year, focusing more on building the functionality to support more AI integrations for the future in a couple of projects (especially the one that had built primitives to streamline AI plugin development)
https://summerofcode.withgoogle.com/archive/2025/organizations/neovim
I am thinking of making the initial decoupled MCP extension more specific with lighter automation commands, but more focus towards actually building an extension agnostic C++ socket pipeline to run those commands. It would be focused more building the infrastructure required to support future extensions and MCPs in BRLCAD and demonstrating its capabilities with a lighter more specific MCP.
Would that be a better direction for the proposal?
Raghav Sharma (Mar 29 2026 at 19:20):https://docs.google.com/document/d/1_0uMLfDUvGBrjnULRvagHpuQczvLZdDslLsSWsLENVI/edit?usp=sharing
@Sean I had gone back and made some architectural changes along with the inclusion of more relevant user stories. The focus being more on the establishment of a secure pipeline to support future extensions for BRL-CAD via a C++ socket API along with the MCP Server.
Would really appreciate it if you could give it a read and let me know if the direction and scope feel aligned with BRL-CAD's goals
Sean (Mar 30 2026 at 05:46):@Raghav Sharma just a caution, a tcp/ip socket is critically problematic for our corporate uses as it will get blocked by security policies or trigger anti-virus flagging or simply be unavailable. pipes (stdio) are typically okay, along with other non-ip based IPC.
That's not to say a tcp socket can't be made to work, but then you'll definitely need to carve out a couple weeks to ensure you're registered correctly (cross-platform) and that it's not triggered/blocked, etc. It may be sufficient to target configuration settings that avoid the potential issues (like limiting to ports >= 1024 if that's enough, don't know).
Raghav Sharma (Mar 30 2026 at 15:13):Ahh that is valid @Sean
I will update the proposal to use a stdio based approach for the primary transport instead of TCP sockets. Perhaps we could experiment with the target configuration settings for TCP later on after the stdio thing is done.
Thanks for the heads up!
Raghav Sharma (May 01 2026 at 03:42):@Sean Thanks a lot for considering this proposal into GSOC'26. I am very excited to work on this.
Could you please let me know when we could perhaps have a meeting and discuss a plan for this summer? I really look forward to working on this
Sean (May 01 2026 at 16:37):@Raghav Sharma we’re excited to be working with you and our other contributors also! There was a lot of tough decisions made this GSoC and a couple more we would have accepted if we had been able to get more slots. You however did and continue to do a great job with community engagement and communication so please do keep it up!
Lots to discuss but we have a little bit of time to get set up. As far meeting synchronous, just need to touch base with the other mentors but likely not in the next couple days, sometime next week/weekend possibly unless the chips align quickly beforehand. Will get back to you on that.
Raghav Sharma (May 01 2026 at 22:52):Sure thing! Really looking forward to it.
Raghav Sharma (May 04 2026 at 10:03):Hey @Sean ,
While doing some dives into the stdio pipeline and libbu for Phase 1, I caught a potential architectural roadblock regarding cross-platform IPC that I wanted to get your thoughts on.
While the POSIX implementation for Linux is straightforward, it looks like Windows handles anonymous pipes very differently. Specifically, it seems the Windows implementation of Tcl's event loop could struggle to monitor standard anonymous pipes natively without blocking. (Seems that Tcl_CreateFileHandler is only supported on Unix. this could be from how windows handles system calls differently from POSIX. Windows is basically blind to pipe data but can receive via TCP)
To ensure I hit the Week 3 and 4 milestones reliably, would you be open to me strictly targeting the Linux/POSIX pipeline first to prove the core C++ listener and the event loop hook? once the architecture is proven stable on Linux, we could use the buffer weeks to explore the Windows port (perhaps using libbu macros or revisiting a local-only TCP configuration if corporate security allows port > 1024).
Or would it be a better idea to start with local-only TCP configuration with some experimental validation on the allowed port numbers? If that does end up being viable for corporate users we could have a fairly reliable crossplatform solution
Please let me know which approach would make more sense
Sean (May 05 2026 at 04:04):@Raghav Sharma there was some new cross-platform IPC API added to libbu just a few days ago. I believe it's in a branch, but potentially helpful for what you described.
Sean (May 05 2026 at 04:05):I think IPC is definitely the way to go and preferred over TCP since that will trigger security watch tools.
Sean (May 05 2026 at 04:07):look for a branch related to remrt/fbserv refactoring to see the interface and how it was wired into existing tools. that should give a good direction for how to integrate for an mcp server (or editor gui).
Raghav Sharma (May 05 2026 at 06:11):Found it
it seems to be on the rtwizard branch, commit 072efe3. Just going through pkg_transport.cpp right now. Thank you for the direction!
Is the rtwizard branch intended to merge to main before GSoC starts, or will I be developing against it directly? Also would the BRL-CAD-side IPC wiring for the MCP server target main or a feature branch during the summer?
starseeker (May 06 2026 at 11:25):I merged the rtwizard work into main a couple days back. The IPC part is primarily in libpkg, rather than libbu.
Raghav Sharma (May 07 2026 at 05:11):Hi again
so while digging into the spawn architecture for the MCP listener I had a question about bu_process_create.
My use case needs the Python subprocess to inherit the pkg pipe fds from pkg_pair() but I noticed the child process has a close loop for (int i = 3; i < 20; i++) close(i) before execvp, and on Windows it uses STARTUPINFO rather than STARTUPINFOEX with PROC_THREAD_ATTRIBUTE_HANDLE_LIST.
Would those prevent the pkg fds from surviving into the Python process? Or is there an existing pattern in the codebase I should be following for this kind of fd inheritance? Trying to figure out whether I need a custom spawn wrapper scoped to the MCP extension or if there's already a clean way to handle this.
Raghav Sharma (May 07 2026 at 05:19):existing uses of bu_process_create in the codebase communicate via the stdio redirects (fd 0/1/2), so the close loop is fine for them. My case is different since I need extra pkg fds to survive beyond stdio as I want my child process to be talking to both the LLM client (over stdio) and then BRL-CAD (an additional pkg pair)
Raghav Sharma (May 08 2026 at 02:26):@Sean @starseeker
would there be any sort of unix domains/named pipes solution that I could go for? libpkg in the current state doesn't seem to support them. Would that be a viable option to go for to build the listener?
There seems to be an issue with anonymous pipes right now. I could start an MCP server with BRL-CAD as a child process and that child could have bidirection communication with BRL-CAD (and an LLM client, considering we solve the problem with an additional fd mentioned above)
The problem would be that VS code or Cursor or any other existing IDE clients would not be able to connect to the application (connections to existing MCP servers is an HTTP only thing, and we're avoiding TCP now)
So the architecture would flip a little bit to accommodate this, by having the LLM client start the MCP and they talk to eachother via stdio (which is fairly standard).
Now this MCP server would need an address to talk to my BRL-CAD listener, and that could be possible via named pipes or Unix Domains (I could essentially pass it to my MCP as an API endpoint and they could communicate)
I can't seem to find anything in libpkg for named pipes or unix pipes but there does seem to be implementations with some TCP stuff going on.
Would there be any existing solution within the libraries for this sort of peer-to-peer communication without TCP
Last updated: May 14 2026 at 02:10 UTC