Raghav Sharma (Feb 24 2026 at 15:38): Raghav Sharma (Feb 24 2026 at 15:38):Hey folks
I've been playing around with BRL-CAD lately on my setup and started building something I thought you guys might find interesting.
I have put together a working local prototype of an MCP (Model Context Protocol) server for BRL-CAD. Right now, it uses a non-blocking Tcl socket bridge to pipe geometry commands into a live MGED session. I hooked it up to an LLM via LangGraph, so I can literally just type something like "make a 15mm cylinder and subtract it from that sphere," and the BRL-CAD GUI updates live with the boolean math. (Right now I've been playing with boolean operations and spheres, but could definitely index more tools with time).
I know one of the long-term goals for BRL-CAD is improving the UX and lowering the learning curve. Letting users interact with the CSG engine using natural language feels like a really fun way to tackle that. Also, I noticed that tools like FreeCAD and OpenSCAD already have active MCP integrations out there, but BRL-CAD doesn't seem to have one yet. It feels like a missed opportunity to get BRL-CAD plugged into the new AI agent ecosystem.
I'm really interested in applying for GSoC this year. I know an MCP integration isn't currently listed on the official problem statements, but would the mentors be open to considering a proposal around this? My goal for the summer would be to turn this Tcl socket prototype into a production-grade interface, perhaps integrated with the main GUI as a model-agnostic MCP interface with a BYOK system.
WhatsApp Video 2026-02-24 at 20.44.18.mp4
Sean (Feb 24 2026 at 20:27):Hi @Raghav Sharma and thanks for the outstanding introduction. Short answer is yes.
Longer answer is that is something that I’ve been actively working and thinking about, albeit with manual scaffolding to explore how practical and effective it can be. So yes you should definitely submit your ideas and we should continue to discuss. The potential is tremendous for UX but also as a means for discovery and productivity. The details of how and what skills/tools/features and data integrity will need to be developed as well.
Love custom ideas like this that align.
Raghav Sharma (Feb 24 2026 at 20:32):Aha I'm glad that's the case @Sean
I'd love to discuss the tools and processes with a mentor while I continue to build upon this scaffold and before I start drafting a proposal. Things like integrations with the mged CLI/GUI window, memory persistence and a priority list for tool integration. Would it be possible to perhaps have a call some time?
Raghav Sharma (Feb 27 2026 at 19:58):I've been building upon this in the meanwhile. I have added support for Sphere, Cylinder and Box creation along with basic boolean operations
I have also added a soft fallback to the user in case of missing parameters (If i ask the AI to make a cylinder of radius 5, without specifying the height or the location, it would give me a message asking me to specify the missing parameters before sending it to the socket bridge)
image.png
Raghav Sharma (Feb 27 2026 at 20:01):
I am also handling command formatting in the MCP layer itself so that there are no weird hallucinated commands going to the editor. (though I need to add some sort of try-catch validation here for invalid values coming from the LLM before they hit the editor, or to evaluate errors raised by the editor after the command goes through)
Sean (Mar 01 2026 at 22:23):@Raghav Sharma I would love to set up a call to talk in more depth. We have held open meetings with all contributors before and may do that again this year as well. It really though often boils down to motivation, having a plan, and determination. If you’re excited about what you’re doing others often will be excited also.
Sean (Mar 01 2026 at 22:25):For mcp integration, I wonder if you could make a tool robust enough to discover how it needs to issue commands, maybe by querying the help system or running commands like “in” interactively where it describes each.
Sean (Mar 01 2026 at 22:25):(As a means for mining hallucinations also)
Sean (Mar 01 2026 at 22:26):Having an error handler sounds like a great idea regardless so it can hopefully recover given expected situations.
Raghav Sharma (Mar 02 2026 at 00:07):Sean said:
Raghav Sharma I would love to set up a call to talk in more depth. We have held open meetings with all contributors before and may do that again this year as well. It really though often boils down to motivation, having a plan, and determination. If you’re excited about what you’re doing others often will be excited also.
@Sean That sounds great! I really am excited to build this and I am in the process of drafting the technical architecture for this. Please let me know whenever you'd be available for it, I'd love to hop on some time to discuss the details and make sure it aligns with BRLCAD's goals.
Raghav Sharma (Mar 02 2026 at 00:17):Sean said:
For mcp integration, I wonder if you could make a tool robust enough to discover how it needs to issue commands, maybe by querying the help system or running commands like “in” interactively where it describes each.
Ah yes I was thinking about this too.
I was initially thinking of building a standard MCP for this (ones the likes of slack or postgres currently use, with static defined tools) but something like that might not work with BRLCAD as it has a large number (around 400+) commands that would have to be indexed that way (each having different parameters too) which would be a pain to index manually and would absolutely blow up the context window of the LLMs
So I was in the process of redefining it as a Dynamic MCP with a discovery and error handling API to actively fetch context from terminal responses for tool selection, query formatting and errors being thrown.
MCPs with requirements like this rely on a ReAct (Reason + Act) sequence which is actually pretty achievable (with our Agent ingesting man pages on the fly to understand commands in runtime, or using ls to query existing elements in the database). I'll try to get a prototype of this approach before our call.
Raghav Sharma (Mar 02 2026 at 00:24):We could also do some visual self-correction stuff with VLMs (Visual LLM Models) too, with the Agent actually getting visual context. The agent could just run rt commands to output PNGs in different camera angles to visually verify things like boolean ops and then self-correct the coordinates if needed
Though it would be more of an experimental thing we could try after building the base MCP
Raghav Sharma (Mar 04 2026 at 16:05):
successfully built a prototype where the agent finds the commands it needs to execute via a resource sheet and can automatically query man pages for their usage
The basic flow is:
the agent first checks the query, goes through a bunch of predefined tools (that I had to define for some tougher syntactic or sequential operations like boolean ops)
Then if the agent does not find them in predefined tools, it loads up a resource (imagine a cheatsheet) with a list of available commands with 1 liner descriptions
if the agent sees something relevant, it opens up the man page for that command, learns how to execute it, and then runs it in the CLI
If there is any error, it goes back to the agent via the socket, the agent reads the error and then corrects the method to run the operation (Done in an incremental loop upto a set number of executions, in this case the limit is set to 5, just so we don't run into a recursive infinite loop)
(In this case, an error occured, with the LLM seeing that a sphere of the name sphere.s already exists, so it went back and changed the name and re-executed the command)
Raghav Sharma (Mar 04 2026 at 16:06):Its pretty rough around the edges right now, but I could certainly refine it across the GSoC developement period
Raghav Sharma (Mar 04 2026 at 16:33):
You can also ask about any commands
It will automatically query the help pages and tell you about it
Raghav Sharma (Mar 04 2026 at 17:01):
An example where the agent finds the necessary command, reads its documentation, and executes it autonomously while also answering a user question about the file location
Sean (Mar 09 2026 at 03:55):@Raghav Sharma Definitely showing some promise in a quick span of time. What would be good to articulate in your proposal is just how far would you plan on taking it? What sorts of user stories do you envision achieving? What about possible RAGification of the tutorials or curated examples of the mcp tools being defined so it has various patterns to leverage? Would it be able to be an instructive agent that helps teach as it works? What sort of and how much base knowledge would be good to have defined in advance? What about making it work offline usably well enough with a local model? Or via some online service, but without consuming credits? Just a few questions like that to think about and help scope what it would and would not attempt to achieve under GSoC. Obviously can't do everything but could probably get a lot done at the pace you're demonstrating. Think about how to make it deployable to users. Think about what tech stack changes are assumed. Think about multiple platforms. Think about maintainability/extensibility, etc.
Володимир (Mar 09 2026 at 11:23):hello! i cant install brlcad on arch linux, i cant find PKGBUILD
Raghav Sharma (Mar 21 2026 at 13:30):Hey @Sean ,
Thank you so much for the detailed feedback. The questions you raised around RAG, offline usability, user stories, and deployability have been incredibly helpful for shaping the scope of the proposal. I've taken all of it into account while drafting.
I've put together an initial proposal draft here:
https://docs.google.com/document/d/174vokenqUojAx9taqwMRZwkrVz4QGqr8Ma2u15KU5EY/edit?usp=sharing
Would really appreciate it if you could give it a read and let me know if the direction and scope feel aligned with BRL-CAD's goals, and if there's anything I should expand on or cut down before submission. Also while submitting it on the GSoC portal, should I put it as a medium or large project?
Sean (Mar 22 2026 at 14:59):@Raghav Sharma This looks good, but obviously heavily planned out by AI which isn't a problem per se but calls into question several aspects that need to be considered. One is the scope is very broad and extensive. While this may be possible, there is lots of baked in assumptions about how things are going to work. For example, I'm unclear on the uplevel tcl injections being feasible or even functional across all command modalities. That said, it's an implementation detail that didn't have to be mentioned and the actual result could be discovered. The risk is that the summer ends with a broad but shallow prototype. A stronger proposal should be described incrementally or narrow success criteria sharply. That doesn't mean simply proposing less, it's about the software engineering being intentional and having room to expect integration problems, bugs, development iterations, etc.
What I would like to see is a few examples in your own words on what types of interesting prompts that are BRL-CAD-specific are going to be the goal. You mention golden tools, but that's so general and vague that it leaves much to the imagination. Basically, show a few specific user stories of something they're attempting to achieve -- and not discuss the solution or the "how" but identify the "what" as a target set of examples. The paper goes into great detail on how but very little is specific to BRL-CAD and what it might mean for users and workflows. If this is integration in MGED, what is the specific nature of the integration (from a user perspective) -- i.e., where is this cursor agent going to reside? Another panel? A command? A menu tool?
The weakest use case is the ambitious but general “generate/manipulate/analyze complex 3D geometry from natural language” promise. That is where ambiguity, hallucination, partial failure, and semantic mismatch are high risk. Your proposal leans heavily on that vision without evidence or measurement activity baked in to ensure it'll be or can be made trustworthy within scope/time.
Architecturally, a big one is you're proposing introducing Python as a new requirement to BRL-CAD. That's a major change and needs to be discussed and justified. How's that integration going to look from a software engineering perspective? Are you going to address the necessary changes to "bext" to support it? You mention json parsing several times -- what's doing that parsing and how is that dependency managed? I'll note we have a couple options already, but you don't mention them.
Lastly, would be very helpful to hear your thoughts (not AIs) on MGED vs Archer vs Arbalest vs future Qt deployments. You can't and shouldn't target them all, but should have done enough research to be aware of them and how your work will transition into a bigger picture or be reusable as we continue to evolve our UX.
Raghav Sharma (Mar 22 2026 at 15:51):Hi @Sean
Yes some of the details were based on assumptions and some were a result of experimentations I had experienced while working on the Tcl socket based prototype. I didn't run into any problems while building it using Tcl uplevel injections, but I wasn't aware that there would be any functional limitations with modalities, maybe we could explore some other way to interface the MCP to it, perhaps a C++ socket, closer to the geometry engine
I'll also update the proposal to be a bit more user centric on the features.
Also if the "generate/manipulate/analyse complex 3D geometry" thing seems a little too ambitious for this summer, would a more analytics/ repetition/ database analysis tool (eg: finding objects with unassigned materials, or batch automation of some repetitive commands) be aligned with the organisation? It would be a lot less prone to hallucinate and less risky, though I'd need some user input from people using the software quite often to actually know what parts they'd need automated. I was also thinking of having transactional actions for this to reduce the risk of partial failure (Someone in GSoC 2021 had apparently implemented transactions and undo features in BRL-CAD but the wiki seemed to be down so I couldn't read about it in depth)
For the python thing, I wasn't planning on having the MCP coming strapped in along with the BRL-CAD codebase (as mentioned in the proposal). Instead I thought of it as a decoupled extension the user could optionally clone and integrate locally with their own agent and use with BRL-CAD. So the parsing stuff would actually be stuff managed between the agent and the MCP. Because Python handles the JSON, BRL-CAD doesn't need a new C-dependency.
For the MGED vs Archer vs Arbalest vs future Qt stuff I had read about them and the direction BRL-CAD is going in with MOOSE and Qt. For this summer I had initially planned for going with Tcl uplevel injections (compatible with MGED and Archer) as they were fairly straight forward places to test the core AI logic and database interactions. As the heavy lifting for the MCP is done in the decoupled Python extension for now (the LLM routing, tool discovery, error recovery etc) and that architecture would inherently stay future proof. When MOOSE does become mature though we'd probably need to switch away from the Tcl thing.
Now that I think about it, a C++ socket listener hooking in closer to the geometry engine would be a better idea than Tcl overall as it would be even more future proof. I'll do some reading into this, thanks a lot for the feedback again Sean
Raghav Sharma (Mar 23 2026 at 16:39):Hi @Sean
Just a follow up, I explored how commands could be piped through both MOOSE and the current MGED/archer exec pipelines and did some more reading into how I could use existing libraries and make this more future proof.
I am thinking of going for a C++ based socket for piping in commands so its more future proof for MOOSE (and has minimal friction to port once it matures)
I am thinking of having a C++ socket that receives strings coming in from the Python FastMCP server (flat commands like 'in sph1 sph 0 0 0 5' with no json). This could be caught by the C++ listener and then broken down by bu_argv_from_string from libbu to break it into argc/argv strings safely
This above part can be the same for both MGED/Archer and the upcoming arbalest as both of them have the ability to run argc/argv commands for backwards compatibility with the rest of the BRLCAD libraries. (for example if i am running a libged command, in MGED/Archer it could be using ged_exec and in MOOSE it could be using CommandString::Parse)
To avoid complex mutex locking and race conditions, the socket itself could be watched by the main thread's event loop. For Phase 1 (MGED/Archer), this means registering the socket file descriptor with Tcl_CreateFileHandler. For Arbalest/MOOSE later, some one can just swap that Tcl hook for a Qt event loop hook. (This synchronous approach feels a lot safer than attempting background thread execution, which would require heavy file-locking, though I'm not sure if there would be any limitations with any particular command modalities in this case as there was with Tcl uplevel injections so please let me know if there are).
The pipeline would be:
Event Loop Trigger -> Read Socket -> Parse (libbu) -> Execute
Also on a side note, I think that proposing a C++ socket connection that could actually enable the user to safely pipe commands over an exposed port could be very impactful even without an MCP directly attached in the particular proposal, as it could actually lead to users building many different web based or local extensions to BRL-CAD that could be able to execute commands safely. While building the MCP I kind of wished there was one, where I could just run a script and be able to send commands to the software via a port. Would it be a good idea to propose that socket connection itself as a proposal considering the future possibilities in extensions/plugins and MCPs it could unlock?
I remember neovim had a bunch of similar projects last year, focusing more on building the functionality to support more AI integrations for the future in a couple of projects (especially the one that had built primitives to streamline AI plugin development)
https://summerofcode.withgoogle.com/archive/2025/organizations/neovim
I am thinking of making the initial decoupled MCP extension more specific with lighter automation commands, but more focus towards actually building an extension agnostic C++ socket pipeline to run those commands. It would be focused more building the infrastructure required to support future extensions and MCPs in BRLCAD and demonstrating its capabilities with a lighter more specific MCP.
Would that be a better direction for the proposal?
Raghav Sharma (Mar 29 2026 at 19:20):https://docs.google.com/document/d/1_0uMLfDUvGBrjnULRvagHpuQczvLZdDslLsSWsLENVI/edit?usp=sharing
@Sean I had gone back and made some architectural changes along with the inclusion of more relevant user stories. The focus being more on the establishment of a secure pipeline to support future extensions for BRL-CAD via a C++ socket API along with the MCP Server.
Would really appreciate it if you could give it a read and let me know if the direction and scope feel aligned with BRL-CAD's goals
Sean (Mar 30 2026 at 05:46):@Raghav Sharma just a caution, a tcp/ip socket is critically problematic for our corporate uses as it will get blocked by security policies or trigger anti-virus flagging or simply be unavailable. pipes (stdio) are typically okay, along with other non-ip based IPC.
That's not to say a tcp socket can't be made to work, but then you'll definitely need to carve out a couple weeks to ensure you're registered correctly (cross-platform) and that it's not triggered/blocked, etc. It may be sufficient to target configuration settings that avoid the potential issues (like limiting to ports >= 1024 if that's enough, don't know).
Raghav Sharma (Mar 30 2026 at 15:13):Ahh that is valid @Sean
I will update the proposal to use a stdio based approach for the primary transport instead of TCP sockets. Perhaps we could experiment with the target configuration settings for TCP later on after the stdio thing is done.
Thanks for the heads up!
Last updated: Mar 31 2026 at 01:22 UTC