Modernized Rust-based Moqui (MR-Moqui)

I’ve talked with a few people about this idea, and seems enough have already thought about it or like it that I did a little research on what it would look like. I also included a few notes on things to do differently from the beginning, since this is starting over with the implementation while trying to keep to the design sufficiently that current XML Screens, Services, Entities, and even configurations will mostly just be the same.

The biggest issues are the scripting language and template engine. I haven’t settled on what we might do about templates, looks like there are options, but for scripting this seems like a good opportunity to migrate to JavaScript so we use a browser-compatible language on the server side.

In addition to moving to JavaScript instead of Groovy, the ElasticFacade is begging to be redone and I’d like to put something in place that uses either the embedded Postgres or OpenSearch for full-text and parametric JSON doc search. Oh, and embedded PGLite would replace H2 so we have a more complete database for dev and small production setups. Small production setups would no longer require anything external, just PGLite embedded.

Here is a first pass on the architecture proposal:

This is VERY early concept stuff, and I figure we should iterate on this and let it soak for a while before anyone starts pointing an LLM at it. Once we do settle on a plan and have it laid out more comprehensively in implementation plans, I’m all for others spending their tokens instead of mine and their initial review eyeballs instead of mine getting the thing implemented. :slight_smile:

4 Likes

I guess before hashing out the architecture I want to understand the purpose of all of this. To me the programming language itself is just a means to an end.

The slowest part of systems based on moqui is network and storage. Compute and memeory are sort of secondary in that respect. So languages like rust, being more efficient won’t make a big difference no?

So what is the end goal here? Why not for example replace groovy with javascript while staying on the JVM, same with freemarker. Why replace everything, what do we get in return? In fact if we stay on the JVM, doesn’t that mean we can support both languages as two different engines in the service facade for example? What would stop us from improving elastic facade as another example?

Maybe clarifying the purpose helps shape the direction?

1 Like

For Rust vs Java one motivation is speed performance, another is memory usage performance, and one that turns out to be sort of interesting is external dependency reduction.

It’s looking like we’ll be able to have Moqui with Postgres embedded, and with support for search & analytics functionality on both OpenSearch and Postgres an embedded copy of Postgres also removes the OpenSearch external dependency (unless it is desired, which for big systems I’m pretty sure it still will be… but I suppose clusters of pgsql servers that do search and analytics might compete well with a cluster of OpenSearch servers dedicated to the same task…).

So, another motivation ends up being a Moqui executable with no external dependencies, not even a JDK or JRE, that still has a full app server and a relational database and data store for search & analytics.

Realistically, none of this would be a consideration without AI-assisted code migration. It just wouldn’t be worth it. With AI-assistance the game changes.

If we look at the biggest pain points that remain with Moqui they are largely things that are really difficult to change at this point. For example we’ve had a poorly functioning prototype for server-side HTML-to-PDF for many years in the hopes of dropping XSL-FO, and a re-architecture gives us options for that and helps us convert the XSL-FO files. That isn’t the best example of what AI assistance enables because we don’t have very many XSL-FO files, but we can also get serious about replacing Groovy with JavaScript for the expressions and scripts in XML Screens so that they can finally turn into a single file that can be rendered client or server… which will require a bunch of changes to inline Groovy in existing screens so AI assistance makes that reasonable to do.

We also end up dropping a lot of the dead weight that comes with Java infrastructure. We lose a lot of the tools, but as I’ve been researching this more it turns out that most of those tools are the ones we’ve been wanting to replace anyway. So yeah, no more Groovy… but we’ve been talking about moving to JavaScript for a long time, and with a native Rust JS engine we can do that quite efficiently for little expressions and expanded strings all the way to “compiled” XML Action script for large services.

What this really does is open up more options for Moqui. It makes Moqui itself easier to embed in things like desktop apps. We can get into running modes with very low startup times for specific tasks, and startup times for production servers configs like current Moqui ones should improve significantly as well. Deployment options improve because it doesn’t have to be a memory monster in every configuration (though of course it will be in some… the goal is even more to be able to stack on thousands of concurrent users and take advantage of app servers with 512GB RAM).

I’ve been vetting this idea and working on plans, so the repo has enough docs now to show in some detail what it would look like, and the README has a section that summarizes some of the more significant changes:

All that said, this is a wild undertaking… probably 100k lines of code, most ported and some needing to be written. Just like the world of Java, options for TX management and DB connection pooling are limited in the world of Rust (even more limited), so my plan so far is to actually port relevant portions of Bitronix… and do so selectively… which scratches another itch I’ve had to pull out some of the parts of Bitronix that are annoying and not useful (like the transaction logs that we have no way to feasibly use in a recovery anyway, can’t restore the app memory state so the TX being processed is lost either way).

Anyway, now I think I’m getting long winded… but hopefully you can see where this is going.

The reason of why at all and why now come down to the cost/benefit formula changing dramatically with current AI. This is recent too, probably earlier this year might have been the earliest time to even consider doing something like this.

And yes, many of the “significant redesigns” could absolutely be done in Moqui Java, even a transition from Groovy to JavaScript for XML Screens with a decent-enough JS engine (or a better one for those wanting to do GraalVM sorts of things). Now that we have AI, doing so for all screens in Moqui and recommending that everyone switch all of theirs over time might actually happen… without AI we have discussed it over and over but the price has always been too high.

So, what is MR-Moqui? In a way it is all the stuff we’ve wanted but the price has been too high for us to get it in the past. The “base” of that iceberg is Rust, the tip being something more like Groovy-to-JavaScript. Yeah, going all the way to Rust is extreme, but there are some significant benefits to it that we just don’t have in the Java world… and it brings a few of these things together with a clear line, people knowing that migration is necessary and much backward compatibility is just gone (well, without hopefully minor migrations anyway… components using Moqui XMl artifacts like the OOTB ones do will be the priority to make easy to port, being the first targets to port).

1 Like

So, in addition to things like ElasticFacade to SearchFacade (with support for OpenSearch and Postgres), Client-rendered XML Screens, using the same language for templates and scripts (GString templates concept to replace FTL, here implemented with JavaScript), and print documents via server-side HTML-to-PDF… what other sorts of improvements to Moqui have you been long wishing for?

There are potentially many things we could change in Moqui with an AI-assisted partial rewrite like this, but I’m trying to focus on the ones where there is a long history of it coming up over and over. The ones I’ve focused on so far in the MR Moqui plans are mostly based on filling the gaps for things lost by leaving the Java ecosystem. That includes JDBC & JTA, Groovy & FreeMarker, and while certain things like XSL-FO have some support it seems weaker.

Those are some big gaps to fill, but they are also tools that some of us have been talking about and looking into replacing for many years… like back in 2015 (IIRC) when I was working on that CSSBox stuff in the moqui-fop component for server-side rendering of HTML to PDF for print.

Going beyond that, what else might be worth considering? For the most part, I don’t want to change anything and for a big code port like this to work out well we’ll want to stick to what exists as much as possible. I’m even planning to keep most of the performance optimizations that exist in Moqui, like in MNode and MCache, and the low-level array index based referencing of entity field info so that tight loop stays tight and we don’t go back to looking up field info by name with a hash table.

So, what would be cool and/or has been a real pain? Let’s talk about it, especially if it is relevant to core framework functionality. There is a lot of other possibly cool stuff related to this, but this initial focus is the framework itself that would enable whatever else.

One way I’m looking at this is as an offering for AI era of information automation, so potential mass adoption is an architecture consideration. That means making it as lightweight and easy to run in various places as possible, as well as moving to more “universal” standards like JavaScript as a scripting language versus Groovy, and even HTML/CSS for print layout instead of XSL-FO. Plus, it’ll have the “Rust” buzzword so people will think it’s faster even if it’s not (we’ll make sure it is, of course).

1 Like

One concern is licensing of dependencies, but from what I’m seeing so far the world of Rust is quite liberal-license-friendly, lots of MIT and Apache licensing. The planning is far enough along to do an initial review, and it checks out clean with a few things to note when attempting to bundle the world into one big binary:

But why Rust and not Go for example? IMO, changing the language from Java to Rust is a very big change for programmers. It will be a very steep learning curve for a Java developer to learn Rust syntax and shenanigans. Another concern for me is the fact that Rust is a static compiled language, so what are we going to do about loading different components at runtime? Also, I’ve read that for some big projects the compilation time is quite high, couple of minutes or even tens of minutes. So moving to Rust is it really going to make development easier, or adoption better?

Definitely interesting stuff. As you and I have discussed before, I think a version of Moqui that can run entirely as a PWA without an uplink could be really useful from a business perspective.

The way Moqui works in Java with dynamic loading is not really something enabled by Java itself, but by things running in Java like Groovy and FreeMarker. In another language like Rust it is exactly the same: the underlying language is compiled and type safe, but we use a scripting environment running inside it for both runtime flexibility and for a more business logic friendly language.

@grozadanut when using Moqui, how often do you write Java code? It would be similar in Rust. Chances are unless working on the framework itself or highly optimized specialized code, you wouldn’t be writing any Rust at all.

As for scripting languages that run inside the environment, options in Rust are far superior to options in Java. Groovy is pretty good, compiles to bytecode, but in business logic we don’t typically use the compile static option, so there is significant runtime overhead. At the level of business logic in services and such, even one with nested loops, those aren’t really the “hot” loops that need fast running compiled code. The really “hot” loops in Moqui are in the Entity Facade and other parts of the framework.

The bigger change that matters for everyday use is really the shift from Groovy and FreeMarker to JavaScript. This is something we have talked about for years (well, some of us who get into architectural day-dreaming), and before considering an AI-enabled port to Rust that was always something we’d do in Java. Part of the problem with doing that is a high performing JavaScript runtime with minimal overhead for small expressions and such… it’s a problem in the world of Java, and while there are options like GraalVM that would cross a line into non-standard Java that limits runtime options, and a separate non-native engine like that has performance overhead issues per execution that are a problem for large numbers of small expressions like we have in Moqui (in addition to the longer scripts for services and screens and such).

The payoffs for switching from Groovy to JavaScript for server-side logic are huge. Tools for, and general knowledge about, JavaScript are far superior to the little world of Groovy. We can also finally get XML Screens that can be rendered on the server or in the client… and that’s a big deal that has been a LONG time coming.

The primary scripting language choice is not a small thing, it’s maybe one of the most important architectural decisions there is for something like Moqui. Using JavaScript is a big change, but also an important one that has been in consideration for a very long time (even from the beginning of Moqui, ie day 1 there was still an open decision between Groovy, JavaScript, or something else.

So, anyway, this is the bigger one to watch and plan for when it comes to MR Moqui, or potentially even for future Moqui Java if a good JavaScript runtime was even to happen. In the world of Rust we have QuickJS which is an all-Rust runtime for JavaScript. For expressions because of the overhead of calls to a big JS runtime like V8, this is even faster. If we ever need it for very large scripts or templates or even apps in JavaScript, with Rust we can link in the uber-optimized V8 engine that powers the modern interwebs with far less overhead than doing the same from Java.

More generally, this is a serious architecture transition from Moqui being something that sits inside other containers (like the Java virtual machine and the Jetty web server) to something that is itself a container. In theory there are disadvantages to that, to not sitting on layers that other things can sit on. In practice that has not played out for Moqui, and the overhead of those layers is significant.

One aspect of that overhead is in dramatic inefficiency in both code size and runtime performance. One thing I’m finding as I work on the architecture of Moqui in Rust is that while we lose a lot from the world of Java, it is mostly the exact stuff that we’ve been fighting for years and wanting to replace anyway. When looking at re-implementing things like MCache without the Java Cache API… it’s a big who cares. For things like XML parsing with just MNode and none of the fancy Java XML parsers… good riddance! MNode was written against the lowest level XML parsing APIs because everything higher level was insanely slow and memory hungry… and I was tempted to take it a step further an skip the XML parser altogether and drop it down to the input stream anyway… which is basically all that the Rust implementation has to do. In this case there are libraries in Rust for XML parsing that we could use, but with the highly optimized MNode to port there is no point… better to just use it and fill in the little parsing gap.

Losing JTA is sort of a double-edged sword, but for the most part it adds a lot of overhead that we don’t and can’t really use effectively. Losing JDBC is a bummer, but the reality over time has been that we have FAR more issues with handling the databases we care about really well versus issues with not supporting this or that particular database. From what I’m aware of, there are very few instances of Moqui not running on Postgres these days, and while multi-database support is a priority a much higher priority is using Postgres more effectively… which is a fairly big part of the MR Moqui plans.

@jonesde I enjoyed reading through all of this. One thing kept me scratching my head though which is templating.

Yeah I can understand groovy to javascript, but I don’t understand freemarker to javascript. Freemarker is a full engine with its own syntax and reach to the underlying runtime with context injection and whatnot. And javascript is … well javascript. We can’t just replace all the templating things with a back tick can we? Or are you intending to actually build a template engine using javascript? if yes, then it’s just another engine right, not straight javascript?

@taher the template engine is an interesting bit… while researching and planning I was initially looking at using Jinja templates with MiniJinja as our primary template language because Jinja is widely used these days and the MiniJinja implementation in Rust looks to be good, performs well.

Going back to discussions and prototype efforts over time about templating one of the big ones, and the best candidate to replace FreeMarker, was some sort of Groovy-based template engine. There is some simple stuff in there using GString, but it was clear that more would need to be built out to replicate FreeMarker features… and in the end it would look very different from FTL files so would represent a major migration. Unless the syntax and flow was way better it wasn’t worth it… and from my prototypes it is a really cool idea, but would have been a huge change and it never became a priority.

The general idea with using a scripting language as a template engine is that you use what is built in as much as possible, GString for Groovy and ES6 Template Literals in JavaScript, and then you wrap the heck out of it to get it to whatever you want. In other words, you build a script that is some boilerplate with the template text inserted, then you compile and run the thing to render the template. The wrapper can add functions and things that are available in the template for macros, includes, and on and on. The thing is, with a scripting language underneath the template side of is just a string/data inversion.

So… the closest thing to a universal template engine in the world today? Looks like that’s JavaScript.

There is also potential to do some significant templating workloads in JavaScript using template engines written for in-browser use like Mustache.js. For the plans so far in MR Moqui I’ve decided against this, instead using something more similar to what I had prototyped with Groovy which is to add some basic tools for templates on top of the scripting language and call it a day. It will run faster than Mustache.js or something, and give us the more precise feature set we need with minimal overhead.

Good fun isn’t it? As a side benefit, fewer dependencies and fewer languages to learn… well, I guess those were actually the priorities and then the rest is what comes of it.

So… how does that look… are we on the genius side of the line, or the crazy side? Or, maybe this is a case where it’s an odd enough problem and environment that the crazy side is the genius side?

The MCP screen interface prototype I built left me wanting a more explicitly structured widget tree. One of the more annoying things about the server side generation is that the UI widget tree that will eventually be rendered in the browser isn’t available to you server side. If we could access those on the server it would make the AI UI integration a lot easier.

For what it’s worth so far, an early trial implementation built at AI-enabled speed is coming together in the ‘initial’ branch of the mr-moqui repo.

It’s looking like with Postgres embedded and the big pile of libraries needed to do all of what moqui-framework and the various tool components do (including sftp, sso, curl, image, poi, and sort of fop) ends up with a binary ~67MB, so size is staying quite reasonable.

Startup time is silly fast, like 2s to port open with just the framework entites/services/etc, and so is data loading from a new DB… like loading into embedded Postgres (sidecar process, connected via file pipe by default in linux and macos) is faster than loading into H2 including the startup time and all.

Build times are indeed slow, and this is expected. The Rust reputation for slow buildings comes from one VERY GOOD thing: no magic binaries. With Java and Maven style repos we download built jar files. With Rust and Cargo we download the source for everything we include, and then build it from source. If we did the same with Java then initial builds would also be very slow. In Rust it is really the initial build that takes a long time, from there incremental builds are pretty quick.

The benefits to this are significant, and becoming more of a trend in modern open source software supply chains: no more magic binaries, download the source and build it. This is a big deal for auditability and software safety concerns.

So, yes, initial builds are much slower because with something like Moqui there are a lot of dependencies that need to be built. If we could reasonably do this on the Java side, it might be worth it (at least as an option) for those with more strict supply chain rules.

Overall, Cargo and the open source ecosystem there is very liberal license friendly (most stuff avoids viral term licenses), and has good auditability of dependencies too.

One benefit of this move to Rust is it gets us into a more “mature” software supply chain, or way better managed even if not as mature as in old.

Anyway, some thoughts so far from poking at this… as a side quest that Michael pushed me to pursue instead of just letting go: there is a WASM build with a browser interface module as well… and because we’re now using Chromium in a sidecar for PDF rendering as well as whatever else we might do with a semi-embedded browser, we can run mr-moqui inside of an mr-moqui instance… a sort of virtualization environment… though I’m not sure quite what to do with that yet, still noodling on it… might be very interesting for running sandboxed Moqui stuff.

1 Like

It’s now working, partly and mostly, most of the framework and tool components are in place, and the mr-mantle-udm fork loads fine, and mr-mantle-usl fork loads and the tests run using the new JavaScript based Spock-like test runner. A lot of mr-mantle-usl still needs to be migrated from Groovy to JavaScript, but that’s next on my list and then moving on to apps (I’m planning on a new app suite instead of literal MarbleERP).

Here is the main repo for MR Moqui:

IIRC it requires Rust 1.94+, easiest to setup locally with rustup, and then there is a little script called moq that has tasks similar to the stuff in the root build.gradle in moqui-framework. To get going just do a moq run and it will build and run, which like Moqui now will create the database (using embedded Postgres), load data, and start the web server. If you run moq load it will load demo data, just like current defaults, and then use moq run and you’ll have the demo user.

In one cold run with mr-mantle-udm and mr-mantle-usl in place for a more significant artifact and data load, it had the web server up in about 11 seconds. That includes loading entities and services, creating tables, loading data, and starting the web server. Search works with a new SearchFacade based on Postgres by default (configurable to run on OpenSearch, which so far does perform better from a few basic tests… hopefully Postgres does better with tuning).

Anyway, not all is working… I just tried browsing around in /qapps (which uses latest Vue and Quasar) and there are some quirks like navigation links aren’t working, but for a first pass and mostly client rendered it’s coming along… amazing what LLMs can do, and all this for maybe $40 so far in AI spend (using mostly Claude and Grok).

1 Like

I LIKE IT!

It does really start and run fast. I also like that the Wiki is using GIT. The UI also feels fresher. I notice various little idiosyncrasies in some of the widgets, are targeted PRs the best way to handle that?

Also, I’d like to shift the BizAgent work I was doing over to this baseline. Is a separate component still the best way to pursue that until development stabilizes?

1 Like

How to use and get involved now… that is a good question. This is a very different way of building things… the effort going into the ideas and the decision making along the way. It also moves really fast and lots of files change. I guess you can see that in the commit logs… it’s madness, even with most of it coming from a single $200/mo Claude account.

Right now I’m still just dumping a lot into mr-moqui and ironstone-boss (Business OS Suite). The mr-mantle-udm and mr-mantle-usl components are pretty much there, and MrSimpleScreens/MrMarbleERP are partly there, Michael has done some work on them and when stuff I’m doing in the framework needs a test case I’ve been migrating a few.

If what you’re looking is minor chances are some big sweep by an LLM will pick it up sooner or later. If it is major or something that works fine but is maybe not the best way to do it, that is where an Issue or PR is fine. With LLM help the ideas and feedback are more important than the code… it takes about as much time to ask the machine to write the thing as it does to ask it to read what a PR and evaluate… unless it’s complex then that’s worth it, having multiple LLMs looking at the problem and iterating on it does seem to improve results in notable ways.

I just started migrating the LLM and Assist functionality, though going with some design cleanups and variations (not a replica; some of these may be worth back-porting), scroll down past the wall of text the agents are using for notes, there are some tables and diagrams to lay it out:

This is a first pass, and feedback would be helpful. Some of it is already implemented and more coming along soon, so can be tried out.

I’m dropping in a lot of stuff that I would hesitate to add to Moqui Java because of its stability… part of my thinking is if this is going to be largely LLM written and maintained, then let’s go ham early on and get all our ideas in there before we start stabilizing and dealing with backward compatibility concerns (beyond database compatibility with Moqui Java, one way; and mitigated pain of migration for all the Groovy to JavaScript).

Some of the things I’ve added already include an SSH mini-shell with a JS interpreter… and now a full-on TUI with a ‘tui’ render mode for screens. That is seriously trippy to see, all the XML Screens rendered for a TUI instead of a browser… pretty cool. It can be used from the command line with ‘moqui tui’ or ‘./moq run tui’, by ssh into the server (default port 2222), and there is even a screen available in the browser that does terminal emulation so you can try it out there.

The git integration is pretty cool. You can have git repos in the filesystem, or in the database (with deltas, garbage collection jobs, the whole lot). For code having it in the database is only so valuable, unless you are using it as an alternative to GitHub/etc, but for content it’s a nice way to get strong revision management into the wiki, new notes stuff, and for general content. There is also LFS support, drops those in blobs in another table. This also ties into the new files/drive functionality in IronStone BOSS, which has NextCloud-like functionality (in Commons and Comms apps; also see the Projects app; those are the only built out at all so far).

Most of this is stuff I would never bother to build by hand, but when the cost is so low… the toolset starts to turn into something where synergies make some cool things possible. The TUI render mode is one of those that is wild to me, and from playing with it a bit the concept is pretty reasonable and implementation already not bad. You could run Moqui locally with an embedded Posgtres DB for various data management tasks, even full ERP, and never use HTTP… just run it all through the TUI in the terminal. :slight_smile:

1 Like

Of course I’m still on my agents as users trip, with projects and messaging as the glue. This rewrite seems like a great opportunity to fix up the appearance as well. Maybe I can survey some other ERP interfaces and see what kind of savory visual sauce I can add.

I’ll try the TUI stuff today. Fun!

What would you think about a research operation that actively trawls Odoo, NetSuite and anything else we can think of for design patterns? Could we feed something like that into the development pipeline?

I think that’s a great idea. :wink:

This is a first pass, and I let the AI make decisions on what to include or not while requiring inclusion of all existing Mantle/SimpleScreens functionality (see ss-inventory.md and usl-inventory.md). Much more could likely be add/incorporated over time.

There are research-*.md documents there for each of the planned applications… lots of AI research. This is mostly on open source apps/systems using local clones so the LLM had full source available.

1 Like

It seems like we need to talk about CI. We have PRs stacking up with no automation around processing them. We need to build that automation before we feed more material into it or it will just stack up.

Qwen3.8-27b-INT4 says the agent is good at producing evidence; the system is missing a machine-readable evidence gate (CI + harness report), a batching policy to kill the stacks, and a risk tier that lets the human skip reviewing Tier 1 entirely. Those three changes would probably cut the human load by 80%+.

My first step toward handling PRs was installing the GitHub CLI. :slight_smile:

That said, I haven’t used it yet. What the process ends up looking like is still a question. We have somewhat of a process with Moqui, but with slower activity for a long time there isn’t so much that is formal or in place to handle more people contributing and such.

The PR situation right now with mr-moqui may actually be caused by something very similar to the solution, just on the other side of it. Michael has been doing automated analysis and PRs, then reviewing and merging them in batches. Our discussion on this so far is that is something Michael is trying out and he owns the merge on the PRs he submits (unless I or someone else decides to review and merge before, point being Michael is trying out some tooling and taking responsibility for the merging too).

If MR Moqui takes off and we do get a lot of PRs coming in, we may very well need some formality around this including AI agents with CI environments, wherever that might live and whoever might own/oversee it.

As I’m thinking about an app for developers (visual editors for entities, services, screens; that sort of thing), I’ve also been considering CI and deployment. With the git integration and code management features coming together in Projects, it get’s me thinking about having more Jenkins-like functionality… sort of like the Instance Management stuff, but built out for real. The next functional step is monitoring, and that’s something I’d like to build in even if just for the internal system, but might as well be built to monitor other systems as well (other moqui instances as well as general web/etc app monitoring). With the SearchFacade the built-in monitoring could be backed by Postgres or OpenSearch.

An interesting potential for all this is doing much more with all-in-one, and without bloating the system too much because of shared tooling and code task-efficiency with the tools we have. It’s a different game when the cost of building just what you want using preferred tools is not just as easy but easier than adopting and managing an additional pre-built system (even if it is open source). I’m not sure if or how well that will play out, but just having embedded git repos and seeing some of these pieces come together gets me thinking. It’s not that far from a GitHub/Gitea type of thing already, and building out CI/CD functionality is a logical next step, and then monitoring a logical next step after that.

Going in the other direction, if we incorporate functionality like what Michael built for Coarchy, then we’re getting to a toolset for the full software lifecycle: Requirements, Design, Implementation, Quality and Acceptance Testing, Deployment, and Monitoring. Slap some AI on this and it could automatically respond by analyzing monitoring data and adjusting code, queries, etc as needed. :slight_smile:

1 Like