LLM Client, AI Assist Screen, Moqui Sim Mode, etc (llm-client branches)

Technical Discussion
1

There are two new branches called ‘llm-client’, in moqui-framework and moqui-runtime:

These contain a proposed architecture and implementation of an LlmClient class in the framework that handles talking to an external LLM, and it handles tool calling, agent loop, logging, etc. The initial tools include artifact info browsing, skill lookup, skill leaning in a simulation overlay, and calling screen transitions, services, etc using the current user’s authz.

The tooling architecture and skill learning are designed in a particular way to help mitigate issues with AI automation in business systems. The primary mode is “skill-first” where the agent looks for a relevant skill, and if it does not find one it spawns a sub-agent in a simulated environment to experiment and find a solution, then document it in a skill.

The simulation environment uses a number of existing context and production isolation mechanisms in the framework, and adds a new one that I’ve been thinking of doing for a long time: TransactionCacheDb. This uses H2 as an in-memory database to overlay the external database and act as a multi-transaction read-through cache where writes stay in the cache DB until flushed to the external DB, or as in the case of the sim application, thrown away.

Built on top of these is the Assist screen. This is a new screen in the tools component available to all users. It has a LLM chat on the left, and a screen canvas on the right. It supports two render modes for the LLM: a simple JSON form spec output, and a Vue Single-File Component output. The overall workflow is ask the LLM to do something, it figures out how to and creates a form for it, even if the form is just descriptive text and a submit button. The user always maintains control, changes only made when buttons are clicked.

With this in place, and once it is all cleaned up and production-worthy, the game changes for what is most important to build and maintain. Creating screens is more about the structure and data+logic encapsulation that they offer than the output they produce. The XML REST API becomes a far more viable approach for internal as well as external data access and logic. Ideally, only very few screens are ever made again, focusing on those with more complex interactions and requirements, and that will be used frequently or for a very particular important purpose.

For general notes see the ReleaseNotes.md file (in the branch):

For an overview of the skill learning and TransactionCacheDb stuff, see the new LlmSkillLearning.md plan:

3 Likes
2

A note on status for llm-client: this was all slapped together quickly with AI assistance. It has automated tests, and has been tested end-to-end including skill learning and use, but it has NOT been evaluated for performance, reasonable resource usage in production settings, or ensuring that the sim environment is sufficiently isolated.

In other words, this should definitely still be considered experimental and NOT used in production. It’s great for local use with an LLM running in LM Studio or vLLM or whatever, and that has been the only real test case so far.

If anyone would like to help with testing, using it on a staging server with a clone of a production DB is a test that needs to be done a bunch sooner or later, but any testing and review would be helpful. I made sure a few things were done in very particular ways, but there could still be a bunch of stuff in there that I haven’t caught… because I have not yet read/scanned all of the code, not tested from various angles to catch things that I’d probably miss in a code read anyway.

You’ve been warned. That said, have fun! I am. :slight_smile:

3

This is an excellent feature — looking forward to the official release soon!

Another project in the community (schue/moqui-mcp: MCP component for Moqui) provides MCP and built-in Agent capabilities. With this feature upgrade, the Agent functionality in that project can be deprecated. But what about its MCP functionality? Could it be promoted to a standard feature of the framework?

4

The ‘tools’ functionality is basically MCP, but Moqui-specific and not a general plugin sort of thing. More tools could be added, but most things would run through services, screen transitions, etc and external tools would plugin better through those interfaces so that security (authz) and other protections are in place. That is an important part of the design.

This is an alternative to what @schue put together with his Moqui MCP work, and is complementary to the more recent SystemMessage based things (ie those could be adapted to use this).

5

Hi David,

I wonder if it is convenient to extend the llm-client branch to also support the A2A standard as an interoperability layer, so a Moqui instance could act as a remote agent for other agents (and reach out to them in turn). Here is the link to the core concepts: Core Concepts - A2A Protocol

Perhaps a mapping could be as follows:

  • Message → LlmMessage, but holding one or more Parts (below) rather than today’s flat content String.
  • Part → the fundamental content container, shared by both Message and Artifact. Carried as typed JSON at first (matching the *Json field pattern), but since it’s referenced from two parents it could probably have its own child entity.
  • Task → perhaps the only piece currently missing, and I don’t think it wants to be grafted onto the LlmConversation lifecycle — a conversation is the session, not a unit of work. It feels more like a new first-class entity living in the skill-execution layer, closer to LlmSkillUse (which already records a run of a skill) than to a conversation. TaskStatus would then be its own lifecycle (submitted/working/input-required/completed/…), separate from conversation status.
  • contextId → this is where LlmConversation fits naturally: the session that groups related Tasks, rather than being a Task itself.
  • Artifact → a new entity (with a name that doesn’t collide with the framework’s own Artifact* concepts — ArtifactHit/ArtifactAuthz/…), likewise composed of Parts.
  • AgentCard → a discovery document assembled from profile config plus a curated projection over LlmSkill; AgentSkill would add the advertise-side metadata (tags/examples/io modes) that LlmSkill doesn’t carry today, since its role is internal procedural memory rather than a public capability list.

What I find most compelling is that a remote agent reaching Moqui over A2A would inherit the skill-first, sim-protected, authz-gated execution for free — which arguably makes Moqui a safer A2A remote agent than a bare LLM wrapper.

Thanks,
Igor

1 Like
6

Thanks @igor.gallingani, that’s a good idea. With the surge of desktop agents something like this might end up being a key enabling feature. I probably won’t get to the implementation until next week, but here is a first pass on an impl plan: