What a champion! Great work! I’ll try out your changes.
Having some deployment issues in various compose files. Fixes are being delivered, will update on what changed soon.
Fixes applied to the above PR. What I discovered is the following:
- opensearch generates more files, added them to gitignore
- moqui creates two unnamed volumes because of dockerfile. The right solution is to add them in compose files as runtime volume binds
- US/Pacific is no longer an acceptable timezone for postgres so using standard UTC for safety on all compose files
- opensearch enforces very strictly a password with minimum requirements that cannot be overridden. Therefore changing it in all compose files
Now one thing remains, and it’s an old issue actually, which is downloading the JDBC driver for whatever is the desired database (which depends on the compose file being selected). I want to automate this instead of forcing users to download the driver manually so that the docker directory is very easy to use.
1 Like
I’ve added a file: docker/README.md to help folks understand how to deploy moqui on docker. It covers all the necessary steps including manual ones. We can automate some of this stuff in the future, but for now I just want to focus on upgrade, without changing behavior or introducing new features.
Will qapps2 changes be merged before releasing Moqui 4.0? I would like to do testing in conjunction with both Java 21 and vue3quasar2 changes.
Code I do some of the grunt work in adding qapps2?
I took a look at the changes done in the new theme system. On the framework side we’re touching core APIs like the MessageFacade, ScreenRenderImpl MoquiDefaultConf.xml and MoquiStart of which the upgrade initiative at least impacted the conf file and moqui start directly, and indirectly by upgrading the libraries in the others.
On the runtime side, we’re touching GroovyShell which was rewritten from scratch to accommodate the removal of Groovysh in groovy 5. We’re also touching some build.gradle files which are all affected by the upgrade to gradle 9.2, I also see some issues like adding jars and a few other minor issues. Then the vast majority of the work of course is the new theme system assets.
This makes me a bit nervous to put everything in 4.0. Both initiatives needs stabilization and we have some conflicts to resolve. Maybe it would be safer if we first proceed with Moqui 4.0 and merging all these already massive changes where every single thing was upgraded and modernized. Then we can push for the new theme system in 4.1, we do it slowly and carefully and look at all things that diverged and conflicted over time as the two initiatives took different directions in different periods.
If on the other hand folks prefer to push the two together, then I can make a few PRs to at least address the critical APIs like MoquiStart and others mentioned above.
Maybe @michael can give us his thoughts on how to proceed.
@byersa @taher Vue 3 and quasar 2 isn’t ready yet to my knowledge and will be way more painful to upgrade.
I think 4.0 should just be taher and service / entity changes with screen changes in a future version.
@taher @schue Docker recently released a new product for hardened docker images and there is an adoptium one. This makes me skeptical of just using the default docker one Image Layer Details - eclipse-temurin:21-jdk by default. The main difference is a guarantee of less CVEs for the hardened image, and the normal docker hub one has a critical and many warnings.
I think personally, a hardened distro is pretty much required for any of my deployments, so I would change them anyways. We could use something like: Image Layer Details - eclipse-temurin:21-jdk-ubi10-minimal which has 0 vulns, but I believe won’t come with the fonts we need that ARE in debian.
Interested in your thoughts.
Definitely a good idea, and worth it. I will need to update the Dockerfile and a few other things and test to make sure all of this stuff still works. We can even try to go for hardened images for the other services if they exist.
Would you like to merge the work and then attempt that, or do you want this done before merging? I am open either way, we’re quite close to the finish line.
It’s already merged.
I pushed a quickfix for running gradle getRuntime and a convience gradle task for setting up vscode.
I’m worried about the docker hardened images becoming commercial (they already require an account where docker hub doesn’t).
It’s would be worth it to me to take a minimal install of temurin and adding the necessary fonts and whatever so we can have a free community based secure Moqui docker image by default (maybe we can even get the other images cleaned up too like open search and postgres).
OK @michael I did some research, and for the most part the thing that needs to be hardened is the application server, less so on the infra side especially if you’re not exposing ports.
Now to avoid commercial lock-in, we can write our own Dockerfile to build everything in a secure way. However I think we should provide it as an opt-out option where you can add a flag to “build” it instead of “pull” it. A ready image with no clear way to rebuild might be too much “magic”.
Anyway, I think the whole /docker directory needs some attention. For one thing, we should have an easy way for people to add more services without touching that folder. Sort of like docker compose -f one.yml -f two.yml, this can be done by adjusting the scripts to allow adding more services files.
Given that this might take a little time, and we have a lot of ideas for a lot of things, I suggest we focus on rolling out 4.0, and start writing a “roadmap” of priorities in moqui-org.
Docker is definitely there at the top of the list for me, but also some work on elasticsearch helper APIs, user interface and the theme systems, updating documentation in moqui-org and all the places where change is happening. I also have some ideas for having an interactive help system that is context aware. And many other things (too many ideas, little time).
So I think moqui-org might need a section for “roadmap” that becomes our guide on what do if we have enough time to do something. And while we’re at it maybe we should find out why those services are coming down a little more often lately.
So let me know your thoughts on all of this. Meanwhile I’ll also do a few rounds of testing on some existing projects to try and iron out any kinks in 4.0.
1 Like
OH on another note, something nice, we accept an environment variable $RUNTIME_IMAGE which allows overriding the eclipse-temurin one. Furthermore our scripts accept overriding the image, docker file is already receiving that image. In fact I’m going to add this documentation to README.md
So @michael to use a hardened image, literally all that you do is something like:
./build-compose-up.sh moqui-acme-postgres.yml .. eclipse-temurin:21-jdk-ubi10-minimal
I just tested this and it works!
2 Likes
I created a framework tiny PR to add documentation to the above comment. So anyone can deploy a hardened image, while we find a more permanent solution.
This seems to be broken or duplicated. I fail on calling:
❯ ./gradlew setupVsCode
> Task :setupVsCode FAILED
FAILURE: Build failed with an exception.
* Where:
Build file '/home/taher/Desktop/active/repositories/fork/moqui-framework/build.gradle' line: 327
* What went wrong:
Execution failed for task ':setupVsCode'.
> Could not find method setupVscode() for arguments [] on task ':setupVsCode' of type org.gradle.api.DefaultTask.
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to generate a Build Scan (powered by Develocity).
> Get more help at https://help.gradle.org.
BUILD FAILED in 1s
But this works correctly:
❯ ./gradlew setupVscode
> Task :setupVscode
VS Code settings updated at /home/taher/Desktop/active/repositories/fork/moqui-framework/.vscode/settings.json
Yeah that works for me.
As far as a roadmap, I suggest it’s not at the global level because that assumes an expectation that it will be done.
I fixed that.
Thanks @taher for all the effort. I have been testing the upgrade to 4.0 in some scenarios, found a few details but got it working. Here are the details:
One problem we found is running hazelcast. The upgrade branch of the github repo for moqui-hazelcast has version 4.2.8. That version no longer has support as of September 2024 if not earlier. It also does not support jakarta and requires e.g. the class javax.servlet.http.HttpSessionListener, the oldest version that migrated to jakarta servlet is 5.1. I have been testing 5.6 and it works without any problems in multi-instance settings.
Another aspect is the absence of the mime.types file (which was renamed to jakarta.mime.types in commit f729e650 of the moqui-framework repo). While it works when executing in gradle or even directly with the jar, it does not work in the docker container, so we had to rename the file back to mime.types.
Another problem we have is due to a change in how groovy 4+ maps properties to methods: we have a bunch of code using templateRr.text where templateRr is a ResourceReference, which is ambiguous and in the current moqui (v3) would invoke the ResourceReference.getText() method, but in moqui v4 invokes the ResourceReference.isText() method, so we end up with a Boolean instead of a String with the expected content. Not sure if we can fix this because any change would probably be worse, so perhaps we should just add some warning that this might happen in the upgrade process and would require updating code.
There are also some behavior changes in the RestClient due to the Jetty 12 upgrade, mainly because it requires stricter standards compliance from the servers it connects to. We need to connect to some servers that have older implementations and for the certificate-based authentication we had to make some adjustments, as well as for cookie handling because of stricter domain mapping.
@jenshp Thank you so much for this feedback. I encourage everyone to please help us with testing too. Every bit helps.
Addressing the points:
I made a moqui-hazelcast PR addressing this upgrade. They have a cool looking startup logo in the logs:
o o o o---o o--o o o---o o o----o o--o--o
| | / \ / | / / \ | |
o----o o o o----o | o o o----o |
| | * \ / | \ * \ | |
o o * o o---o o--o o----o o---o * o o----o o
Huh Interesting! it suggests a classpath or resource resolution difference between the Docker image and local execution. I can’t reproduce it with the current /docker image, can you share a way to reproduce this? that would help confirm whether this is an edge case in resource loading.
This is a groovy 4 / 5 language change, not moqui-specific. It can surface anywhere property style access maps ambiguously to getX() vs isX(). We cannot document this everywhere, instead maybe it is worth mentioning this as a warning in the release notes.
My first question is were you able to fix this? If not, can you give me an easy way to repeat and I’ll make the fix. However I think even when making a fix we should adopt this strict behavior by default, and maybe provide a call or a flag to relax when calling older infrastructure by having a call like relaxed(). However please note that even with this fix, we still have hard limits. Due to upgrading java, jetty and jakarta we can no longer support TLS 1.0 / 1.1 and similar old protocols. They are simply dropped. Also invalid handshakes cannot be overridden like malformed TLS records or or old unsupported cyphers and so on. Same for some invalid certificates (for example malformed chains or unsupported algorithms).
@michael above moqui-hazelcast PR for you to process. I forgot to add credit to @jenshp for reporting this, maybe in the merge commit ?
The cuplrit of the mime.types issue I found is a dependency problem:
org.apache.commons:commons-email2-jakarta:2.0.0-M1 depends on jakarta.activation:2.0.1, which reads only the mime.type file, not the jakarta.mime.type, which is read by the newer (and also included) jakarta.activation-api:2.1.4 / angus-activation:2.0.3. Also the jakarta.mail-2.0.1.jar was included along with the newer jakarta.mail-api-2.1.5. See Pull Request 689