Docker Image Automation and Releases

The moqui docker images haven’t been updated in several years.

Thus they have the log4j vulnerability and should be updated.

I propose that we / I setup an automatic way to publish docker containers using github actions or similar.

What do you think @jonesde?

It’s an interesting idea, maybe as part of the larger effort of releases. I bring up releases because to date the docker images in the moqui group on Docker Hub have been released versions that match the release tags and downloads available on GitHub.

Is there another purpose you have in mind for docker images, and specifically for more frequent auto-built docker images?

I think over time this has not been a priority because most users of Moqui do not use released versions (they get updates much more frequently from the git repos, some using forked git repositories as stabilization and/or customization forks). This pattern of using direct from source instead of released versions only also makes it easier for users of the software to contribute and in fact I’m not aware of anyone who contributes who does not use recent version directly from the git repos (there may be some, but everyone whose method I’m aware of uses recent from git and not releases).

Another reason ready to run release builds (or interim builds) have not been as useful is that many have custom components that are added to their builds. There is an option in the OOTB MoquiDefaultConf.xml file and Dockerfile to use an ‘ecomponent’ directory that is mounted to a directory in the host filesystem that has external components, ie external to the docker container. This helps in some cases, but not in common cases like using the AWS Elastic Container Service where it is common to use a generic host OS and not rely on anything in the host filesystem or OS except basic services (memory, cpu/processes/threads, network, virtual local filesystem, etc).

Okay, I think that’s everything major on the topic…

Like we talked about briefly on the call last Friday it might be nice to do a release, and that would get a more recent docker image plus more recent versions of the other ‘binary’ builds in executable WAR files for moqui-framework (demo), HiveMind, and PopCommerce (Marble ERP to be added in the next release series).

The releases take a fair amount of effort but aren’t too bad. The largest effort is writing up the release notes (in the ReleaseNotes.md or ReleaseNotes.txt files in various repos). In the last release series I did only very brief release notes and added a link to the commits since the last release for those who want more info. This is somewhat unfortunate as people aren’t as likely to find out about new features, but finding out about features is also a separate and larger issue.

The general release process is something like this, and each step is fairly repetitive across the roughly two dozen repositories that will be part of the next release series:

  1. (trigger) get all code and content (including ReleaseNotes.* files) ready and propose and vote in favor of a release
  2. update version numbers in addons.xml (I usually start there to have a reference for all other version number bumps), but don’t commit or push yet
  3. update and finalize release notes (including references to version numbers in the release series)
  4. update version numbers in build.gradle files (just a few), component.xml files (nearly every component), etc
  5. on GitHub add a release tag for each repository that will be released (this also creates the per-repo source archives)
  6. create other release artifacts
    • use the source downloads (starting with the moqui-framework ZIP file for the source for the release, then using gradle getComponentSet commands, which support source archive downloads and are meant for this purpose)
    • there are binary and combined source archive release artifacts for the moqui-framework (demo build), HiveMind, PopCommerce (to be released one last time), and MarbleERP (next release will be its first because it is new)
    • for each there will be a combined source archive (uploaded to GitHub with the release files), an executable WAR file (also uploaded to GitHub), and a docker image (uploaded to hub.docker.com)
  7. update the moqui.org web site by making changes to the moqui-org and then build and upload the new version
  8. announce on LinkedIn in Moqui Ecosystem group, on moqui@googlegroups.com mailing list, on forum.moqui.org, and maybe other places too

One trick with most of these is that they require commit access to every repository that will be released, and permissions to upload and such. One thing that helps a lot with releases is PRs for more release notes, or rather I think that is something that adds value to releases that I only do minimally in recent years.

Anyway, to your point about security and released artifacts, which include docker images and the executable WAR files (and really the source archives too), Moqui could release much more frequently if that made a difference to anyone. To date it has not bee much of a use to anyone, because of the reasons listed above about how most people and organizations use Moqui (with add ons and custom builds, sometimes with forked repos for stabilization and/or customization).

I guess that goes back to the question of how these docker images might be used… and I’m not sure what that might be.

On the call we also discussed the general objective of making Moqui easy to try out, but I’m not sure if a production-ready deployment is the best way to try it out (a LOT more work than a simple demo environment with embedded H2 and OpenSearch), and along with not being a production ready deployment I’m not sure there is much point in using really recent code for such an evaluation… as long as releases are somewhat frequent (and they have not been over the last couple years).

For trying it out the demo.moqui.org site is an okay resource, and from there I’m not sure… it really depends on the target audience… and for some trying it out might actually mean reading the docs. End users and business analysts often need info about what terms mean and how to go about doing things, and big questions about what is supported and what isn’t, and the end user documentation is maybe the most useful resource for that.

1 Like

Here are some old planning notes from the last release:

  • finish all ReleaseNotes.md/.txt files
  • update addons.xml all component.xml files
  • commit addons.xml, framework version update in framework/build.gradle
  • github repo tags: framework, runtime, tool components, mantle udm/usl and components, app components
  • update all README.md files
  • source and binary builds in moqui-framework, PopCommerce, HiveMind
  • MoquiDemo, HiveMind, PopCommerce on docker Hub
  • update demo.moqui.org, moqui.org
  • announce on LinkedIn, files on SourceForge

gradle getComponent -Pcomponent=moqui-elasticsearch -PlocationType=release
gradle getComponent -Pcomponent=moqui-poi -PlocationType=release

gradle getComponentSet -PcomponentSet=demo -PlocationType=release
gradle getComponentSet -PcomponentSet=popc -PlocationType=release
gradle getComponent -Pcomponent=HiveMind -PlocationType=release

zip -r HiveMind-1.4.2.zip HiveMind-1.4.2/*
gradle load test addRuntime

moqui/moquiframework:2.1.3

moqui/moquidemo:2.1.3
moqui/hivemind:1.4.2
moqui/popcommerce:2.1.2

docker login -u -p (server needed for docker hub? no)

NOTE: make sure mysql and postgres jdbc jars in runtime/lib
cp ~/Downloads/mysql-connector-java-5.1.41/mysql-connector-java-5.1.41-bin.jar runtime/lib/
cp ~/Downloads/postgresql-42.1.1.jar runtime/lib/

./docker-build.sh …/… moqui/moquidemo
docker tag moqui/moquidemo moqui/moquidemo:2.1.3
docker tag moqui/moquidemo moqui/moquidemo:2.1
docker push moqui/moquidemo

./docker-build.sh …/… moqui/hivemind
docker tag moqui/hivemind moqui/hivemind:1.4.2
docker tag moqui/hivemind moqui/hivemind:1.4
docker push moqui/hivemind

./docker-build.sh …/… moqui/popcommerce
docker tag moqui/popcommerce moqui/popcommerce:2.1.2
docker tag moqui/popcommerce moqui/popcommerce:2.1
docker push moqui/popcommerce