Updating Docker Compose Files - OpenSearch, Compose Plugin, Postgres

[jonesde]

I was recently working on a simple docker-based setup and ran across a couple things that I think should change in the OOTB/example docker compose YML files in moqui-framework/docker.

1. the new version of Docker Compose seems to be a plugin to docker instead of a separate app and all old docker-compose options are now in docker itself under the compose command; to update this all we need to do is change docker-compose to docker compose in the compose-up.sh and compose-down.sh files
2. update all compose files to use OpenSearch instead of ElasticSearch; in a first attempt this went pretty well, but OpenSearch has security on by default so the compose file includes user and password default settings, and uses https instead of http to access the OpenSearch server (these are benefits in addition to OpenSearch vs ElasticSearch licensing, and a good example of how OpenSearch is filling in the old ElasticSearch paywalled functionality)
3. add opensearch-dashboards which has some admin functionality as well as Kibana-like functionality
4. update to Postgres 14.5; in limited tests this is working well with moqui and with the same settings and such as the prior version referenced there (12.1); note that 12.1 isn’t terribly old, but Postgres has been quite active in recent years
5. add JAVA_TOOL_OPTIONS env var to the moqui-server container config to limit heap size to 1024m by default, along with the OPENSEARCH_JAVA_OPTS that limits OpenSearch to 512m; these should of course be changed as needed, along with everything else in these sample compose YML files

I also added a little script for rotating backups of a Postgres DB with comments about crontab settings and such, by default for once per day (via crontab) with a 30 day retention (change the script to change that).

Here is a PR with the changes: Comparing master...dcompose-updates · moqui/moqui-framework · GitHub

These together make a good example for a single-server setup using docker containers for all the infrastructure Moqui needs, plus a bit to help when using Postgres which these days I’m thinking is a good default recommended database for Moqui.

I think most changes are for sure a good idea, like using OpenSearch and in a more secure way than ElasticSearch, but not sure about changing from docker-compose to docker compose to use the new plugin… it looks like that’s the standard way to do it now but I’m wondering what others are seeing, or might run into given that this is not a backward compatible change (which I’m not as worried about, these are example config files meant to be changed… at least for the passwords and encrypt key!

If all looks good the next step will be making similar changes to the other compose files in moqui-framework/docker.

[michael]

Response

@jonesde

but not sure about changing from docker-compose to docker compose

I think that this is fine as long as we’re not worried about backward compatible changes.

@jonesde

add opensearch-dashboards

We may want to use grafana for this role. It’s more generic, powerful, and commonly used.

@jonesde

update all compose files to use OpenSearch instead of ElasticSearch

It’s good to get around to doing this.

@jonesde

1. these should of course be changed as needed, along with everything else in these sample compose YML files

For the stuff that is expected to change, or is used in multiple places it might be a good idea to have a file that contains the configuration. This can be done like this:

postgres:
    image: postgres:alpine
    container_name: postgres
    restart: always
    volumes:
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - db.env

db.env:

POSTGRES_PASSWORD=password          # TODO! This needs to be a unique and good password!
POSTGRES_DB=moqui
POSTGRES_USER=moqui

Putting them in a file can also make it so that you can edit permissions so that only root or a certain user can access them.

In the Future:

Secret Storage

We may want to have sensitive secrets like passwords, and keys stored encrypted and not in plain text (see: docker secrets, kubernetes secrets, and hashicorp vault for secret management).

Example Configurations

We may want to have the dockerfiles here generatable so that we can avoid the maintenance of maintaining…

Computation

If:

• x is a set of attributes
• x_r are the required ones
• x_n are not required
• each attributes has n options

Then:
x_r * n + (optionally) x_n * n

… Lots of different combinations.

Summary

I don’t have any problems with this.

[girindrag]

If anyone doesn’t want to use https for OpenSearch then add this below environment in docker compose file.
- "DISABLE_SECURITY_PLUGIN=true"
@jonesde i have checked as per above change everything working fine for me.
Thanks for the new update .