Podman Compatibility Issues

[eltonfss]

I've been trying to deploy the latest image with podman-compose and haven't been unable to figure out why it does no work. Can anyone help me?

Zip containing the compose.yml and additional config files: agraph_issue_example.zip

Log:

➜  agraph_issue_example podman-compose up   
['podman', '--version', '']
using podman version: 4.3.1
** excluding:  set()
['podman', 'network', 'exists', 'agraph_issue_example_default']
['podman', 'network', 'create', '--label', 'io.podman.compose.project=agraph_issue_example', '--label', 'com.docker.compose.project=agraph_issue_example', 'agraph_issue_example_default']
['podman', 'network', 'exists', 'agraph_issue_example_default']
podman create --name=agraph_issue_example_agraph_1 --label io.podman.compose.config-hash=123 --label io.podman.compose.project=agraph_issue_example --label io.podman.compose.version=0.0.1 --label com.docker.compose.project=agraph_issue_example --label com.docker.compose.project.working_dir=/Users/eltons/Code/agraph_issue_example --label com.docker.compose.project.config_files=compose.yml --label com.docker.compose.container-number=1 --label com.docker.compose.service=agraph -v /Users/eltons/Code/agraph_issue_example/agraph-config/agraph.cfg:/agraph/etc/agraph.cfg -v /Users/eltons/Code/agraph_issue_example/agraph-data:/agraph/data/ --net agraph_issue_example_default --network-alias agraph -p 10035:10035 -p 11036-11436:11036-11436 --shm-size 4g franzinc/agraph:v7.3.0
Resolving "franzinc/agraph" using unqualified-search registries (/etc/containers/registries.conf.d/999-podman-machine.conf)
Trying to pull docker.io/franzinc/agraph:v7.3.0...
Getting image source signatures
Copying blob sha256:480132409357ae5587627204f0ced161306f9a836aa5b85b97041179bcf65d65
Copying blob sha256:5bed26d33875e6da1d9ff9a1054c5fef3bbeb22ee979e14b72acf72528de007b
Copying blob sha256:78bf9a5ad49e4ae42a83f4995ade4efc096f78fd38299cf05bc041e8cdda2a36
Copying blob sha256:f11b29a9c7306674a9479158c1b4259938af11b97359d9ac02030cc1095e9ed1
Copying blob sha256:930bda195c84cf132344bf38edcad255317382f910503fef234a9ce3bff0f4dd
Copying blob sha256:6c2cabc6d26383ea81907a94da3b6878eb761df490db9601d9e925389b9b4834
Copying blob sha256:7ff2721009c62901f977ed5b2a3537c1ec513bad8d37bb43ace5734f67d7d6b3
Copying blob sha256:1b0ea6736978c3f15e6e461062e0a69d87d20c5f6f85653b84283194d5730da1
Copying config sha256:a2fbc773ea405121c32a709e197e8f02c8a355669e1b1f67d0d435aca37dbed4
Writing manifest to image destination
Storing signatures
42040d8d260eb8f34bcb18e9331d0284789184e4addbf63bc22f5686b571c8a9
exit code: 0
podman start -a agraph_issue_example_agraph_1
sudo: unable to send audit message
sudo: pam_open_session: System error
sudo: policy plugin failed session initialization
exit code: 1

Podman Machine Info:

➜  agraph_issue_example podman machine info
Host:
  Arch: amd64
  CurrentMachine: podman-machine-default
  DefaultMachine: ""
  EventsDir: /var/folders/0w/3rvhqpr16rb46fxbpklj93000000gn/T/podman-run--1/podman
  MachineConfigDir: /Users/eltons/.config/containers/podman/machine/qemu
  MachineImageDir: /Users/eltons/.local/share/containers/podman/machine/qemu
  MachineState: Running
  NumberOfMachines: 1
  OS: darwin
  VMType: qemu
Version:
  APIVersion: 4.3.1
  Built: 1668026638
  BuiltTime: Wed Nov  9 17:43:58 2022
  GitCommit: ""
  GoVersion: go1.18.8
  Os: darwin
  OsArch: darwin/amd64
  Version: 4.3.1

Podman Machine Inspect:

agraph_issue_example podman machine inspect
[
     {
          "ConfigPath": {
               "Path": "/Users/eltons/.config/containers/podman/machine/qemu/podman-machine-default.json"
          },
          "ConnectionInfo": {
               "PodmanSocket": {
                    "Path": "/Users/eltons/.local/share/containers/podman/machine/podman-machine-default/podman.sock"
               }
          },
          "Created": "2022-11-21T21:49:55.772295-03:00",
          "Image": {
               "IgnitionFilePath": {
                    "Path": "/Users/eltons/.config/containers/podman/machine/qemu/podman-machine-default.ign"
               },
               "ImageStream": "testing",
               "ImagePath": {
                    "Path": "/Users/eltons/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-37.20221106.2.1-qemu.x86_64.qcow2"
               }
          },
          "LastUp": "2022-11-22T15:28:48.181752-03:00",
          "Name": "podman-machine-default",
          "Resources": {
               "CPUs": 3,
               "DiskSize": 50,
               "Memory": 8192
          },
          "SSHConfig": {
               "IdentityPath": "/Users/eltons/.ssh/podman-machine-default",
               "Port": 60771,
               "RemoteUsername": "core"
          },
          "State": "running"
     }
]

OS Info:

agraph_issue_example sw_vers
ProductName:		macOS
ProductVersion:		13.0.1
BuildVersion:		22A400

Hardware Info:

MacBook Pro 2019
2,6 GHz 6-Core Intel Core i7
16 GB 2667 MHz DDR4
SSD 500GB (249,98 GB available)

[theihor]

Hello. Thank you for reporting an issue.

I was able to run the example you've provided with podman-compose on Linux, so this is not a podman compatibility problem.

From the output log it looks like container misses some OS permissions.

sudo: unable to send audit message
sudo: pam_open_session: System error
sudo: policy plugin failed session initialization
exit code: 1

This is controlled not by a container image, but by a program that starts a container.

This is a guess, but you may try adding cap_add with relevant capabilities to your compose.yaml. For example:

    cap_add:
      - ALL

Documentation on cap_add here: https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities

This may be a Mac-specific issue. Let me know if cap_add helps.
If not, we'll take a closer look on a Mac machine.

---

A separate problem I noticed is that username and password are not generated if the /agraph/etc/agraph.cfg exists at a time entrypoint.sh is running. This is the case in your example, because you mount agraph.cfg in compose.yml.

This is a bug in the entrypoint.sh, we'll let you know when it's fixed.

In the meanwhile, here is a hacky workaround.
Create a custom start.sh

#!/bin/bash
/entrypoint.sh echo "##### Don't start yet, replacing agraph.cfg"
cp -f /agraph/etc/custom-agraph.cfg /agraph/etc/agraph.cfg
/entrypoint.sh # Start for real

The first entrypoint here will create a superuser for AG.

Change volumes section to

    volumes:
      - ./agraph-config/agraph.cfg:/agraph/etc/custom-agraph.cfg
      - ./agraph-data:/agraph/data/
      - ./start.sh:/agraph/start.sh

Override entrypoint:

    entrypoint: /agraph/start.sh

I hope this is helpful.

[eltonfss]

Thank you very much for your help @theihor!

I've performed the steps you indicated and the server was able to start up successfully.

Nonetheless, now there is an error occurring when I try to create a repository named test.

Here is my current configuration: agraph_issue_example.zip

OBS1: Had to perform chmod +x start.sh for it to work properly.
OBS2: Placed start.sh on agraph-config dir instead of putting it on the root dir.

[theihor]

That error is very unexpected. I could not reproduce this on Linux.

Our current hypothesis is that there is an incompatibility of podman's VM with MacOS when using memory mapped files. We'll look into it more closely.

Could you try using docker to run your compose.yml? Docker runs natively on Macs, so this way we can verify if it is indeed a podman problem, and not a bug in AG code.

Regarding superuser, I missed that there is a simpler way to set it for your setup (no need for custom entrypoint). You can add this line to agraph.cfg.

SuperUser user:password

[eltonfss]

Hi @theihor,

Again, thanks for the help!

Unfortunately I cannot use Docker Desktop for MacOS on my organization due to Docker's new licensing model. That's the reason I'm moving to Podman since everything worked "fine" with Docker Desktop.

P.S. Sorry for the delay for answering, I'm a bit short of time at this moment, but will get back to it as soon as things calm down over here.

[theihor]

Unfortunately I cannot use Docker Desktop for MacOS on my organization due to Docker's new licensing model. That's the reason I'm moving to Podman since everything worked "fine" with Docker Desktop.

In this situation I would recommend running AllegroGraph containers on Linux.

We are looking into this to make sure there is no bug in AGraph or ACL, but if the mmap error turns out to be caused by podman + VM-to-run-on-Mac, there is little Franz can do.

I'll post updates here if we find something out.