Podman "Permission Denied" Mystery: Why Can't Root in Container Access Mounted Directories?

Have you ever encountered this scenario? You enthusiastically configured a new systemd service using Podman and Quadlet, mounting a host directory into the container. You use podman exec to enter the container, and the whoami command proudly tells you that you are the almighty root user. But when you try to access that mounted directory, you’re ruthlessly told: Permission denied.

It’s like having the keycard to the presidential suite but finding you can’t even open the room door. What’s going on?

Recently, I encountered this puzzling problem. This article will document my troubleshooting process and reveal the “invisible bodyguard” silently protecting our system security behind the scenes—SELinux.

The Crime Scene

Let’s first recreate the “crime scene”. I was configuring a service, wanting to mount the host ~/sniproxy/app directory to the container’s /app directory.

My Quadlet .container file (sniproxy.container) roughly looked like this:

[Container]
Entrypoint=/app/entry.sh
Image=docker.io/library/alpine:3.15
Pod=sniproxy.pod
Volume=/home/nite/pod/sniproxy/data/bin:/app:ro
Volume=/home/nite/pod/sniproxy/data/etc:/config:ro

[Service]
Restart=always

[Install]
WantedBy=default.target

After configuration, I ran systemctl --user daemon-reload and systemctl --user start sniproxy.service, but the sniproxy service failed to start.

Looking at sniproxy logs through journalctl, I found Permission denied errors.

I then modified the sniproxy.container file, commented out the Entrypoint, set Exec to sleep infinity, and restarted the sniproxy service.

podman exec -it systemd-sniproxy /bin/sh entering the container and executing ls -l /app resulted in Permission denied error.

Instant confusion washed over me. I’m god (root) inside the container, but can’t even enter my own territory (the mounted directory)? Traditional Linux file permissions (owner, group, other) seemed to have failed here.

Solving the Mystery: The Real “Gatekeeper” SELinux

After ruling out conventional file permission issues, I turned my attention to the powerful security subsystem in modern Linux distributions (like Fedora, CentOS, RHEL): SELinux (Security-Enhanced Linux).

We can think of SELinux as an extremely strict bouncer in a club.

• Traditional file permissions (chmod): Like a bouncer checking your ID to see if you’re of age. The root user is like the club owner, defaulting to access anywhere.
• SELinux: This bouncer goes further, not only checking your ID but also examining your pass type (security context). For example, if you hold a “general audience” pass, even if you’re the owner (root), you can’t enter the “backstage staff area”. Every area and every person is tagged with specific labels, and only matching labels allow passage.

In our Podman scenario:

1. Container process labels: When Podman starts a container, all processes inside the container are assigned an SELinux label, typically container_t. This is equivalent to a “container visitor” pass.
2. Host directory labels: The ~/sniproxy/app directory you created in your host’s home directory has a default label of user_home_t. This is equivalent to a “private residence” area.

SELinux’s default policy rules very clearly state: processes holding container_t passes are forbidden from accessing user_home_t labeled areas.

This is entirely for security reasons. This rule exists to prevent container processes (even if compromised) from escaping and randomly reading and destroying your personal files on the host.

So the real reason for Permission denied isn’t insufficient permissions, but that you “went to the wrong place” and were stopped by SELinux, the dutiful bouncer.

Solution: Get Your Directory a New “Pass”

Since the problem lies with SELinux labels, we just need to tell Podman to “get the right pass” for this directory when mounting. This is achieved by adding a flag at the end of the volume definition:

• :z: Modifies the host directory’s SELinux label to container_file_t. This is a shared label, meaning multiple different containers can mount and access this directory.
• :Z: Creates a private SELinux label exclusively for this container for the host directory. No other container can access it.

For most scenarios, :z is the most commonly used and convenient choice.

Fix method:

Modify your .container file by adding :z at the end of the Volume directive.

Before modification:

Volume=/home/nite/pod/sniproxy/data/bin:/app:ro
Volume=/home/nite/pod/sniproxy/data/etc:/config:ro

After modification:

Volume=/home/nite/pod/sniproxy/data/bin:/app:ro,z
Volume=/home/nite/pod/sniproxy/data/etc:/config:ro,z

Important! After modifying the file, you must execute the following two steps to apply the changes:

1. Reload systemd configuration:

   systemctl --user daemon-reload

2. Restart your service:

   systemctl --user restart sniproxy.service

Now, exec into the container again, and ls -l /app finally works normally!