I switched at work because of the license changes docker did. I noticed that for my work workflow, podman was a direct remplacement of docker.
For my homelab, I wanted to experiment with rootless and I also prefer to have my services handled by systemd. Also I really like the built-in auto update from podman
I wanted to do something similar. But I grouped some containers using pods and it seems it broke the networking.
Eventually I kept the pods, and exposed everything to the host where caddy can meet the services there. Not the cleanest way, especially as my firewall is turned off.