You should be able to use any reverse proxy to replace the one it includes, but you’ll want to understand the various things that need to happen. Specific HTTP methods and content-types must be routed to the lemmy and not the lemmy-ui container, and not all of these (e.g. paths) are well documented…I just found this out the hard way: watch the caddy logs carefully as you are dealing with federation issues.
Both Mastodon and Lemmy have this problem. Make the default where the most new content is, which is going to be the federated tab and all tabs respectively.