I have run into the same dilemma. I love the idea of isolated "environments", but it adds complexity. If you are on Windows and you run your containers in WSL2, you may actually see the agents running faster. When I switched from Claude Code running on Windows against a git repo on the Windows file system to Claude Code running on WSL2 and/or a container in WSL2, I noticed a significant performance improvement. I guess it really depends on your motivation. If it's performance and isolation, go for it. If you want things simple and don't care about performance, then run them locally.