How to allow Guest Network access to internal server with port forwarding?

Guest Network prevents access to internal network. That's perfect.

However I have a server machine that is accessible externally. I have set up port forwarding to allow this. I can access the LAN IP within my network and the External IP: through LTE on my phone or at work. That's perfect.

Guest Network cannot access server through LAN IP. That's what I want.

Guest Network cannot access server through External IP:. This is my issue, I want access.

Is External IP: being resolved to LAN IP so Guest Network blocks it? Is there anyway I can allow access to the server on Guest Network by going to the out to the Internet and coming back in?