GUI remote connection to a server freezes

The issue relates to a remote connection to a server. After some time (possibly due to inactivity), the access through a GUI (window) application (regardless which application) stops, freezes, and cannot be accessed until after a system reboot. Other than that, the SSH connection via terminal can be reset and it works fine again. But the GUI access remains frozen. Is there any way to "unfreeze" the access without a system restart?