I am using Ubuntu 17.10 with the default Ubuntu desktop (no extensions added) and my computer has 4 GB of RAM. I noticed that every time I use Gnome Shell, either by pressing the Super key, Super+A, Super+Tab or by clicking an element on the top bar, the RAM usage of the gnome-shell process increases by a couple of MB. This doesn't seem much of a problem, but after some hours of work gnome-shell usage climbs higher than 1.5 GB, with the total RAM usage being close on 4 GB. Then swap is starts getting filled up, making my computer extremely slow.
Does anyone else notice the same behavior? Why does gnome-shell memory keep adding up? Is there a way to clear gnome-shell memory usage other than logging out?
Several years later and after the question was closed and reopened, I have a pretty good idea of what the issue was. So I'm answering my own question.
The issue was due to the infamous GNOME Shell memory leak that was plaguing GNOME Shell since before Ubuntu 17.10. There was a bug report for the issue that dated back to 13/03/2017, but the issue probably existed even before that.
A GNOME developer (Georges Basile Stavracas Neto) went on in 2018 to find the cause of the leak and found that it was related to Garbage Collection. The fix was released later that year for both Ubuntu 18.04 and 17.10.
I have not observed any more severe memory leaks like this since the fix was released. Some mostly minor memory leaks have been observed by other users though, as one can see in the linked bug report above, but they are attributed to different causes.
On an Xorg session, restarting GNOME Shell by pressing Alt+F2, entering
r, and pressing Enter would suffice to clear the leaked memory and restore the computer to a usable state. Restarting GNOME Shell on Xorg is still a good way to solve a multitude of problems.However, it was not (and still isn't) possible to restart a Wayland session in a similar way, thus the only to solve the issue at the time on Wayland was to log out and log back in.