Pushed route is ignored by client

I am trying to set up a vpn server to get a virtual LAN with some friends. The vpn will not be used to route traffic to the internet.

The server is configured to hand out IP addresses x.y.z.* with:

server x.y.z.0 255.255.255.0

and it tells clients to use the vpn to access addresses in the vpn by pushing a route:

push "route x.y.z.0 255.255.255.0"

We also configure clients to "Use this connection only for resources on its network".

This works as intended from a Debian computer but it does not work from a Ubuntu computer. It cannot ping the server x.y.z.1 and I assume it is caused by the subnet mask being 255.255.255.255. The pushed route seems to be ignored. When trying to change the subnet mask manually from the client side, it cannot even connect to the vpn.

Am I understanding this correctly? Any hints on how to troubleshoot?