Reach private subnets on secondary interfaces between GRE tunnel endpoints

I have two servers in different locations. Each server has an eth0 with a public IP (internet accessible) and an eth1 with a private IP connected to the internal network.

Server A

eth0: 1.2.3.4 (public IP)
eth1: 10.0.0.10/24

Server B

eth0: 5.6.7.8 (public IP)
eth1: 10.10.1.10/24

I have a GRE tunnel between the two servers, created like this:
Server A

ip tunnel add gre1 mode gre remote 5.6.7.8 local 1.2.3.4 ttl 255
ip link set gre1 up
ip addr add 10.128.128.1/24 dev gre1

Server B

ip tunnel add gre1 mode gre remote 1.2.3.4 local 5.6.7.8 ttl 255
ip link set gre1 up
ip addr add 10.128.128.2/24 dev gre1

Now the goal here is to be able to access the other side's private network from each server. So from Server A I want to be able to access the devices at 10.10.1.0/24, and from Server B I want to be able to access the devices at 10.0.0.0/24.

I tried adding ip route add 10.10.1.0/24 dev gre1 on Server A and ip route add 10.0.0.0/24 dev gre1 on Server B, but it didn't help.

What am I missing? Do I need to use source route tables?

Thanks in advance,
George