I thought locked SSH access to my IP would be a good thing but now I'm completely locked out and nothing I do in rescue mode seems to work.
I've tried changing/removing/uninstalling iptables but nothing I do in rescue mode seems to save and as soon as I boot the machine back up from the HDD all the settings have returned :/
Many VPS hosting companies provide "console" access to fix problems like this. In a comment you mentioned you are using OVH. Here is their help page explaining how to use their implementation of console access: https://docs.ovh.com/us/en/public-cloud/access_console_horizon/
Tom Brossman was correct there is a console but it's in a slightly different place for dedicated servers.
I feel a bit dumb so I'm going to leave this here for any other dumbasses like me :P
https://docs.ovh.com/gb/en/dedicated/use-ipmi-dedicated-servers/