If I understand correctly, a VPN connection will make the whole computer behave like it is on the target network. Is it correct to say that a SOCKS proxy via an SSH tunnel would accomplish the same thing as VPN, except only on an application specific context (the application with the SOCKS proxy enabled)
That's a reasonably fair assessment. The connections coming from the SOCKS-ified application are going to originate, in terms of a source IP address, from the SOCKS proxy server on the end of the SSH tunnel. In an VPN configuration you'll see the source IP address being the IP assigned to the VPN interface on the client computer.