Launch SSH access from a remote Server error, possible caused by ssh key forwarding

There is a VM ubuntu server I am using which is named vm.

I used to remotely log into it through SSH public key.

But now I need to use ssh to other servers FROM vm, ex. pushing to heroku.

There is an unexpected results for that:

1. When use my local console to remote control vm, use ssh push won't succeed, it complains about failed publickey.
2. However, when I use the vm directly (from the linux terminal), I could push with ssh without error.

My first thought would be some mis-configuration of the ssh key forwarding, any clues for me to solve this problem?