ssh-agent lifetime refresh on use

I have an ssh key in ssh-agent set with a lifetime of 10 minutes. I want this 10 minutes to be the time since it was last used to successfully authenticate into a host. There's nothing immediately popping out in the man pages, so I'm wondering if it's been attempted via scripts or similar elsewhere.

The driver for this is that I have a key protected with a password. When I'm at my desk SSHing here there and everywhere I want the key in the agent so I don't have to continually put in the passphrase, but I also want it to expire on inactivity, similar to a screensaver.