Other than the obvious danger of someone physically coming upon your open computer when you aren't looking and having access to your server, is there any danger if I leave an ssh connection open for a long time (like days and days)?
SnapOverflow
No, it's a secure connection end to end.
The recent releases of SSH is secure end to end just as @Iain mentions. The more pertinent security concern comes from the physical security at your end point. You don't want to walk away from an unlocked PC with domain admin credentials active .. the same goes for your SSH connections. A lot of the developers I used to work with had issues in this regard - they would leave connections open regularly with full sudo rights and leave their workstations unattended.