In practice, based on the few that I run in production, the ESXi core uses around 200 or so mb. Personally, I try to keep around a gig free of non-reserved ram per server (200 reserved for ESXi, 256 mb for Linux guests, 512 to 1024 mb for Windows guests depending on version and services)...
Memory overhead is going to hinge on a lot of things. Luckily, VMWare has a nice little table to let you know what to expect:
http://pubs.vmware.com/vsp40_e/resmgmt/wwhelp/wwhimpl/common/html/wwhelp.htm#href=r_overhead_memory_on_virtual_machines.html&single=true
Also see this
http://pubs.vmware.com/vsp40_e/resmgmt/wwhelp/wwhimpl/common/html/wwhelp.htm#href=c_understanding_memory_overhead.html#1_10_9_9_7_1&single=true
In practice, based on the few that I run in production, the ESXi core uses around 200 or so mb. Personally, I try to keep around a gig free of non-reserved ram per server (200 reserved for ESXi, 256 mb for Linux guests, 512 to 1024 mb for Windows guests depending on version and services)...