I have a Linux rhel xen host which will be running many 32 bit guests. (The guests have to be 32bit). The server has 48gb of ram, and currently it runs a xen kernel but this only sees 16gb of ram on the dom0.
Is there a way to avoid this problem ? Each guest only needs 256-512mb
Not hopeful, but any suggestions would be great.
The trick here is that the bitness of the Xen hypervisor and the bitness of the dom0 kernel and other guests can be different.
You can run a 64bit hypervisor and have a 32bit dom0 and guests.
Also of note, the dom0 doesn't manage the memory, but instead Xen the hypervisor does.
In your dom0 you will only see the memory that is allocated to dom0 by Xen. Xen the hypervisor (as long as it is 64bit (or even 32bit pae should work)) can handle the larger amount of memory. top will show the memory of dom0, which is what you checked, but xm top (or xentop) will show the amount of memory that Xen knows about. Check xm top (xentop) and see if Xen knows about all of the memory. If not, you should be able to install the 64bit version of the xen hypervisor (not that this is different than the dom0 kernel, which can still be 32bit - it is actually recommended that it is 32bit for performance reasons - see http://www.slideshare.net/xen_com_mgr/6-stefano-spvhvm
According to http://www.redhat.com/rhel/compare/
[3] The "SMP" kernel supports a maximum of 16GB of main memory. Systems with more than 16GB of main memory use the "Hugemem" kernel. In certain workload scenarios it may be advantageous to use the "Hugemem" kernel on systems with more than 12GB of main memory.
[4] The x86 "Hugemem" kernel is not provided in Red Hat Enterprise Linux 5 or 6.