Ok, so it turns out running all the vm's from ram and turning down the swappiness in the kernel is most of the solution
- why does disk IO kill vmware more than kvm?
Lots of links, but nothing so far - beware some of this info will apply to VMWare Server 1 and some to 2
-----------------------------------------------------------------
| Also - some of this info will probably destroy your system |
| Beware! |
-----------------------------------------------------------------
Have VMWare Server 2 and getting wait states for the piggy vm's?
Here is what I have googled and have not tried them all yet:
Do your own research on what each one does cause I don't want to hear if you break things!
Shared memory - are you hitting swap when you don't need to be? Stolen from here and paraphrased in case it goes away:
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=844
I think this one reads better:
http://www.fewt.com/2008/06/performance-tuning-vmware-server-on.html
Longer but catches some things the one above does not:
http://daverdave.com/node/77
And a quick summary of vmx settings
http://blog.gulfsoft.com/2008/04/some-useful-vmware-parameters.html
One stop shopping and a nice css layout!
http://blog.mbentley.net/2010/02/optimizing-vmware-server-2-0-on-a-linux-host-2/
------------------------------------------------------------------------------------------
Changes on host and guest:
- before you give this a shot, read this about changing the size of tmpfs:
size: The limit of allocated bytes for this tmpfs instance. The
default is half of your physical RAM without swap. If you
oversize your tmpfs instances the machine will deadlock
since the OOM handler will not be able to free that memory.
------------------------------------------------------------------------------------------
Did you catch that part on oversize and deadlock? - O.K. on with the show, you have been warned!
Up the limit on the shm filesystem: (shared memory pool) - pull it up to the amount of ram
mount -o remount,size=4G /dev/shm
and add to /etc/vmware/config (global config)
tmpDirectory="/dev/shm"
If that works for ya, put it in /etc/fstab to keep after reboot.
tmpfs /dev/shm tmpfs size=4G 0 0
(read about the tmpfs in your kernel source: kernel/Documentation/filesystems/tmpfs.txt)
---------------------------------
To tune the host OS: (reduce the need to hit swap) - make sure you have not oversubscribed the physical memory!
/etc/sysctl.conf
vm.swappiness = 0
vm.dirty_background_ratio = 5
vm.dirty_ratio = 100
- late edit - might want to set vm.swappines = 1 (for 3.5 and later kernels as 0 disables swap it looks like)!
To set it before rebooting:
echo 0 > /proc/sys/vm/swappiness
echo 5 > /proc/sys/vm/dirty_background_ratio
echo 100 > /proc/sys/vm/dirty_ratio
How about errors like SCSI0:0: Command WRITE(10) took 3.145 seconds (ok)
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008885
Again, summary:
If you have lots of ram, edit the vmx on the host that is having issues and edit / change: MemTrimRate = "0" mainMem.useNamedFile=false sched.mem.pshare.enable = "FALSE" prefvmx.useRecommendedLockedMemSize = "TRUE" Also check to see if you can: 1. Move the vm image apart from the /tmp or base OS runtime to different filesystems 2. Add disks to your raid5 setup (more disk, more spindles, increased performance). 3. Defrag both guest and host filesystems. 4. Use preallocated mode for the disk image. 5. Make sure your disk is not failing!