Here are some tidbits I have picked up along the way....
To do usb (and pci pass-through):
edit /etc/default/grub Tack on to the end of the kernel boot arg line: GRUB_CMDLINE_LINUX="intel_iommu=on" (or amd_iommu=on for amd) update-grub Enable Vt-d in the bios - reboot to test and grep for: dmesg | grep Vir dmesg | grep iomm
Q: I have a Vmware image. What should I do?
A: Why, convert it of course! qemu-img convert vmwareimage.vmdk -O qcow2 outputimage.qcow2 And import as an existing image: virt-install --connect qemu:///system --ram 1024 -n newvmname -r 2048 --os-type=linux --os-variant=ubuntu --disk path=/root/outputimage.qcow2,device=disk,format=qcow2 --vcpus=2 --vnc --noautoconsole --import -later update (cache none for performance when we have battery backing, as well as virtio for disk) virt-install --connect qemu:///system -n newvmname -r 6144 --vcpus=2 -c /path/to/cdrom-iso/ubuntu-14.04.1-server-amd64.iso --disk path=/path/to/vm/file/newvmname,bus=virtio,cache=none,size=123 --vnc --noautoconsole --os-type linux --network=bridge:br0 --hvm Skip the parts you don't have of course, and later define the nic as a bridge interface if you don't want the "local" networking.
If you get boot failure: http://wiki.libvirt.org/page/After_import_a_guest_from_an_existing_disk_image_using_virt-install,_the_guest_starting_stalls_with_%22No_boot_device%22
Q: What do you do if Windows Server 2008 fails to start the install?:
A: Run it manually - and configure your xml later.... kvm -m 1024 -drive file=/vms/vmdisk2k8.raw -boot d -cdrom /isos/w2k8.iso
Q: How do you determine the machine types available from KVM vs libvirt:
A: kvm -M ? qemu -M ? virsh capabilities Make sure they match :-)
debugging tips: http://fedoraproject.org/wiki/How_to_debug_Virtualization_problems
Q: How to log from libvirt?
A: /etc/libvirt/libvirtd.conf - or see here http://libvirt.org/logging.html
-or- (to start libvirtd) echo -e "log_level = 1\nlog_outputs="1:file:/var/log/libvirt/libvirtd.log"" >> /etc/libvirt/libvirtd.conf && service libvirtd restart
Q: So you have a machine down for the count with a qcow image and you want to copy files off of it:
A: You can mount it to the host using qemu-nbd:
Lots of good info on this site (in fact this bit that follows was taken from here: http://en.wikibooks.org/wiki/QEMU/Images
modprobe nbd max_part=16 qemu-nbd -c /dev/nbd0 image.qcow2 partprobe /dev/nbd0 (partprobe might complain about not being able to find partitions and you might have to use fdisk /dev/nbd0 and print the partition table...) mount /dev/nbd0p1 /mnt/image lvdisplay / lvscan & lvchange -a y /dev/blah-blah/root (for LVM2). When done: umount /mnt/image lvchange -a n /dev/blah-blah/root (if LVM2 is in use) qemu-nbd -d /dev/nbd0 lvdisplay (to check that it is no longer visible to LVM) Might just be faster to use: guestmount -a /usr/local/vms/vmfile -i --ro /mnt/test/
Q: How do you tune the bridge networking for libvirt / kvm operation?
A: Here are some pointers
http://wiki.libvirt.org/page/Networking#Bridged_networking_.28aka_.22shared_physical_device.22.29
Q: How do you p2v a Windows 2003 server without downtime?
A: SelfImage and qemu-nbd
Answer stolen from here: http://pve.proxmox.com/wiki/Migration_of_servers_to_Proxmox_VE#Physical_.28running.29_server_to_Proxmox_VE_.28KVM.29_using_SelfImage
First, install SelfImage on the 2k3 server. Now make a qcow2 image on the host server (make sure it is bigger than the total disk image you are p2v'ing): qemu-img create -f qcow2 newvm.qcow2 100G And then, export that image like so: qemu-nbd -t ./newvm.qcow2 Make sure you get the input and output portions correct if using this tool - make sure you have a good backup before starting as well! Back on the 2003 server, fire up SelfImage and point the input to the whole disk you want to copy. Point the output to NBD with the host name of the vm server and port 1024. SelfImage should detect your disk size. You are ready to start. Make sure you installed the correct drivers before imaging or you won't be able to boot. Also, make sure you have a decent connection between the server to be imaged and the vm server (i.e. 100Mb or better).
Q: Why can't I get console access via "virsh console vmname" in Ubuntu?
A: I think Ubuntu says console access is evil. Check on /etc/init/ttyS0.conf:
# ttyS0 - getty # # This service maintains a getty on ttyS0 from the point the system is # started until it is shut down again. start on stopped rc RUNLEVEL=[2345] stop on runlevel [!2345] respawn exec /sbin/getty -L 115200 ttyS0 xterm
Followed by: sudo start ttyS0
That should get you going. Stole the info from here on the "correct" way to set up a getty in Ubuntu: https://help.ubuntu.com/community/KVM/Access
Q: Why can't the cdrom be access without a reboot?
A: Make sure you are not getting issues from apparmor by testing with - touch /etc/apparmor.d/disable/usr.sbin.libvirtd and reboot and re-test.
If that works, you will need to wade through the apparmor allow / deny stuff.
Q: Why is the virsh shutdown domain command not shutting down your guest:
A: Make sure you have acpid installed (not default in Ubuntu server!)
A: Or, if using Windows 2008 Server, make sure nobody is logged in or shutdown won't work!
Q: Can a vm be autostarted when the host boots up and libvirt is loaded?
A: You betcha. Just run virsh autostart vmname
Some folks like scp or SystemImage for p2v. I am kicking the tires on mondoarchive.
This can work if you can stand outages: (dd across the wire from rescue disk to disk image, virt-import and run)
http://rwmj.wordpress.com/2009/10/13/poor-mans-p2v/
The virt-rescue mentioned looks cool as well.
Q: So how do you change the cd / dvd on a virtual machine?
A: (NEW ANSWER!) : virsh attach-disk virtmachinename /dev/sr0 hdc --type cdrom --mode readonly
Old answer below (will give you hints on how to add a hdc stanza to the xml definition for the virtual server...
A: attach-disk virtmachine /usr/local/disk1.iso hdc --type cdrom --mode readonly
Assuming your cdrom is defined as being hdc ide.
assuming you have the device defined as (for Linux guest - seems not to work for Windows guest):
<disk type='block' device='cdrom'>
<driver name='qemu'/>
<target dev='hdc' bus='ide'/>
<readonly/>
</disk>
If you need a physical disk, say /dev/sr0 do this:
attach-disk virtmachine /dev/sr0 hdc --type cdrom --mode readonly
assuming you have the device defined as (for Windows - not sure if this works for Linux guests):
This actually works for a Windows guest for either physical access or file access - strange, oh well...
--- and even more confusing, only if you have a disk in the cdrom / dvd drive!
<disk type='block' device='cdrom'>
<source dev='/dev/sr0'/>
<target dev='hdc' bus='ide'/>
<readonly/>
</disk>
So, I guess you will want to point it to an iso file that really exists on disk and just allow Windows to chew on that file unless it is detached with the eject in the gui and then attached again with an attach-disk command -ugh-
Actually, this works well as it boots up quickly if the iso image is on the local disk, and allows you to eject in the gui and mount up physical or file (iso) based cd / dvd images
<disk type='file' device='cdrom'>
<driver name='qemu'/>
<source file='/usr/local/vms/kvmnetworking.iso'/>
<target dev='hdc' bus='ide'/>
<readonly/>
</disk>
And rid yourself of the disk with this:
detach-disk hdc
or just right click eject under Windows guests (domains)
Q: Hot mount another disk file?
A: attach-disk, but modprobe first!
In the guest: modprobe acpiphp modprobe pci_hotplug Back on the vm host: virsh attach-disk guestdomname /path/to/diskimage/thisdiskimage vdc --cache none Now dmesg, and find your disk. If you need a rescan of the bus in the guest, do this: echo "- - -" > /sys/class/scsi_host/host0/scan Detach it (after un-mounting silly): virsh detach-disk guestdomname vdc (or whatever you called it in the attach above). This is only temporary as it does not write the config for reboots! I have had issues with putting back the same device name (probably older libvirt / virsh): error: Failed to attach disk error: internal error unable to execute QEMU command 'device_add': Duplicate ID 'virtio-disk1' for device Silly, but I was able to attach the disk using another target device name: (in this case vde): VIRSH # attach-disk guestdomname /path/to/newdiskfile vde --cache none Disk attached successfully - i.e., "You now have SCSI!"
Q: So how do you mount a virtual machine's disk image?
A: Use libguestfs! http://libguestfs.org/
Q: Having some trouble starting after iso image mount for the cdrom (i.e. virsh start machine - gives qemu: could not open disk image):
A:
Remove the cd rom image stanza from the xml file Re-define the image after the edit (unless the edit function in virsh is fixed by the time you try it - ha ha) virsh define /etc/libvirt/qemu/machine.xml Re-start the domain - virs start machine
Q: How to you use the generous donation of 1Gb network drivers by Redhat?
A: - Here is where to get them : http://www.linux-kvm.org/page/WindowsGuestDrivers/Download_Drivers
A: - Here is how to put them in after the fact; http://www.linux-kvm.com/content/redhat-54-windows-virtio-drivers-part-2-block-drivers
(if after the fact, see below to remove the old nic card if you happen to need to remove it later).
A:
Download the driver from them, change your xml definition for your bridge interface to have a <model type="virtio"/> stanza and then point the new hardware at the driver you just downloaded.
A:
Add a stanza for a fake qcow disk (not your boot disk), and give it a virtio bus type eg:
<target dev='hdb' bus='virtio'/>
Have the cd mounted in the drive from the iso image and have it add the driver for that new "hdd" and
then after you are finished, down the vm and change your boot drive to the same type of bus (virtio) and you should be good
Q: I am getting Error 39 when I try to install the virtio driver.
A: Don't just scan for the driver, select the correct one by path. Other drivers will install, but pick the one for your specific OS release!
Q: Now that you have the Red Hat network drivers installed under Windows, how do you remove the old one?
A: You need to enable the device manager to show the devices that are not installed:
set devmgr_show_nonpresent_devices=1 Run the device manager from that same elevated cmd session: start devmgmt.msc Then select View -> Show Hidden Devices. The old nic should be visible again.
Q: Network drops under Windows 2008 R2 guest.
A: You might want to disable the Network Location Awareness bit:
- change reg key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\NlaSvc\Parameters\Internet -> EnableActiveProbing From 1 to 0 (and of course reboot as this is MS).
If you are daring (or stupid) here is a reg file noprobe-stub.reg If it breaks your server, you get to keep both halves.
Seriously, don't apply that reg key unless you want to have no networking or worse folks!
A: Alternatively, do what Matt Shadbolt did! Pretty cool. Short circuit it with hosts file and an IIS site locally to answer!
A: Matt's answer: http://mattlog.net/2011/01/18/windows-vista72008-no-internet-access-network-status/
Q: How do you convert a raw dd image file to a sparse image file? line A: Try this:
qemu-img convert -O qcow2 windoze.img windoze.qcow2
Of course that won't work on Ubuntu with Apparmor (search above this post for help - or just turn that thing off... Mollyguard eh?).
Q: How to get the clock to use the real time clock with a Windows guest instead of the cpu based one:
A: If you find the slew is in hours, make sure you don't need to change the config xml to have the clock line read -> <clock offset='localtime'/> <- just below the features block
A: This is taken from the docs from Redhat here - http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Virtualization/chap-Virtualization-KVM_guest_timing_management.html
Using the Real-Time Clock with Windows Server 2003 and Windows XP guests
Windows uses the both the Real-Time Clock (RTC) and the Time Stamp Counter (TSC). For Windows guests the Real-Time Clock can be used instead of the TSC for all time sources which resolves guest timing issues.
To enable the Real-Time Clock for the PMTIMER clock source (the PMTIMER usually uses the TSC) add the following line to the Windows boot settings. Windows boot settings are stored in the boot.ini file. Add the following line to the boot.ini file:
/use pmtimer
For more information on Windows boot settings and the pmtimer option, refer to Available switch options for the Windows XP and the Windows Server 2003 Boot.ini files.
Using the Real-Time Clock with Windows Vista, Windows Server 2008 and Windows 7 guests
Windows uses the both the Real-Time Clock (RTC) and the Time Stamp Counter (TSC). For Windows guests the Real-Time Clock can be used instead of the TSC for all time sources which resolves guest timing issues.
The boot.ini file is no longer used from Windows Vista and newer. Windows Vista, Windows Server 2008 and Windows 7 use the Boot Configuration Data Editor (bcdedit.exe) to modify the Windows boot parameters.
This procedure is only required if the guest is having time keeping issues. Time keeping issues may not affect guests on all host systems.
1.
Open the Windows guest.
2.
Open the Accessories menu of the start menu. Right click on the Command Prompt application, select Run as Administrator.
3.
Confirm the security exception, if prompted.
4.
Set the boot manager to use the platform clock. This should instruct Windows to use the PM timer for the primary clock source. The system UUID ({default} in the example below) should be changed if the system UUID is different than the default boot device.
C:\Windows\system32>bcdedit /set {default} USEPLATFORMCLOCK on
The operation completed successfully
This fix should improve time keeping for Windows Vista, Windows Server 2008 and Windows 7 guests.
Q: Disk writes slow?:
A: Make sure you are using the virtio drivers (Windows and Linux guests), and you might want to consider a cache change to the virtual disk:
The cache types and the virtio bus are what you need to change up:
(Warning!!!: cache="none" is not recommended as you may destroy the disk image on loss of power. Check on the other types before trying anything this risky!
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none'/>
<source file='/usr/local/vms/dcps'/>
<target dev='vda' bus='virtio'/>
</disk>
As I seem to have a a bug in my install, I can't make changes in the config stick on a reboot, only on a full shutdown, edit and then bring up the vm.