[one-users] KVM Hypervisor Failed Image Start

Ryan Aydelott ryan.aydelott at vibes.com
Tue Oct 18 14:47:40 PDT 2011 

Javier - I did note the following on the hypervisor /var/log/messages on restart:

Oct 18 11:05:52 localhost libvirtd: 11:05:52.794: error : virRunWithHook:856 : internal error '/sbin/iptables --table filter --delete INPUT --in-interface virbr0 --protocol udp --destination-port 69 --jump ACCEPT' exited with non-zero status 1 and signal 0: iptables: Bad rule (does a matching rule exist in that chain?)
Oct 18 11:05:53 localhost libvirtd: 11:05:53.211: warning : qemudStartup:1667 : Unable to create cgroup for driver: No such device or address

The only reason I did not investigate further is that I was able to start hosts locally using virsh/xml.

This would seem to imply to me that libvirtd wouldn't be listening (which could be the whole source of the problem?)


RYAN AYDELOTT | SR. SYSTEMS/NETWORK ADMIN | VIBES MEDIA
300 W Adams St, Floor 7 | Chicago, IL 60606
312-589-6931 (o) | 312-388-2929 (m)
ryan.aydelott at vibes.com<mailto:ryan.aydelott at vibes.com> | www.vibes.com<http://www.vibes.com/>

On Oct 18, 2011, at 4:37 PM, Javier Fontan wrote:

I don't see anything strange in the deployment file and with so little
information from libvirt I would create a new VM with the same
paremeters but without vnc or virtio. I can only see those as the
cause for non descriptive problems.

Also as the libvirt FAQ [1] says check /var/log/messages for some info.

[1] http://wiki.libvirt.org/page/FAQ#error:_Unknown_Failure

On Tue, Oct 18, 2011 at 11:13 PM, Ryan Aydelott <ryan.aydelott at vibes.com<mailto:ryan.aydelott at vibes.com>> wrote:
As requested:
[root at kvm02 10]# cat deployment.1
<domain type='kvm'>
<name>one-10</name>
<vcpu>4</vcpu>
<memory>4194304</memory>
<os>
<type arch='x86_64'>hvm</type>
<boot dev='cdrom'/>
</os>
<devices>
<emulator>/usr/bin/kvm</emulator>
<disk type='file' device='cdrom'>
<source file='/srv/cloud/one/var//10/images/disk.0'/>
<target dev='cdromc' bus='ide'/>
<readonly/>
<driver name='qemu' type='raw' cache='default'/>
</disk>
<disk type='file' device='disk'>
<source file='/srv/cloud/one/var//10/images/disk.1'/>
<target dev='hde' bus='virtio'/>
<driver name='qemu' type='raw' cache='default'/>
</disk>
<graphics type='vnc' listen='172.20.90.21' port='5911'/>
</devices>
<features>
<acpi/>
</features>
[root at kvm02 10]# cat deployment.2
<domain type='kvm'>
<name>one-10</name>
<vcpu>4</vcpu>
<memory>4194304</memory>
<os>
<type arch='x86_64'>hvm</type>
<boot dev='cdrom'/>
</os>
<devices>
<emulator>/usr/bin/kvm</emulator>
<disk type='file' device='cdrom'>
<source file='/srv/cloud/one/var//10/images/disk.0'/>
<target dev='cdromc' bus='ide'/>
<readonly/>
<driver name='qemu' type='raw' cache='default'/>
</disk>
<disk type='file' device='disk'>
<source file='/srv/cloud/one/var//10/images/disk.1'/>
<target dev='hde' bus='virtio'/>
<driver name='qemu' type='raw' cache='default'/>
</disk>
<graphics type='vnc' listen='172.20.90.21' port='5911'/>
</devices>
<features>
<acpi/>
</features>
</domain>
[root at kvm02 10]# virsh list --inactive
 Id Name                 State
----------------------------------

RYAN AYDELOTT | SR. SYSTEMS/NETWORK ADMIN | VIBES MEDIA
300 W Adams St, Floor 7 | Chicago, IL 60606
312-589-6931 (o) | 312-388-2929 (m)
ryan.aydelott at vibes.com<mailto:ryan.aydelott at vibes.com> | www.vibes.com<http://www.vibes.com>
On Oct 18, 2011, at 4:01 PM, Javier Fontan wrote:

Can you send us the deployment files (I can see both deployment.1 and
deployment.2) files? Also go to the node and execute

# virsh list --inactive

as root and check that there are no inactive VMs defined.

Cheers

On Tue, Oct 18, 2011 at 6:28 PM, Ryan Aydelott <ryan.aydelott at vibes.com<mailto:ryan.aydelott at vibes.com>>
wrote:

I am having an issue with bootstrapping an image into a KVM hypervisor, I

have had success with Xen prior to this, but now trying KVM. Some basic

info:

OpenNebula 3.0

libvirtd on KVM (Centos 5.6) is 0.8.2

Shared NFS Filesystem

oneadmin account adding to KVM/QEMU groups

onehost successfully monitored:

Tue Oct 18 11:23:11 2011 [ReM][D]: HostPoolInfo method invoked

  ID NAME               RVM   TCPU   FCPU   ACPU   TMEM   FMEM   AMEM   STAT

   0 kvm01                0    400    397    400  31.4G    17G  31.4G     on

   6 kvm02                0    400    400    400  31.4G  30.8G  31.4G     on

I can successfully start KVM hosts via Virsh using the command line. However

when I attempt to do so using OpenNebula it fails. It may be important to

note that in /var/log/libvirt/qemu where I would normally see log files for

one-5.log, one-6.log, etc - no file is created for this instance. This leads

me to believe that kvm is not even being called on the machine.

I'm hoping this is just a really dumb beginners mistake. If anyone has any

suggestions - let me know. Thanks!

Information on hypervisor for oned.log is as follows:

Tue Oct 18 11:14:52 2011 [TM][D]: Message received: LOG D 10 tm_clone.sh:

opennebula:/srv/cloud/one/var/images/c38ebcb9c8b7d7f658feb6e9e62653e0

kvm02:/srv/cloud/one/var//10/images/disk.1

Tue Oct 18 11:14:52 2011 [TM][D]: Message received: LOG D 10 tm_clone.sh:

DST: /srv/cloud/one/var//10/images/disk.1

Tue Oct 18 11:14:52 2011 [TM][D]: Message received: LOG I 10 tm_clone.sh:

Creating directory /srv/cloud/one/var//10/images

Tue Oct 18 11:14:52 2011 [TM][D]: Message received: LOG I 10 tm_clone.sh:

Executed "mkdir -p /srv/cloud/one/var//10/images".

Tue Oct 18 11:14:52 2011 [TM][D]: Message received: LOG I 10 tm_clone.sh:

Executed "chmod a+w /srv/cloud/one/var//10/images".

Tue Oct 18 11:14:52 2011 [TM][D]: Message received: LOG I 10 tm_clone.sh:

Cloning /srv/cloud/one/var/images/c38ebcb9c8b7d7f658feb6e9e62653e0

Tue Oct 18 11:14:52 2011 [TM][D]: Message received: LOG I 10 tm_clone.sh:

Executed "cp -r /srv/cloud/one/var/images/c38ebcb9c8b7d7f658feb6e9e62653e0

/srv/cloud/one/var//10/images/disk.1".

Tue Oct 18 11:14:52 2011 [TM][D]: Message received: LOG I 10 tm_clone.sh:

Executed "chmod a+rw /srv/cloud/one/var//10/images/disk.1".

Tue Oct 18 11:14:52 2011 [TM][D]: Message received: LOG I 10 ExitCode: 0

Tue Oct 18 11:14:52 2011 [TM][D]: Message received: TRANSFER SUCCESS 10 -

Tue Oct 18 11:14:53 2011 [VMM][D]: Message received: LOG I 10 Command

execution fail: 'if [ -x "/var/tmp/one/vmm/kvm/deploy" ]; then

/var/tmp/one/vmm/kvm/deploy /srv/cloud/one/var//10/images/deployment.2 kvm02

10 kvm02; else                              exit 42; fi'

Tue Oct 18 11:14:53 2011 [VMM][D]: Message received: LOG I 10 error: Failed

to create domain from /srv/cloud/one/var//10/images/deployment.2

Tue Oct 18 11:14:53 2011 [VMM][D]: Message received: LOG I 10 error: Unknown

failure

Tue Oct 18 11:14:53 2011 [VMM][D]: Message received: LOG E 10 Could not

create domain from /srv/cloud/one/var//10/images/deployment.2

Tue Oct 18 11:14:53 2011 [VMM][D]: Message received: LOG I 10 ExitCode: 255

Tue Oct 18 11:14:53 2011 [VMM][D]: Message received: DEPLOY FAILURE 10 Could

not create domain from /srv/cloud/one/var//10/images/deployment.2

vm.log is as follows:

Tue Oct 18 10:50:42 2011 [DiM][I]: New VM state is ACTIVE.

Tue Oct 18 10:50:43 2011 [LCM][I]: New VM state is PROLOG.

Tue Oct 18 10:50:43 2011 [VM][I]: Virtual Machine has no context

Tue Oct 18 10:56:09 2011 [TM][D]: tm_clone.sh:

opennebula:/srv/cloud/images/rhel-server-5.4-x86_64-dvd.iso

kvm02:/srv/cloud/one/var//10/images/disk.0

Tue Oct 18 10:56:09 2011 [TM][D]: tm_clone.sh: DST:

/srv/cloud/one/var//10/images/disk.0

Tue Oct 18 10:56:09 2011 [TM][I]: tm_clone.sh: Creating directory

/srv/cloud/one/var//10/images

Tue Oct 18 10:56:09 2011 [TM][I]: tm_clone.sh: Executed "mkdir -p

/srv/cloud/one/var//10/images".

Tue Oct 18 10:56:09 2011 [TM][I]: tm_clone.sh: Executed "chmod a+w

/srv/cloud/one/var//10/images".

Tue Oct 18 10:56:09 2011 [TM][I]: tm_clone.sh: Cloning

/srv/cloud/images/rhel-server-5.4-x86_64-dvd.iso

Tue Oct 18 10:56:09 2011 [TM][I]: tm_clone.sh: Executed "cp -r

/srv/cloud/images/rhel-server-5.4-x86_64-dvd.iso

/srv/cloud/one/var//10/images/disk.0".

Tue Oct 18 10:56:09 2011 [TM][I]: tm_clone.sh: Executed "chmod a+rw

/srv/cloud/one/var//10/images/disk.0".

Tue Oct 18 10:56:09 2011 [TM][I]: ExitCode: 0

Tue Oct 18 11:01:13 2011 [TM][D]: tm_clone.sh:

opennebula:/srv/cloud/one/var/images/c38ebcb9c8b7d7f658feb6e9e62653e0

kvm02:/srv/cloud/one/var//10/images/disk.1

Tue Oct 18 11:01:13 2011 [TM][D]: tm_clone.sh: DST:

/srv/cloud/one/var//10/images/disk.1

Tue Oct 18 11:01:13 2011 [TM][I]: tm_clone.sh: Creating directory

/srv/cloud/one/var//10/images

Tue Oct 18 11:01:13 2011 [TM][I]: tm_clone.sh: Executed "mkdir -p

/srv/cloud/one/var//10/images".

Tue Oct 18 11:01:13 2011 [TM][I]: tm_clone.sh: Executed "chmod a+w

/srv/cloud/one/var//10/images".

Tue Oct 18 11:01:13 2011 [TM][I]: tm_clone.sh: Cloning

/srv/cloud/one/var/images/c38ebcb9c8b7d7f658feb6e9e62653e0

Tue Oct 18 11:01:13 2011 [TM][I]: tm_clone.sh: Executed "cp -r

/srv/cloud/one/var/images/c38ebcb9c8b7d7f658feb6e9e62653e0

/srv/cloud/one/var//10/images/disk.1".

Tue Oct 18 11:01:13 2011 [TM][I]: tm_clone.sh: Executed "chmod a+rw

/srv/cloud/one/var//10/images/disk.1".

Tue Oct 18 11:01:13 2011 [TM][I]: ExitCode: 0

Tue Oct 18 11:01:14 2011 [LCM][I]: New VM state is BOOT

Tue Oct 18 11:01:14 2011 [VMM][I]: Generating deployment file:

/srv/cloud/one/var/10/deployment.1

Tue Oct 18 11:01:14 2011 [VMM][I]: Command execution fail: 'if [ -x

"/var/tmp/one/vmm/kvm/deploy" ]; then /var/tmp/one/vmm/kvm/deploy

/srv/cloud/one/var//10/images/deployment.1 kvm02 10 kvm02; else

exit 42; fi'

Tue Oct 18 11:01:14 2011 [VMM][I]: error: Failed to create domain from

/srv/cloud/one/var//10/images/deployment.1

Tue Oct 18 11:01:14 2011 [VMM][I]: error: Unknown failure

Tue Oct 18 11:01:14 2011 [VMM][E]: Could not create domain from

/srv/cloud/one/var//10/images/deployment.1

Tue Oct 18 11:01:14 2011 [VMM][I]: ExitCode: 255

Tue Oct 18 11:01:14 2011 [VMM][E]: Error deploying virtual machine: Could

not create domain from /srv/cloud/one/var//10/images/deployment.1

Tue Oct 18 11:01:17 2011 [DiM][I]: New VM state is FAILED

RYAN AYDELOTT | SR. SYSTEMS/NETWORK ADMIN | VIBES MEDIA

300 W Adams St, Floor 7 | Chicago, IL 60606

312-589-6931 (o) | 312-388-2929 (m)

ryan.aydelott at vibes.com<mailto:ryan.aydelott at vibes.com> | www.vibes.com<http://www.vibes.com>

_______________________________________________

Users mailing list

Users at lists.opennebula.org<mailto:Users at lists.opennebula.org>

http://lists.opennebula.org/listinfo.cgi/users-opennebula.org





--
Javier Fontán Muiños
Project Engineer
OpenNebula Toolkit | opennebula.org<http://opennebula.org>


_______________________________________________
Users mailing list
Users at lists.opennebula.org<mailto:Users at lists.opennebula.org>
http://lists.opennebula.org/listinfo.cgi/users-opennebula.org





--
Javier Fontán Muiños
Project Engineer
OpenNebula Toolkit | opennebula.org<http://opennebula.org>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20111018/fa8eb5d2/attachment-0002.htm>

More information about the Users mailing list