[one-users] KVM drivers fail to load? [MAD did not answer INIT command]

Sat Apr 21 02:13:53 PDT 2012

Hi,

I've found out that the reason for my problems was that libvirtd was continuously crashing at start-up on my OpenNebula Front-End. Since I didn't manage to repair it so far, for the time being, I disabled it completely, and this seemed to make the error disappear, that is, my KVM drivers are loaded successfully ever since.

However, I got a question here, which might be naive, but I got really curious now: shouldn't the hypervisor, virtualization drivers, libvirt settings and all the related stuff be affecting OpenNebula only in case of the hosts? How (and why) does the fact that libvirt is crashing on my front-end affect the ONE system?

Thanks,
Lehel.

--- On Thu, 4/19/12, biro lehel <lehel.biro at yahoo.com> wrote:

From: biro lehel <lehel.biro at yahoo.com>
Subject: Re: [one-users] KVM drivers fail to load? [MAD did not answer INIT command]
To: "Ruben S. Montero" <rsmontero at opennebula.org>
Cc: users at lists.opennebula.org
Date: Thursday, April 19, 2012, 10:50 PM

Hi Ruben,

No, I'm running OpenNebula on a 'normal' physical machine, with which I didn't have problems so far. Apart from heavy load, IYHO, could anything else be possibly causing such malfunction?

Cheers,
Lehel.

--- On Thu, 4/19/12, Ruben S. Montero <rsmontero at opennebula.org> wrote:

From: Ruben S. Montero <rsmontero at opennebula.org>
Subject: Re: [one-users] KVM drivers fail to load? [MAD did not answer INIT command]
To: "biro lehel" <lehel.biro at yahoo.com>
Cc: users at lists.opennebula.org
Date: Thursday, April 19, 2012, 3:35 PM

Hi,
Are you running the OpenNebula  on a VM? We've seen this type of error when OpenNebula is started under
 heavy-load.... (usually in a VM low on CPU resources)
Cheers

Ruben

On Thu, Apr 19, 2012 at 11:28 AM, biro lehel <lehel.biro at yahoo.com> wrote:

Dear all,

In my ONE system, sometimes, after the front-end's startup, I've noticed that the KVM drivers (vmm_kvm and im_kvm) are not able to load. At first I didn't understand what could be causing such ONE malfunction, but soon I tracked back all related errors and found the answer in the very initial part of oned.log, which I copied at the end of this mail.

It seems to be a strange issue, since it apparently happens rather randomly, sometimes a reboot of the front-end makes the issue disappear, but sometimes the error just sticks.

I'm aware that it might possibly not be an ONE issue. However, I would be extremely grateful if you could provide some hints or point-outs to where should I at least be looking for the cause of this problem. Is this a front-end issue? Is this an issue with the respective host? Both have KVM installed and I've
 performed virtualizations on both of them (with virsh+qemu) since my ONE infrastructure is up. Are there any logs which I could check to possibly find more detail?

Here is the initial part of oned.log, highlighted with the errors:

Wed Apr 18 16:24:58 2012 [ONE][I]: Starting OpenNebula 3.2.0
----------------------------------------
     OpenNebula Configuration File      
----------------------------------------
AUTH_MAD=ARGUMENTS=--authn ssh,x509,ldap,server_cipher,server_x509,EXECUTABLE=one_auth_mad

DB=BACKEND=sqlite
DEBUG_LEVEL=3
DEFAULT_DEVICE_PREFIX=hd
DEFAULT_IMAGE_TYPE=OS
ENABLE_OTHER_PERMISSIONS=YES
HM_MAD=EXECUTABLE=one_hm
HOST_MONITORING_INTERVAL=600
HOST_PER_INTERVAL=15
IMAGE_MAD=ARGUMENTS=fs -t 15,EXECUTABLE=one_image

IM_MAD=ARGUMENTS=-r 0 -t 15
 kvm,EXECUTABLE=one_im_ssh,NAME=im_kvm
MAC_PREFIX=02:00
MANAGER_TIMER=15
NETWORK_SIZE=254
PORT=2633
SCRIPTS_REMOTE_DIR=/var/tmp/one
SESSION_EXPIRATION_TIME=900
TM_MAD=ARGUMENTS=tm_shared/tm_shared.conf,EXECUTABLE=one_tm,NAME=tm_shared

VM_DIR=/var/lib/one/
VM_MAD=ARGUMENTS=-t 15 -r 0 kvm,DEFAULT=vmm_exec/vmm_exec_kvm.conf,EXECUTABLE=one_vmm_exec,NAME=vmm_kvm,TYPE=kvm
VM_PER_INTERVAL=5
VM_POLLING_INTERVAL=600
VNC_BASE_PORT=5900
----------------------------------------

Wed Apr 18 16:24:58 2012 [ONE][I]: Log level:3 [0=ERROR,1=WARNING,2=INFO,3=DEBUG]
Wed Apr 18 16:24:58 2012 [ONE][I]: Checking database version.
Wed Apr 18 16:24:58 2012 [VMM][I]: Starting Virtual Machine Manager...

Wed Apr 18 16:24:58 2012 [VMM][I]: Virtual Machine Manager started.
Wed Apr 18 16:24:58 2012 [LCM][I]: Starting Life-cycle Manager...
Wed Apr 18 16:24:58 2012 [LCM][I]: Life-cycle Manager started.
Wed Apr 18 16:24:58 2012
 [InM][I]: Starting Information Manager...
Wed Apr 18 16:24:58 2012 [InM][I]: Information Manager started.
Wed Apr 18 16:24:58 2012 [TrM][I]: Starting Transfer Manager...
Wed Apr 18 16:24:58 2012 [TrM][I]: Transfer Manager started.

Wed Apr 18 16:24:58 2012 [DiM][I]: Starting Dispatch Manager...
Wed Apr 18 16:24:58 2012 [DiM][I]: Dispatch Manager started.
Wed Apr 18 16:24:58 2012 [HKM][I]: Starting Hook Manager...
Wed Apr 18 16:24:58 2012 [HKM][I]: Hook Manager started.

Wed Apr 18 16:24:58 2012 [AuM][I]: Starting Auth Manager...
Wed Apr 18 16:24:58 2012 [AuM][I]: Authorization Manager started.
Wed Apr 18 16:24:58 2012 [ImM][I]: Starting Image Manager...
Wed Apr 18 16:24:58 2012 [ImM][I]: Image Manager started.

Wed Apr 18 16:24:58 2012 [ReM][I]: Starting Request Manager...
Wed Apr 18 16:24:58 2012 [ReM][I]: Request Manager started.
Wed Apr 18 16:24:58 2012 [ReM][I]: Starting XML-RPC server, port 2633 ...
Wed Apr
 18 16:25:00 2012 [VMM][I]: Loading Virtual Machine Manager drivers.
Wed Apr 18 16:25:00 2012 [VMM][I]:     Loading driver: vmm_kvm (KVM)
Wed Apr 18 16:25:05 2012 [MAD][E]: MAD did not answer INIT command

Wed Apr 18 16:25:05 2012 [InM][I]: Loading Information Manager drivers.
Wed Apr 18 16:25:05 2012 [InM][I]:     Loading driver: im_kvm
Wed Apr 18 16:25:10 2012 [MAD][E]: MAD did not answer INIT command

Wed Apr 18 16:25:10 2012 [TM][I]: Loading Transfer Manager drivers.
Wed Apr 18 16:25:11 2012 [VMM][I]:     Loading driver: tm_shared
Wed Apr 18 16:25:12 2012 [TM][I]:     Driver tm_shared loaded.
Wed Apr 18 16:25:12 2012 [HKM][I]: Loading Hook Manager
 driver.
Wed Apr 18 16:25:13 2012 [HKM][I]:     Hook Manager loaded
Wed Apr 18 16:25:13 2012 [ImM][I]: Loading Image Manager driver.
Wed Apr 18 16:25:13 2012 [InM][I]: Monitoring host pcls (6)
Wed Apr 18 16:25:13 2012 [InM][E]: Could not find information driver im_kvm

Sometimes, only vmm_kvm fails to load, while im_kvm loads successfully, which is another twist to the story.

Any suggestion would be highly appreciated.

Thank you,
Lehel.

_______________________________________________

Users mailing list

Users at lists.opennebula.org

http://lists.opennebula.org/listinfo.cgi/users-opennebula.org

-- 
Ruben S. Montero, PhD
Project co-Lead and Chief Architect
OpenNebula - The Open Source Solution for Data Center Virtualization
www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula

-----Inline Attachment Follows-----

_______________________________________________
Users mailing list
Users at lists.opennebula.org
http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20120421/72e64726/attachment-0002.htm>