<html>
<head>
<meta content="text/html; charset=ISO-8859-2"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<small><big>Maybe bingo?</big><br>
</small>
<pre wrap=""><big>In /etc/apparmor.d/usr.sbin.libvirtd I have set (on ONE-server and host too):
/var/lib/one/** lrwk,</big></pre>
and /var/lib/syslog on host is saying (at deployment time):<br>
<small><small><br>
Jun 20 15:10:16 opennebula-host kernel: [11202.067916] type=1400
audit(1340197816.112:73): apparmor="STATUS"
operation="profile_load"
name="libvirt-ffa4b319-b4fb-c261-92ee-4ccdd4f06b9b" pid=9080
comm="apparmor_parser"<br>
Jun 20 15:10:16 opennebula-host kernel: [11202.591541] type=1400
audit(1340197816.636:74): apparmor="DENIED" operation="open"
parent=1 profile="libvirt-ffa4b319-b4fb-c261-92ee-4ccdd4f06b9b"
name="/var/lib/one/images/9c52b90a79dba7c26a912d05ff5190b8"
pid=9085 comm="kvm" requested_mask="r" denied_mask="r" fsuid=108
ouid=108<br>
Jun 20 15:10:16 opennebula-host kernel: [11202.592449] type=1400
audit(1340197816.640:75): apparmor="DENIED" operation="open"
parent=1 profile="libvirt-ffa4b319-b4fb-c261-92ee-4ccdd4f06b9b"
name="/var/lib/one/images/9c52b90a79dba7c26a912d05ff5190b8"
pid=9085 comm="kvm" requested_mask="r" denied_mask="r" fsuid=108
ouid=108<br>
Jun 20 15:10:16 opennebula-host kernel: [11202.593430] type=1400
audit(1340197816.640:76): apparmor="DENIED" operation="open"
parent=1 profile="libvirt-ffa4b319-b4fb-c261-92ee-4ccdd4f06b9b"
name="/var/lib/one/images/9c52b90a79dba7c26a912d05ff5190b8"
pid=9085 comm="kvm" requested_mask="rw" denied_mask="rw"
fsuid=108 ouid=108<br>
Jun 20 15:10:17 opennebula-host kernel: [11203.282562] type=1400
audit(1340197817.328:77): apparmor="STATUS"
operation="profile_remove"
name="libvirt-ffa4b319-b4fb-c261-92ee-4ccdd4f06b9b" pid=9088
comm="apparmor_parser"<br>
</small></small><br>
<br>
after /etc/init.d/apparmor teardown syslog is saying the same (at
deployment time):<br>
<br>
<small><small>Jun 20 15:13:16 opennebula-host kernel: [11382.242000]
type=1400 audit(1340197996.288:84): apparmor="STATUS"
operation="profile_load"
name="libvirt-e57f7c9b-8d8c-ad64-a87d-cde092e83eb1" pid=9281
comm="apparmor_parser"<br>
Jun 20 15:13:16 opennebula-host kernel: [11382.867109] type=1400
audit(1340197996.912:85): apparmor="DENIED" operation="open"
parent=1 profile="libvirt-e57f7c9b-8d8c-ad64-a87d-cde092e83eb1"
name="/var/lib/one/images/9c52b90a79dba7c26a912d05ff5190b8"
pid=9286 comm="kvm" requested_mask="r" denied_mask="r" fsuid=108
ouid=108<br>
Jun 20 15:13:16 opennebula-host kernel: [11382.867866] type=1400
audit(1340197996.912:86): apparmor="DENIED" operation="open"
parent=1 profile="libvirt-e57f7c9b-8d8c-ad64-a87d-cde092e83eb1"
name="/var/lib/one/images/9c52b90a79dba7c26a912d05ff5190b8"
pid=9286 comm="kvm" requested_mask="r" denied_mask="r" fsuid=108
ouid=108<br>
Jun 20 15:13:16 opennebula-host kernel: [11382.868606] type=1400
audit(1340197996.916:87): apparmor="DENIED" operation="open"
parent=1 profile="libvirt-e57f7c9b-8d8c-ad64-a87d-cde092e83eb1"
name="/var/lib/one/images/9c52b90a79dba7c26a912d05ff5190b8"
pid=9286 comm="kvm" requested_mask="rw" denied_mask="rw"
fsuid=108 ouid=108<br>
Jun 20 15:13:17 opennebula-host kernel: [11383.551792] type=1400
audit(1340197997.596:88): apparmor="STATUS"
operation="profile_remove"
name="libvirt-e57f7c9b-8d8c-ad64-a87d-cde092e83eb1" pid=9289
comm="apparmor_parser"<br>
</small></small><br>
On server machine (where one is running) apparmon is saying nothing
at time of deployment (setting is the same).<br>
<small><small>Jun 20 12:17:01 nebula-3 CRON[19424]: (root) CMD (
cd / && run-parts --report /etc/cron.hourly)<br>
Jun 20 12:30:56 nebula-3 dhclient: DHCPREQUEST of 10.0.1.125 on
eth0 to 10.0.10.12 port 67<br>
Jun 20 12:30:56 nebula-3 dhclient: DHCPACK of 10.0.1.125 from
10.0.10.12<br>
Jun 20 12:30:56 nebula-3 dhclient: bound to 10.0.1.125 --
renewal in 8162 seconds.<br>
Jun 20 13:17:01 nebula-3 CRON[22347]: (root) CMD ( cd /
&& run-parts --report /etc/cron.hourly)<br>
</small></small><br>
Why?<br>
And what I can do?<br>
<br>
Jan<br>
<br>
Dňa 20.06.2012 12:55, Javier Fontan wrote / napísal(a):
<blockquote
cite="mid:CAK+uMM8D8M+qs_6xsbjM15zXv2KvFW5wE_uirfgbF73goOSTXA@mail.gmail.com"
type="cite">
<pre wrap="">Can you check that you are not getting apparmor error messages in
/var/log/syslog at the time of VM deployment?
On Wed, Jun 20, 2012 at 12:23 PM, Jan Benadik <a class="moz-txt-link-rfc2396E" href="mailto:jan.benadik@atos.net"><jan.benadik@atos.net></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
And of course - libvirtd daemon is restarted and running on both machines.
oneadmin@opennebula-host:~$ ps aux|grep libv
root 1010 0.0 0.0 852624 6612 ? Sl 12:03 0:00 /usr/sbin/libvirtd -d -l
106 1107 0.0 0.0 25964 992 ? S 12:03 0:00 /usr/sbin/dnsmasq -u libvirt-dnsmasq --strict-order --bind-interfaces --pid-file=/var/run/libvirt/network/default.pid --conf-file= --except-interface lo --listen-address 192.168.122.1 --dhcp-range 192.168.122.2,192.168.122.254 --dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases --dhcp-lease-max=253 --dhcp-no-override
oneadmin 7053 0.0 0.0 9352 652 pts/0 S+ 14:23 0:00 grep libv
oneadmin@opennebula-host:~$
oneadmin@nebula-3:~$ ps aux|grep libv
106 2439 0.0 0.0 25964 928 ? S 10:09 0:00 /usr/sbin/dnsmasq -u libvirt-dnsmasq --strict-order --bind-interfaces --pid-file=/var/run/libvirt/network/default.pid --conf-file= --except-interface lo --listen-address 192.168.122.1 --dhcp-range 192.168.122.2,192.168.122.254 --dhcp-leasefile=/var/lib/libvirt/dnsmasq/default.leases --dhcp-lease-max=253 --dhcp-no-override
root 19329 0.0 0.1 262560 5864 ? Sl 12:14 0:00 /usr/sbin/libvirtd -d -l
oneadmin 19659 0.0 0.0 8072 648 pts/1 S+ 12:22 0:00 grep libv
oneadmin@nebula-3:~$
Jan
----- Pôvodná správa -----
Predmet: Re: [one-users] Error when instantiating VM from image
Dátum: Wed, 20 Jun 2012 12:15:01 +0200
Od: Jan Benadik <a class="moz-txt-link-rfc2396E" href="mailto:jan.benadik@atos.net"><jan.benadik@atos.net></a>
Odpoveď komu: <a class="moz-txt-link-abbreviated" href="mailto:jan.benadik@atos.net">jan.benadik@atos.net</a>
Organizácia: Atos IT Solutions and Services s.r.o.
Pre: Jaime Melis <a class="moz-txt-link-rfc2396E" href="mailto:jmelis@opennebula.org"><jmelis@opennebula.org></a>
Kópia: cloud.b.lab <a class="moz-txt-link-rfc2396E" href="mailto:cloud.b.lab@zoho.com"><cloud.b.lab@zoho.com></a>, <a class="moz-txt-link-abbreviated" href="mailto:users@lists.opennebula.org">users@lists.opennebula.org</a>
oneadminoneadmin@nebula-3:~$ ls -l `readlink -f /var/lib/one/0/images/disk.1`
-rw-rw---- 1 oneadmin root 10737418241 Jun 20 10:57 /var/lib/one/images/9c52b90a79dba7c26a912d05ff5190b8
oneadmin@nebula-3:~$ id
uid=108(oneadmin) gid=115(cloud) groups=115(cloud),6(disk),105(kvm),111(libvirtd)
oneadmin@nebula-3:~$ grep -Ev '^($|#)' /etc/libvirt/qemu.conf
user = "oneadmin"
group = "cloud"
dynamic_ownership = 0
oneadmin@nebula-3:~$
Dňa 20.06.2012 11:23, Jaime Melis wrote / napísal(a):
Hello Jan,
can you please revert to your initial conf (dyn_ownership = 0) and send us:
# something like this (the disk will have probably changed by now):
$ ls -l `readlink -f /var/lib/one/5/images/disk.1`
and:
$ id
$ grep -Ev '^($|#)' /etc/libvirt/qemu.conf
can you confirm that libvirtd is running and restarted?
regards,
Jaime
On Wed, Jun 20, 2012 at 11:14 AM, Jan Benadik <a class="moz-txt-link-rfc2396E" href="mailto:jan.benadik@atos.net"><jan.benadik@atos.net></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">
ad1)
- doesn't help, libvirtd daemon didn't start (dnsmasq only)
- yes - it is owned by oneadmin (but this is link only, original file is owned by oneadmin too)
ad2) - doesn't help
Still the same error message.
Jan
Dňa 20.06.2012 09:12, cloud.b.lab wrote / napísal(a):
Jan,
This reply is from a ONE user.
Try after making following change:
1) In /etc/libvirt/libvirtd.conf set :
#unix_sock_group = "libvirtd"
unix_sock_group = "oneadmin"
Restart Libvirt-bin.
Also just check if the ownership of /var/lib/one/5/images/disk.1 is with oneadmin.
If that does't help try with following too
2) In /etc/libvirt/qemu.conf I have set
dynamic_ownership = 1
Regards,
Anil.
---- On Tue, 19 Jun 2012 23:32:43 -0700 Jan Benadik<a class="moz-txt-link-rfc2396E" href="mailto:jan.benadik@atos.net"><jan.benadik@atos.net></a> wrote ----
Thanks for reply, but it doesn't help.
Still the same result ...:-(
Jan
Dňa 19.06.2012 19:06, Jaime Melis wrote / napísal(a):
Hello,
You probably need to add oneadmin to the disk group.
Let us know if that doesn't work.
Cheers,
Jaime
On Mon, Jun 18, 2012 at 12:44 PM, Jan Benadik <a class="moz-txt-link-rfc2396E" href="mailto:jan.benadik@atos.net"><jan.benadik@atos.net></a> wrote:
Hi all,
I have two machines with Ubuntu 12.04 Server ("central" and "host"), KVM hypervisor, Opennebula 3.2.1, shared /var/lib/one folder (nfs)
User "oneadmin" and group "cloud" has the same uid and gid on both systems, user oneadmin is a member of kvm and libvirtd group too.
In /etc/libvirt/libvirtd.conf I have set :
listen_tls = 0
listen_tcp = 1
unix_sock_group = "libvirtd"
unix_sock_ro_perms = "0777"
unix_sock_rw_perms = "0777"
unix_sock_dir = "/var/run/libvirt"
auth_unix_ro = "none"
auth_unix_rw = "none"
In /etc/libvirt/qemu.conf I have set :
- user = "oneadmin"
- group = "cloud"
- dynamic_ownership = 0
In /etc/apparmor.d/usr.sbin.libvirtd I have set:
/var/lib/one/** lrwk,
Daemon libvirtd is running on both machines.
Permissions for /var/lib/one folder are:
drwxr-xr-x 15 oneadmin root 4096 June 18 10:46 one
Permissions of folder /var/lib/one/images are:
drwsrws--T 2 oneadmin cloud 4096 June 18 10:46 images
Permissions of images are:
-rw-rw---- 1 oneadmin cloud 688914432 June 18 10:46 e9203521a014fd8045d64206277acaa6f
-rw-rw---- 1 oneadmin cloud 10737418241 June 18 10:46 6f2589756c6432563546cc36543c55465
Monitoring of host is working, but if I want to start VM, the folloving error is in /var/log/one/oned.log:
Mon Jun 18 10:17:56 2012 [DiM][I]: New VM state is ACTIVE.
Mon Jun 18 10:17:57 2012 [LCM][I]: New VM state is PROLOG.
Mon Jun 18 10:17:57 2012 [VM][I]: Virtual Machine has no context
Mon Jun 18 10:17:58 2012 [TM][D]: tm_clone.sh: seed:/var/lib/one/images/e9203521a14fd8045d64206277acaa6f myto:/var/lib/one/5/images/disk.0
Mon Jun 18 10:17:58 2012 [TM][D]: tm_clone.sh: DST: /var/lib/one/5/images/disk.0
Mon Jun 18 10:17:58 2012 [TM][I]: tm_clone.sh: Creating directory /var/lib/one/5/images
Mon Jun 18 10:17:58 2012 [TM][I]: tm_clone.sh: Executed "mkdir -p /var/lib/one/5/images".
Mon Jun 18 10:17:58 2012 [TM][I]: tm_clone.sh: Executed "chmod a+w /var/lib/one/5/images".
Mon Jun 18 10:17:58 2012 [TM][I]: tm_clone.sh: Cloning /var/lib/one/images/e9203521a14fd8045d64206277acaa6f
Mon Jun 18 10:17:58 2012 [TM][I]: tm_clone.sh: Executed "cp -r /var/lib/one/images/e9203521a14fd8045d64206277acaa6f /var/lib/one/5/images/disk.0".
Mon Jun 18 10:17:58 2012 [TM][I]: tm_clone.sh: Executed "chmod a+rw /var/lib/one/5/images/disk.0".
Mon Jun 18 10:17:58 2012 [TM][I]: ExitCode: 0
Mon Jun 18 10:17:58 2012 [TM][I]: tm_ln.sh: Creating directory /var/lib/one/5/images
Mon Jun 18 10:17:58 2012 [TM][I]: tm_ln.sh: Executed "mkdir -p /var/lib/one/5/images".
Mon Jun 18 10:17:58 2012 [TM][I]: tm_ln.sh: Executed "chmod a+w /var/lib/one/5/images".
Mon Jun 18 10:17:58 2012 [TM][I]: tm_ln.sh: Link /var/lib/one/images/6f540e1c32177f6e5f5cc9a51bc42408
Mon Jun 18 10:17:58 2012 [TM][I]: tm_ln.sh: Executed "ln -s /var/lib/one/images/6f540e1c32177f6e5f5cc9a51bc42408 /var/lib/one/5/images/disk.1".
Mon Jun 18 10:17:58 2012 [TM][I]: ExitCode: 0
Mon Jun 18 10:18:04 2012 [LCM][I]: New VM state is BOOT
Mon Jun 18 10:18:04 2012 [VMM][I]: Generating deployment file: /var/lib/one/5/deployment.0
Mon Jun 18 10:18:04 2012 [VMM][I]: ExitCode: 0
Mon Jun 18 10:18:04 2012 [VMM][I]: Successfully execute network driver operation: pre.
Mon Jun 18 10:18:07 2012 [VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/deploy /var/lib/one/5/images/deployment.0 myto 5 myto
Mon Jun 18 10:18:07 2012 [VMM][I]: error: Failed to create domain from /var/lib/one/5/images/deployment.0
Mon Jun 18 10:18:07 2012 [VMM][I]: error: internal error process exited while connecting to monitor: kvm: -drive file=/var/lib/one/5/images/disk.1,if=none,id=drive-ide0-0-0,format=raw: could not open disk image /var/lib/one/5/images/disk.1: Permission denied
Mon Jun 18 10:18:07 2012 [VMM][I]:
Mon Jun 18 10:18:07 2012 [VMM][E]: Could not create domain from /var/lib/one/5/images/deployment.0
Mon Jun 18 10:18:07 2012 [VMM][I]: ExitCode: 255
Mon Jun 18 10:18:07 2012 [VMM][I]: Failed to execute virtualization driver operation: deploy.
Mon Jun 18 10:18:07 2012 [VMM][E]: Error deploying virtual machine: Could not create domain from /var/lib/one/5/images/deployment.0
Mon Jun 18 10:18:12 2012 [DiM][I]: New VM state is FAILED
I dont knew where is issue - could somebody help me?
Thx
--
Jan Benadik
+421 46 5151 332
+421 903 691 634
<a class="moz-txt-link-abbreviated" href="mailto:jan.benadik@atos.net">jan.benadik@atos.net</a>
Vinohradn cka 6, 971 01 Prievidza
<a class="moz-txt-link-abbreviated" href="http://www.sk.atos.net">www.sk.atos.net</a>
__________________________________
_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a>
--
Jaime Melis
Project Engineer
OpenNebula - The Open Source Toolkit for Cloud Computing
<a class="moz-txt-link-abbreviated" href="http://www.OpenNebula.org">www.OpenNebula.org</a> | <a class="moz-txt-link-abbreviated" href="mailto:jmelis@opennebula.org">jmelis@opennebula.org</a>
--
Ján Beňadik
Managed Services - Solution Design Architect
+421 46 5151 332
+421 903 691 634
<a class="moz-txt-link-abbreviated" href="mailto:jan.benadik@atos.net">jan.benadik@atos.net</a>
Vinohradnícka 6, 971 01 Prievidza
<a class="moz-txt-link-abbreviated" href="http://www.sk.atos.net">www.sk.atos.net</a>
__________________________________
_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a>
--
Ján Beňadik
Managed Services - Solution Design Architect
+421 46 5151 332
+421 903 691 634
<a class="moz-txt-link-abbreviated" href="mailto:jan.benadik@atos.net">jan.benadik@atos.net</a>
Vinohradnícka 6, 971 01 Prievidza
<a class="moz-txt-link-abbreviated" href="http://www.sk.atos.net">www.sk.atos.net</a>
__________________________________
</pre>
</blockquote>
<pre wrap="">
--
Jaime Melis
Project Engineer
OpenNebula - The Open Source Toolkit for Cloud Computing
<a class="moz-txt-link-abbreviated" href="http://www.OpenNebula.org">www.OpenNebula.org</a> | <a class="moz-txt-link-abbreviated" href="mailto:jmelis@opennebula.org">jmelis@opennebula.org</a>
--
Ján Beňadik
Managed Services - Solution Design Architect
+421 46 5151 332
+421 903 691 634
<a class="moz-txt-link-abbreviated" href="mailto:jan.benadik@atos.net">jan.benadik@atos.net</a>
Vinohradnícka 6, 971 01 Prievidza
<a class="moz-txt-link-abbreviated" href="http://www.sk.atos.net">www.sk.atos.net</a>
__________________________________
--
Ján Beňadik
Managed Services - Solution Design Architect
+421 46 5151 332
+421 903 691 634
<a class="moz-txt-link-abbreviated" href="mailto:jan.benadik@atos.net">jan.benadik@atos.net</a>
Vinohradnícka 6, 971 01 Prievidza
<a class="moz-txt-link-abbreviated" href="http://www.sk.atos.net">www.sk.atos.net</a>
__________________________________
_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a>
<a class="moz-txt-link-freetext" href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a>
</pre>
</blockquote>
<pre wrap="">
--
Javier Fontán Muiños
Project Engineer
OpenNebula - The Open Source Toolkit for Data Center Virtualization
<a class="moz-txt-link-abbreviated" href="http://www.OpenNebula.org">www.OpenNebula.org</a> | <a class="moz-txt-link-abbreviated" href="mailto:jfontan@opennebula.org">jfontan@opennebula.org</a> | @OpenNebula
</pre>
</blockquote>
<br>
<div class="moz-signature">-- <br>
<meta http-equiv="content-type" content="text/html;
charset=ISO-8859-2">
<title></title>
<font color="black" face="Verdana" size="3"> <img alt=""
src="cid:part1.08020708.02050308@atos.net" height="15"
width="252"><br>
<b>Ján Beňadik</b><br>
<font face="Verdana" size="2"> Managed Services - Solution
Design Architect<br>
+421 46 5151 332<br>
+421 903 691 634<br>
<a href="mailto://jan.benadik@atos.net">jan.benadik@atos.net</a><br>
Vinohradnícka 6, 971 01 Prievidza<br>
<a href="http://www.sk.atos.net">www.sk.atos.net</a><br>
__________________________________<br>
<br>
<img alt="" src="cid:part4.04080903.01060402@atos.net"
height="58" width="261"><br>
</font> </font></div>
</body>
</html>