Dear list,<div><br></div><div>I'm writing this in case someone stumbles with the same problem. In the end Robert managed to fix this by setting <br><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<span style>I needed to modify /etc/libvirt/qemu.conf to set</span><br style><span style>user = "oneadmin"<br></span><span style>group = "cloud"</span></blockquote><div class="gmail_quote"><br></div><div class="gmail_quote">
You can read more about this in OpenNebula's documentation, specifically in:</div><div class="gmail_quote"><a href="http://www.opennebula.org/documentation:rel3.4:kvmg#kvm_configuration">http://www.opennebula.org/documentation:rel3.4:kvmg#kvm_configuration</a></div>
<div class="gmail_quote"><br></div><div class="gmail_quote">Cheers,</div><div class="gmail_quote">Jaime</div><div class="gmail_quote"><br></div><div class="gmail_quote">On Mon, May 14, 2012 at 10:19 PM, Robert Schweikert <span dir="ltr"><<a href="mailto:rjschwei@suse.com" target="_blank">rjschwei@suse.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On 05/14/2012 10:11 AM, Jaime Melis wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Robert,<br>
<br>
Let's see if you can pinpoint where the problem is. A few ideas you may check:<br>
<br>
- Has the 'oneadmin' user the same unix id across all the servers?<br>
</blockquote>
<br></div>
Yes<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
- You said in your email that "ownership change is disabled" however<br>
in the documentation [1] it states that "To be able to use the images<br>
copied by OpenNebula, change also the user and group under which the<br>
libvirtd is run to “oneadmin”." Is it properly configured?<br>
</blockquote>
<br></div>
If I run libvirtd as the oneadmin user no socket gets created. However, since access is goverened by Policykit it should not be necessary to run libvirtd as the oneadmin user. Will testing libvirtd as oneadmin user, I did change the permissions of /var/run/libvirtd to allow oneadmin to write there to assure that the socket creation process would not fail because of write permission issues, but libvirt-sock was still not created.<br>
<br>
I've also asked one of our libvirt experts and he suggested to change the libvirtd.conf (unix_sock_group = "cloud"). I made the suggested changes and the socket (libvirt-sock) became world read- and write-able as documented. Still with this change I get the warning about the socket and launching a VM still fails with the same problem.<br>
<br>
Interestingly enough I can connect to the node using virsh:<br>
<br>
# sudo -u oneadmin virsh -c qemu+ssh://<a href="http://oneadmin@192.168.1.203/system" target="_blank">oneadmin@192.168.1.<u></u>203/system</a><div class="im"><br>
WARNING: no socket to connect to<br></div>
Welcome to virsh, the virtualization interactive terminal.<br>
<br>
Type: 'help' for help with commands<br>
'quit' to quit<br>
<br>
virsh # hostname<br>
node1<br>
<br>
virsh #<br>
<br>
<br>
Note that virsh also complains about the socket issue, but I am still on the host, i.e. the connection was<br>
<br>
That would beg the question why oned cannot deal with this and use a connection mechanism that is more robust.<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
- You could prevent OpenNebula from removing the images after a fail,<br>
so you can do a manual "virsh create":<br>
1. [frontend] comment the ssh_exec_and_log $DST_HOST "rm -rf ..." line<br>
in /var/lib/one/remotes/tm/<TM_<u></u>MAD>/delete<br>
</blockquote>
<br></div>
I am using "shared" as the disk with the image is NFS mounted to the node. Is this the correct approach?<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
2. [host] cd /var/lib/one/datastores/0/<VM_<u></u>ID>; virsh create deployment.0<br>
</blockquote>
<br></div>
OK, this fails and mybe the overall problem?<br>
<br>
# sudo -u oneadmin virsh create deployment.2<div class="im"><br>
WARNING: no socket to connect to<br></div>
error: Failed to create domain from deployment.2<br>
error: Failed to add tap interface to bridge 'br0': Operation not permitted<br>
<br>
# ifconfig<br>
br0 Link encap:Ethernet HWaddr 00:22:4D:4C:12:DE<br>
inet addr:192.168.1.203 Bcast:192.168.1.255 Mask:255.255.255.0<br>
inet6 addr: fe80::222:4dff:fe4c:12de/64 Scope:Link<br>
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1<br>
RX packets:305725 errors:0 dropped:0 overruns:0 frame:0<br>
TX packets:2070530 errors:0 dropped:0 overruns:0 carrier:0<br>
collisions:0 txqueuelen:0<br>
RX bytes:618635967 (589.9 Mb) TX bytes:3006271991 (2867.0 Mb)<div class="im"><br>
<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
3. [host] ls -l `readlink -f /var/lib/one/datastores/0/<VM_<u></u>ID>/disk.0`<br>
</blockquote>
<br></div>
# ls -l `readlink -f /var/lib/one/datastores/0/0/<u></u>disk.0`<br>
-rw-r----- 1 oneadmin cloud 573308928 May 14 20:08 /var/lib/one/datastores/0/0/<u></u>disk.0<div class="im"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
4. Are you able to manipulate as oneadmin the<br>
/var/lib/one/datastores/0/<VM_<u></u>ID>/disk.0 image? try doing echo "foo"><br>
/path/to/image<br>
</blockquote>
<br></div>
Yes I am:<br>
<br>
[frontend] # sudo -u oneadmin touch /var/lib/one/datastores/0/0/<u></u>aFile.txt<br>
[frontend] # ls -l /var/lib/one/datastores/0/0/<u></u>aFile.txt<br>
-rw-r--r-- 1 oneadmin cloud 0 May 14 15:57 /var/lib/one/datastores/0/0/<u></u>aFile.txt<br>
[node] # sudo -u oneadmin touch /var/lib/one/datastores/0/0/<u></u>anotherFile.txt<br>
[node] # # ls -l /var/lib/one/datastores/0/0/<u></u>anotherFile.txt<br>
-rw-r--r-- 1 oneadmin cloud 0 May 14 2012 /var/lib/one/datastores/0/0/<u></u>anotherFile.txt<br>
<br>
Thanks for the continued help with this.<div class="HOEnZb"><div class="h5"><br>
<br>
Robert<br>
<br>
<br>
-- <br>
Robert Schweikert MAY THE SOURCE BE WITH YOU<br>
SUSE-IBM Software Integration Center LINUX<br>
Tech Lead<br>
<a href="mailto:rjschwei@suse.com" target="_blank">rjschwei@suse.com</a><br>
<a href="mailto:rschweik@ca.ibm.com" target="_blank">rschweik@ca.ibm.com</a><br>
<a href="tel:781-464-8147" value="+17814648147" target="_blank">781-464-8147</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Jaime Melis<br>Project Engineer<br>OpenNebula - The Open Source Toolkit for Cloud Computing<br><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:jmelis@opennebula.org" target="_blank">jmelis@opennebula.org</a><br>
</div>