Hi Ross,<div><br></div><div>actually in my experience disabling apparmor won't work either. You will have to modify one of its configuration files in order to make it work.</div><div><br>Add this:</div><div>-------8<--------</div>
<div> /srv/cloud/one/var/** rw,</div><div>------->8--------</div><div>(If you have a different VMDIR change the above line accordingly).</div><div>To the end of /etc/apparmor.d/abstractions/libvirt-qemu</div><div>And restart the apparmor service.</div>
<div><br></div><div>Regards,</div><div>Jaime</div><div><br></div><div><br></div><div><br></div><div><br><div class="gmail_quote">On Mon, Jul 26, 2010 at 5:30 PM, Tino Vazquez <span dir="ltr"><<a href="mailto:tinova@fdi.ucm.es">tinova@fdi.ucm.es</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">Hi Ross,<br>
<br>
Are you using Ubuntu per chance? It may be a issue with the apparmor<br>
service, try disabling it to see if that is the one to blame. In case<br>
it is, we can provide rules to disable this apparmor behavior.<br>
<br>
Regards,<br>
<div class="im"><br>
-Tino<br>
<br>
--<br>
Constantino Vázquez Blanco | <a href="http://dsa-research.org/tinova" target="_blank">dsa-research.org/tinova</a><br>
Virtualization Technology Engineer / Researcher<br>
OpenNebula Toolkit | <a href="http://opennebula.org" target="_blank">opennebula.org</a><br>
<br>
<br>
<br>
</div><div><div></div><div class="h5">On Mon, Jul 26, 2010 at 5:13 PM, Ross Nordeen <<a href="mailto:rjnordee@mtu.edu">rjnordee@mtu.edu</a>> wrote:<br>
> Tino,<br>
><br>
> I figured out my live migrate problem which turned out to be a bad default gw. As far as the migration and check pointing though I have the /srv/cloud/one directory shared out to all nodes via nfs and full permissions for oneadmin... I think it is /srv/cloud/one/var/18. I will check the VM_DIR variable in the oned.conf file though and see if it is right. Still if everything else is working it seems like the VM_DIR is exported correctly and functioning for the running vm's.<br>
><br>
> -Ross<br>
><br>
> ----- Original Message -----<br>
> From: "Tino Vazquez" <<a href="mailto:tinova@fdi.ucm.es">tinova@fdi.ucm.es</a>><br>
> To: "Ross Nordeen" <<a href="mailto:rjnordee@mtu.edu">rjnordee@mtu.edu</a>><br>
> Cc: <a href="mailto:users@lists.opennebula.org">users@lists.opennebula.org</a><br>
> Sent: Monday, July 26, 2010 8:41:37 AM GMT -07:00 US/Canada Mountain<br>
> Subject: Re: [one-users] migration not working completly<br>
><br>
> Hi Ross,<br>
><br>
> There seems to be two issues here:<br>
><br>
> 1) Not live/migrate between cn2 and cn1 --> could it be that the<br>
> oneadmin user cannot passwordlessly ssh from cn2 to cn1, but it can<br>
> from cn1 to cn2?<br>
><br>
> 2) The save problem seems to come from the impossibility to save the<br>
> checkpoint file. This may be due to the fact that /srv/cloud/one<br>
> directory doesn't exist in the remote nodes, in which case you will<br>
> need to use the VM_DIR variable in the oned.conf file.<br>
><br>
> Hope it helps,<br>
><br>
> -Tino<br>
><br>
> --<br>
> Constantino Vázquez Blanco | <a href="http://dsa-research.org/tinova" target="_blank">dsa-research.org/tinova</a><br>
> Virtualization Technology Engineer / Researcher<br>
> OpenNebula Toolkit | <a href="http://opennebula.org" target="_blank">opennebula.org</a><br>
><br>
><br>
><br>
> On Thu, Jul 22, 2010 at 11:39 PM, Ross Nordeen <<a href="mailto:rjnordee@mtu.edu">rjnordee@mtu.edu</a>> wrote:<br>
>> I have open nebula deployed with one head node and 2 compute nodes, I have no problems live migrating from cn1 to cn2 but I get failures live/cold migrating from cn2 to cn1. is there any reason I would not able to a) not save the state of any of my machines and why live-migration works one way but not the other?? Thanks<br>
>><br>
>> -Ross<br>
>><br>
>><br>
>> here is my vm.log file after a live-migration, migration, and than suspend:<br>
>><br>
>><br>
>> Thu Jul 22 11:40:22 2010 [LCM][I]: New VM state is MIGRATE<br>
>> Thu Jul 22 11:40:22 2010 [VMM][I]: Command execution fail: virsh --connect qemu:///system migrate --live one-18 qemu+ssh://cn1/session<br>
>> Thu Jul 22 11:40:22 2010 [VMM][I]: STDERR follows.<br>
>> Thu Jul 22 11:40:22 2010 [VMM][I]: Warning: Permanently added 'cn2,192.168.1.105' (RSA) to the list of known hosts.<br>
>> Thu Jul 22 11:40:22 2010 [VMM][I]: error: cannot recv data: Connection reset by peer<br>
>> Thu Jul 22 11:40:22 2010 [VMM][I]: ExitCode: 1<br>
>> Thu Jul 22 11:40:22 2010 [VMM][E]: Error live-migrating VM, -<br>
>> Thu Jul 22 11:40:23 2010 [LCM][I]: Fail to life migrate VM. Assuming that the VM is still RUNNING (will poll VM).<br>
>> Thu Jul 22 11:40:23 2010 [VMM][D]: Monitor Information:<br>
>> .<br>
>> .<br>
>> .<br>
>> .<br>
>> .<br>
>> Thu Jul 22 15:09:04 2010 [LCM][I]: New VM state is MIGRATE<br>
>> Thu Jul 22 15:09:04 2010 [VMM][I]: Command execution fail: virsh --connect qemu:///system migrate --live one-18 qemu+ssh://cn1/session<br>
>> Thu Jul 22 15:09:04 2010 [VMM][I]: STDERR follows.<br>
>> Thu Jul 22 15:09:04 2010 [VMM][I]: Warning: Permanently added 'cn2,192.168.1.105' (RSA) to the list of known hosts.<br>
>> Thu Jul 22 15:09:04 2010 [VMM][I]: error: cannot recv data: Connection reset by peer<br>
>> Thu Jul 22 15:09:04 2010 [VMM][I]: ExitCode: 1<br>
>> Thu Jul 22 15:09:04 2010 [VMM][E]: Error live-migrating VM, -<br>
>> Thu Jul 22 15:09:05 2010 [LCM][I]: Fail to life migrate VM. Assuming that the VM is still RUNNING (will poll VM).<br>
>> Thu Jul 22 15:09:05 2010 [VMM][D]: Monitor Information:<br>
>> .<br>
>> .<br>
>> .<br>
>> .<br>
>> .<br>
>> Thu Jul 22 15:11:25 2010 [LCM][I]: New VM state is SAVE_MIGRATE<br>
>> Thu Jul 22 15:11:25 2010 [VMM][I]: Command execution fail: 'touch /srv/cloud/one/var//18/images/checkpoint;virsh --connect qemu:///system save one-18 /srv/cloud/one/var//18/images/checkpoint'<br>
>> Thu Jul 22 15:11:25 2010 [VMM][I]: STDERR follows.<br>
>> Thu Jul 22 15:11:25 2010 [VMM][I]: Warning: Permanently added 'cn2,192.168.1.105' (RSA) to the list of known hosts.<br>
>> Thu Jul 22 15:11:25 2010 [VMM][I]: error: Failed to save domain one-18 to /srv/cloud/one/var//18/images/checkpoint<br>
>> Thu Jul 22 15:11:25 2010 [VMM][I]: error: operation failed: failed to create '/srv/cloud/one/var//18/images/checkpoint'<br>
>> Thu Jul 22 15:11:25 2010 [VMM][I]: ExitCode: 1<br>
>> Thu Jul 22 15:11:25 2010 [VMM][E]: Error saving VM state, -<br>
>> Thu Jul 22 15:11:25 2010 [LCM][I]: Fail to save VM state while migrating. Assuming that the VM is still RUNNING (will poll VM).<br>
>> Thu Jul 22 15:11:26 2010 [VMM][I]: VM running but new state from monitor is PAUSED.<br>
>> Thu Jul 22 15:11:26 2010 [LCM][I]: VM is suspended.<br>
>> Thu Jul 22 15:11:26 2010 [DiM][I]: New VM state is SUSPENDED<br>
>> Thu Jul 22 15:13:20 2010 [DiM][I]: New VM state is ACTIVE.<br>
>> Thu Jul 22 15:13:20 2010 [LCM][I]: Restoring VM<br>
>> Thu Jul 22 15:13:20 2010 [LCM][I]: New state is BOOT<br>
>> Thu Jul 22 15:13:21 2010 [VMM][I]: Command execution fail: virsh --connect qemu:///system restore /srv/cloud/one/var//18/images/checkpoint<br>
>> Thu Jul 22 15:13:21 2010 [VMM][I]: STDERR follows.<br>
>> Thu Jul 22 15:13:21 2010 [VMM][I]: Warning: Permanently added 'cn2,192.168.1.105' (RSA) to the list of known hosts.<br>
>> Thu Jul 22 15:13:21 2010 [VMM][I]: error: Failed to restore domain from /srv/cloud/one/var//18/images/checkpoint<br>
>> Thu Jul 22 15:13:21 2010 [VMM][I]: error: operation failed: cannot read domain image<br>
>> Thu Jul 22 15:13:21 2010 [VMM][I]: ExitCode: 1<br>
>> Thu Jul 22 15:13:21 2010 [VMM][E]: Error restoring VM, -<br>
>> Thu Jul 22 15:13:21 2010 [DiM][I]: New VM state is FAILED<br>
>> Thu Jul 22 15:13:21 2010 [TM][W]: Ignored: LOG - 18 tm_delete.sh: Deleting /srv/cloud/one/var//18/images<br>
>><br>
>> Thu Jul 22 15:13:21 2010 [TM][W]: Ignored: LOG - 18 tm_delete.sh: Executed "rm -rf /srv/cloud/one/var//18/images".<br>
>><br>
>> Thu Jul 22 15:13:21 2010 [TM][W]: Ignored: TRANSFER SUCCESS 18 -<br>
>><br>
>> _______________________________________________<br>
>> Users mailing list<br>
>> <a href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a><br>
>> <a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><br>
>><br>
><br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><br>
</div></div></blockquote></div><br></div>