<div dir="ltr">Just a wild guess... The 'one-294' argument for the poll script is taken from VM/DEPLOYMENT_ID. Maybe a bug caused the core to lose that string?<div><br></div><div style>Can you please check that attribute in the onevm show -x output? If is looks good, edit <span style="font-size:13px;font-family:arial,sans-serif">/var/tmp/one/vmm/kvm/poll and write the arguments somewhere, just to double check.</span></div>
<div style><span style="font-family:arial,sans-serif;font-size:13px"><br></span></div><div style><span style="font-family:arial,sans-serif;font-size:13px">Cheers</span></div></div><div class="gmail_extra"><br clear="all">
<div>--<br>Carlos Martín, MSc<br>Project Engineer<br>OpenNebula - The Open-source Solution for Data Center Virtualization<div><span style="border-collapse:collapse;color:rgb(136,136,136);font-family:arial,sans-serif;font-size:13px"><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:cmartin@opennebula.org" target="_blank">cmartin@opennebula.org</a> | <a href="http://twitter.com/opennebula" target="_blank">@OpenNebula</a></span><span style="border-collapse:collapse;color:rgb(136,136,136);font-family:arial,sans-serif;font-size:13px"><a href="mailto:cmartin@opennebula.org" style="color:rgb(42,93,176)" target="_blank"></a></span></div>
</div>
<br><br><div class="gmail_quote">On Thu, Apr 4, 2013 at 11:42 PM, Ruben S. Montero <span dir="ltr"><<a href="mailto:rsmontero@opennebula.org" target="_blank">rsmontero@opennebula.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">I've been thinking about this and I can't see anypoint where this information is cached. It is executed and send right away to the core to process it. In fact you should see the same line "... STATE=a" in the logs.<div>
<br></div><div>Cheers</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 3, 2013 at 4:18 PM, Duverne, Cyrille <span dir="ltr"><<a href="mailto:cyrille.duverne@euranova.eu" target="_blank">cyrille.duverne@euranova.eu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Ok ok, that's indeed fun :<br><br><font size="1"><span style="font-family:Sans;text-align:-webkit-auto;white-space:pre-wrap">ruby -wd /var/tmp/one/vmm/kvm/poll one-294</span><span style="text-align:-webkit-auto"><br style="word-wrap:break-word;word-break:break-word;font-family:Sans;text-align:-webkit-auto;white-space:pre-wrap">
</span><span style="font-family:Sans;text-align:-webkit-auto;white-space:pre-wrap">STATE=a NETTX=19039830 USEDCPU=0.1 USEDMEMORY=1121828 NETRX=416126660</span></font><br><br>Seems that the polling is correctly working.<br>
Possible that the state is still on cache or in the DB and not updated or something ? <br><br>Cheers<br>Cyrille<br><br><br><br>At Wednesday, 03/04/2013 on 15:15 Ruben S. Montero wrote:<br><blockquote style="font-size:13px;border-width:0px 0px 0px 2px;border-left-style:solid;border-left-color:rgb(34,67,127);padding:0px 0px 0px 5px;margin:0px 0px 0px 5px">
<div dir="ltr">Could you execute the vmm probe in the host<div><br></div><div>/var/tmp/one/vmm/kvm/poll one-294</div><div><br></div><div>and check for errors, or try to debug the script... (maybe running it with ruby -wd)</div>
<div><br></div><div>Ruben</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 3, 2013 at 10:42 AM, Duverne, Cyrille <span dir="ltr"><<a href="mailto:cyrille.duverne@euranova.eu" target="_blank">cyrille.duverne@euranova.eu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Hello,<br><br>Indeed, state is still "d" , as you can see here :<br><br><ol style="margin:0px;padding:0px 0px 0px 48px;color:rgb(172,172,172);font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono',monospace,serif;font-size:12px;line-height:21px;background-color:rgb(248,248,248)">
<li><div style="padding:0px 5px;vertical-align:top;color:rgb(0,0,0);border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);background-color:rgb(255,255,255);font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono',monospace,serif">
Wed Apr 3 10:34:13 2013 [VMM][I]: Monitoring VM 294.</div></li><li><div style="padding:0px 5px;vertical-align:top;color:rgb(0,0,0);border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);background-color:rgb(255,255,255);font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono',monospace,serif">
Wed Apr 3 10:34:13 2013 [VMM][D]: Message received: LOG I 294 ExitCode: 0</div></li><li><div style="padding:0px 5px;vertical-align:top;color:rgb(0,0,0);border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);background-color:rgb(255,255,255);font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono',monospace,serif">
Wed Apr 3 10:34:13 2013 [VMM][D]: Message received: POLL SUCCESS 294 STATE=d</div><br></li><li></li></ol>Any thought ?<br>By consciousness, I verified that all users etc... were still correct on all machines, the oneadmin is able to ssh directly etc...<br>
<br>Thanks in advance<br>Cyrille<br><br><br><br>At Tuesday, 02/04/2013 on 22:31 Ruben S. Montero wrote:<br><blockquote style="border:0;border-left:2px solid #22437f;padding:0px;margin:0px;padding-left:5px;margin-left:5px">
<div dir="ltr">So the VMs are now running, and correctly reported by libvirt, but OpenNebula does not move them from UNKNOWN to RUNNING?, Are the messages still reporting STATE=d for these VMs in oned.log?<div><br></div>
<div>
Ruben</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Apr 2, 2013 at 3:57 PM, Duverne, Cyrille <span dir="ltr"><<a href="mailto:cyrille.duverne@euranova.eu" target="_blank">cyrille.duverne@euranova.eu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Hello,<br><br>Anything new on this ?<br><br>Seems really weird to me...<br><br>Thanks in advance<br>Cyrille<div>
<br><br><br><br>At Friday, 29/03/2013 on 10:06 Duverne, Cyrille wrote:<br></div><blockquote style="border:0;border-left:2px solid #22437f;padding:0px;margin:0px;padding-left:5px;margin-left:5px"><div>Hello Ruben !<br>
<br>Thanks for this feedback.<br><br></div>I tried to restart libvirt, which succeeded (WOW ! <img alt=":p">)<div><div><br><br>But the VMs are still stuck on Unknown state.<br><br>the 'virsh list' shows correctly the domains, which are running :<br>
<br><div>virsh list</div><div> Id Name State</div><div>----------------------------------</div><div> 1 one-294 running</div><div> 2 one-304 running</div><div><br>Any other thought ? I'm a bit confused by this behaviour and the workflow to monitor the VMs, it could be interesting to have a 'refresh monitoring' button or whatever on Sunstone to try to get fresh monitoring information.</div>
<br>Thanks in advance<br>Cyrille<br><br><div>
"Always do right. This will gratify some people and astonish the rest."<br>
Mark Twain</div><br><br><br>At Thursday, 28/03/2013 on 0:56 Ruben S. Montero wrote:<br><blockquote style="border:0;border-left:2px solid #22437f;padding:0px;margin:0px;padding-left:5px;margin-left:5px"><div dir="ltr">Ok<div>
<br></div><div>So this is strange... </div><div><br></div><div>On one hand you try to restart the VM and virsh says it is already defined (vm.log: main 'one-294' already exists) . And on the other hand when you monitor the VM virsh list does not show it (oned.log: POLL SUCCESS 294 STATE=d)</div>
<div><br></div><div>Is the domain really defined at the host (virsh list)? Can this be a libvirt issue, any chance to restart libvirt and try again?</div><div><br></div><div><br></div><div>Cheers</div>
<div><br></div><div>Ruben</div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Mar 26, 2013 at 10:37 PM, Duverne, Cyrille <span dir="ltr"><<a href="mailto:cyrille.duverne@euranova.eu" target="_blank">cyrille.duverne@euranova.eu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>Hello Ruben,<br><br>Indeed this happens for some of them, but for some others they are still in UNKNOWs state.<br>
Here is an extract of the VM log :<br><br>"<span style="color:rgb(34,34,34)">Thu Mar 21 11:55:56 2013 [LCM][I]: New VM state is SAVE_SUSPEND</span><pre style="color:rgb(34,34,34)">Thu Mar 21 11:57:49 2013 [VMM][I]: ExitCode: 0
Thu Mar 21 11:57:49 2013 [VMM][I]: Successfully execute virtualization driver operation: save.
Thu Mar 21 11:57:50 2013 [VMM][I]: ExitCode: 0
Thu Mar 21 11:57:50 2013 [VMM][I]: Successfully execute network driver operation: clean.
Thu Mar 21 11:57:50 2013 [DiM][I]: New VM state is SUSPENDED
Tue Mar 26 17:27:48 2013 [DiM][I]: New VM state is ACTIVE.
Tue Mar 26 17:27:48 2013 [LCM][I]: Restoring VM
Tue Mar 26 17:27:48 2013 [LCM][I]: New state is BOOT_SUSPENDED
Tue Mar 26 17:27:49 2013 [VMM][I]: ExitCode: 0
Tue Mar 26 17:27:49 2013 [VMM][I]: Successfully execute network driver operation: pre.
Tue Mar 26 17:28:37 2013 [VMM][I]: ExitCode: 0
Tue Mar 26 17:28:37 2013 [VMM][I]: Successfully execute virtualization driver operation: restore.
Tue Mar 26 17:28:37 2013 [VMM][I]: ExitCode: 0
Tue Mar 26 17:28:37 2013 [VMM][I]: Successfully execute network driver operation: post.
Tue Mar 26 17:28:38 2013 [LCM][I]: New VM state is RUNNING
Tue Mar 26 17:28:38 2013 [VMM][I]: ExitCode: 0
Tue Mar 26 17:28:39 2013 [VMM][I]: VM running but it was not found. Restart and delete actions available or try to recover it manually
Tue Mar 26 17:28:39 2013 [LCM][I]: New VM state is UNKNOWN
Tue Mar 26 17:36:48 2013 [LCM][I]: New VM state is BOOT_UNKNOWN
Tue Mar 26 17:36:48 2013 [VMM][I]: Generating deployment file: /var/lib/one/294/deployment.1
Tue Mar 26 17:36:52 2013 [VMM][I]: ExitCode: 0
Tue Mar 26 17:36:52 2013 [VMM][I]: Successfully execute network driver operation: pre.
Tue Mar 26 17:36:52 2013 [VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/deploy /var/lib/one/datastores/0/294/deployment.1 whitefall.local 294 whitefall.local
Tue Mar 26 17:36:52 2013 [VMM][I]: error: Failed to create domain from /var/lib/one/datastores/0/294/deployment.1
Tue Mar 26 17:36:52 2013 [VMM][I]: error: operation failed: domain 'one-294' already exists with uuid 326bc42b-1f8a-8984-e610-4c35f0bdd56f
<span>Tue Mar 26 17:36:52 2013 [VMM][E]: Could not create domain from /var/lib/one/datastores/0/294/deployment.1</span>
Tue Mar 26 17:36:52 2013 [VMM][I]: ExitCode: 255
Tue Mar 26 17:36:52 2013 [VMM][I]: Failed to execute virtualization driver operation: deploy.
<span>Tue Mar 26 17:36:52 2013 [VMM][E]: Error deploying virtual machine: Could not create domain from /var/lib/one/datastores/0/294/deployment.1</span>
Tue Mar 26 17:36:52 2013 [LCM][I]: Fail to boot VM. New VM state is UNKNOWN
Tue Mar 26 17:37:21 2013 [LCM][I]: New VM state is BOOT_UNKNOWN
Tue Mar 26 17:37:21 2013 [VMM][I]: Generating deployment file: /var/lib/one/294/deployment.1
Tue Mar 26 17:37:22 2013 [VMM][I]: ExitCode: 0
Tue Mar 26 17:37:22 2013 [VMM][I]: Successfully execute network driver operation: pre.
Tue Mar 26 17:37:22 2013 [VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/deploy /var/lib/one/datastores/0/294/deployment.1 whitefall.local 294 whitefall.local
Tue Mar 26 17:37:22 2013 [VMM][I]: error: Failed to create domain from /var/lib/one/datastores/0/294/deployment.1
Tue Mar 26 17:37:22 2013 [VMM][I]: error: operation failed: domain 'one-294' already exists with uuid 326bc42b-1f8a-8984-e610-4c35f0bdd56f
<span>Tue Mar 26 17:37:22 2013 [VMM][E]: Could not create domain from /var/lib/one/datastores/0/294/deployment.1</span>
Tue Mar 26 17:37:22 2013 [VMM][I]: ExitCode: 255
Tue Mar 26 17:37:22 2013 [VMM][I]: Failed to execute virtualization driver operation: deploy.
<span>Tue Mar 26 17:37:22 2013 [VMM][E]: Error deploying virtual machine: Could not create domain from /var/lib/one/datastores/0/294/deployment.1</span>
Tue Mar 26 17:37:23 2013 [LCM][I]: Fail to boot VM. New VM state is UNKNOWN
Tue Mar 26 17:38:39 2013 [VMM][I]: ExitCode: 0
Tue Mar 26 17:38:41 2013 [VMM][I]: VM running but it was not found. Restart and delete actions available or try to recover it manually
Tue Mar 26 17:48:45 2013 [VMM][I]: ExitCode: 0
Tue Mar 26 17:48:45 2013 [VMM][I]: VM running but it was not found. Restart and delete actions available or try to recover it manually
Tue Mar 26 17:58:45 2013 [VMM][I]: ExitCode: 0
Tue Mar 26 17:58:45 2013 [VMM][I]: VM running but it was not found. Restart and delete actions available or try to recover it manually </pre><span style="color:rgb(34,34,34)">Tue Mar 26 18:08:45 2013 [VMM][I]: ExitCode: 0</span>"<br>
<br>The RESTART didn't do anything.<br><br>Here is the oned.log's extract for the same VM : <br><br>"Tue Mar 26 22:18:45 2013 [VMM][I]: Monitoring VM 294.<br><div>Tue Mar 26 22:18:45 2013 [VMM][D]: Message received: LOG I 294 ExitCode: 0</div>
Tue Mar 26 22:18:45 2013 [VMM][D]: Message received: POLL SUCCESS 294 STATE=d"<br><br>The VMs that are in UNKNOWN state are located on 2 different hosts.<br>All hosts are configurated in the same way.<br><br>Thanks in advance<br>
Cyrille<br><br><br>At Tuesday, 26/03/2013 on 18:53 Ruben S. Montero wrote:<br><blockquote style="border:0;border-left:2px solid #22437f;padding:0px;margin:0px;padding-left:5px;margin-left:5px"><div dir="ltr">They should appear after a while, when the VM is monitored... Look for messages Monitoring VM... in oned.log.<div>
<br></div><div>Cheers</div><div><br></div><div>Ruben</div></div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Tue, Mar 26, 2013 at 5:39 PM, Duverne, Cyrille <span dir="ltr"><<a href="mailto:cyrille.duverne@euranova.eu" target="_blank">cyrille.duverne@euranova.eu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>Hello,<br><br>I just finished the reboot of our lab after electric shutdown, everything went fine.<br><br>But some of the VMs are stuck in UNKNOWN state after resuming them.<br>I tried to restart them, but they are actually running on the Hypervisors, it's just that sunstone is displaying UNKNOWN.<br>
<br>Any thought to solve this ?<br><br>Thanks in advance<span><font color="#888888"><br>Cyrille<br><br></font></span></div>
<br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org" target="_blank">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><br>
<br></blockquote></div><br><br clear="all"><span><font color="#888888"><div><br></div>-- <br>Ruben S. Montero, PhD<br>Project co-Lead and Chief Architect<br>OpenNebula - The Open Source Solution for Data Center Virtualization<br>
<a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:rsmontero@opennebula.org" target="_blank">rsmontero@opennebula.org</a> | @OpenNebula
</font></span></div></blockquote></div>
</blockquote></div><br><br clear="all"><span><font color="#888888"><div><br></div>-- <br>Ruben S. Montero, PhD<br>Project co-Lead and Chief Architect<br>OpenNebula - The Open Source Solution for Data Center Virtualization<br>
<a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:rsmontero@opennebula.org" target="_blank">rsmontero@opennebula.org</a> | @OpenNebula
</font></span></div></blockquote></div></div></blockquote></div><span><font color="#888888">
</font></span></blockquote></div><span><font color="#888888"><br><br clear="all"><span><font color="#888888"><div><br></div>-- <br>Ruben S. Montero, PhD<br>Project co-Lead and Chief Architect<br>OpenNebula - The Open Source Solution for Data Center Virtualization<br>
<a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:rsmontero@opennebula.org" target="_blank">rsmontero@opennebula.org</a> | @OpenNebula
</font></span></font></span></div></blockquote></div><span><font color="#888888">
</font></span></blockquote></div><span><font color="#888888"><br><br clear="all"><div><br></div>-- <br>Ruben S. Montero, PhD<br>Project co-Lead and Chief Architect<br>OpenNebula - The Open Source Solution for Data Center Virtualization<br>
<a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:rsmontero@opennebula.org" target="_blank">rsmontero@opennebula.org</a> | @OpenNebula
</font></span></div></blockquote></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Ruben S. Montero, PhD<br>Project co-Lead and Chief Architect<br>OpenNebula - The Open Source Solution for Data Center Virtualization<br><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:rsmontero@opennebula.org" target="_blank">rsmontero@opennebula.org</a> | @OpenNebula
</div>
</div></div><br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><br>
<br></blockquote></div><br></div>