<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=WordSection1>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Hi Dudu, Tino and all,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>We have seen the exact same message (</span>Command execution
fail and bad interpreter: Text file busy))<span style='font-size:11.0pt;
font-family:"Calibri","sans-serif";color:#1F497D'> on our cluster last week when
we expanded it from 12 to 16 hosts (with add host)and deploying 10 Vmachines at
the same time. We did not have multiple instances of opennebula running, we
only added to a running one, so it is unlikely that was the issue (the cluster
was already running stable for a while). We investigated and thought it was a
timing issue with the monitoring (ssh) driver set to 60 seconds and having many
hosts and many VMs. <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>We started using the ssh-monitoring driver again in after the
latest update to opennebula, before that we used our in hous developed snmp
monitoring driver. <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>When we deployed our snmp driver, the error message stopped and
for the last week we have a stable cloud again, now with 16 hosts…<o:p></o:p></span></p>
<p><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>For
people who think see the same timing issues as we did , the snmp_driver is
available in the ecosystem (but make sure you know what snmp is before you try
;-)): </span><span style='font-size:9.5pt;font-family:"Verdana","sans-serif";
color:#484848'><a href="http://opennebula.org/software:ecosystem:snmp_im_driver">http://opennebula.org/software:ecosystem:snmp_im_driver</a><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Regards,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Floris <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>HPC project leader<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'>Sara<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";
color:#1F497D'><o:p> </o:p></span></p>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'>
<p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span
style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>
users-bounces@lists.opennebula.org [mailto:users-bounces@lists.opennebula.org] <b>On
Behalf Of </b>Tino Vazquez<br>
<b>Sent:</b> maandag 19 juli 2010 16:15<br>
<b>To:</b> DuDu<br>
<b>Cc:</b> users@lists.opennebula.org<br>
<b>Subject:</b> Re: [one-users] oned hang<o:p></o:p></span></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal>Dear DuDu,<o:p></o:p></p>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>This happens when two monitorization actions take place at
the same time.<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>First thing, which OpenNebula version are you using?<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>Are you per chance running two OpenNebula instances? Did you
change the host polling time?<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>Regards,<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>-Tino<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal style='margin-bottom:12.0pt'><br clear=all>
--<br>
Constantino Vázquez Blanco | <a href="http://dsa-research.org/tinova">dsa-research.org/tinova</a><br>
Virtualization Technology Engineer / Researcher<br>
OpenNebula Toolkit | <a href="http://opennebula.org">opennebula.org</a><br>
<br>
<o:p></o:p></p>
<div>
<p class=MsoNormal>On Wed, Jul 14, 2010 at 3:13 PM, DuDu <<a
href="mailto:blackass@gmail.com">blackass@gmail.com</a>> wrote:<o:p></o:p></p>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>Hi,<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>We deployed a small cluster of opennebula, with 8 hosts. It
is the default opennebula installation, however, we found that after several
days of running, oned hung. All CLI commands hang too. No new logs generated in
one_xmlrpc.log. And there are quite some error message like the following in
oned.log:<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>[root@vm-container-31-0 logdir]# tail oned.log<br>
Wed Jul 14 14:51:02 2010 [InM][I]: Warning: untrusted X11 forwarding setup
failed: xauth key data not generated<br>
Wed Jul 14 14:51:02 2010 [InM][I]: Warning: No xauth data; using fake
authentication data for X11 forwarding.<br>
Wed Jul 14 14:51:02 2010 [InM][I]: bash:
/tmp/one-im//one_im-c4718299a313d89398ea693104dcce5f: /bin/sh: bad interpreter:
Text file busy<br>
Wed Jul 14 14:51:02 2010 [InM][I]: ExitCode: 126<br>
Wed Jul 14 14:51:02 2010 [InM][I]: Command execution fail: 'mkdir -p
/tmp/one-im/; cat > /tmp/one-im//one_im-f3817715aa24450225bafb4c19b23822; if
[ "x$?" != "x0" ]; then exit -1; fi; chmod +x
/tmp/one-im//one_im-f3817715aa24450225bafb4c19b23822;
/tmp/one-im//one_im-f3817715aa24450225bafb4c19b23822'<br>
Wed Jul 14 14:51:02 2010 [InM][I]: STDERR follows.<br>
Wed Jul 14 14:51:02 2010 [InM][I]: Warning: untrusted X11 forwarding setup
failed: xauth key data not generated<br>
Wed Jul 14 14:51:02 2010 [InM][I]: Warning: No xauth data; using fake
authentication data for X11 forwarding.<br>
Wed Jul 14 14:51:02 2010 [InM][I]: bash:
/tmp/one-im//one_im-f3817715aa24450225bafb4c19b23822: /bin/sh: bad interpreter:
Text file busy<br>
Wed Jul 14 14:51:02 2010 [InM][I]: ExitCode: 126<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>We have to sigkill oned and restart it. And that solves all
problems.<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal>Any idea of this?<o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
<div>
<p class=MsoNormal style='margin-bottom:12.0pt'>Thanks!<o:p></o:p></p>
</div>
<p class=MsoNormal style='margin-bottom:12.0pt'><br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org"
target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><o:p></o:p></p>
</div>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</div>
</body>
</html>