<div dir="ltr">Hi Armin,<div><br></div><div>apologies for the delay in the reponse.</div><div><br></div><div>The problem here is that ideally we want to report the maximum size of the images if they are sparse images. If we have a 10G sparse image that is taking up only 1G in space, we would like to report the 10G amount, and not 1G, since we would like to guarantee available space in the datastore. That's why we cannot use `df`.</div>
<div><br></div><div>In any case, the system now in place is not properly reporting the image size, since we would need to be using `du --aparent-size`. So as for now, switching from `du` to `df` should be a plausible workaround with no real impact, except better performance. On the other hand, as you have reported `du` generates a lot of I/O, so ideally we would need to be doing is to get the USED disk space directly from OpenNebula.</div>
<div><br></div><div>To sum it up, we have these two related issues to improve this:</div><div><a href="http://dev.opennebula.org/issues/2783">http://dev.opennebula.org/issues/2783</a><br></div><div><a href="http://dev.opennebula.org/issues/2402">http://dev.opennebula.org/issues/2402</a><br>
</div><div><br></div><div>As for your particular case, I think you should modify monitor_ds.sh with a call to `df` instead of `du`.</div><div><br></div><div><div style="font-family:arial,sans-serif;font-size:13.333333969116211px">
> - is it needed to check the datastores from every single host (if it is a shared environment and the opennebula server itself has access to the datastores)?<br>The monitorization driver is very simple (in order to be easily adapted), and there is no way to orchestrate this. It would be a very nice thing to have though, feel free to open a feature request to implement this.</div>
<div style="font-family:arial,sans-serif;font-size:13.333333969116211px"><br></div><span style="font-family:arial,sans-serif;font-size:13.333333969116211px"> > - why is the base_path checked at all? isn't it enough to check the datastores? there is du run on all the directories twice and du is very resource consuming ...</span><br style="font-family:arial,sans-serif;font-size:13.333333969116211px">
</div><div><span style="font-family:arial,sans-serif;font-size:13.333333969116211px">This is a tricky one. If the mountpoint does not exist (/var/lib/one/datastores/<id>) it can be created automatically by opennebula, but only when the first VM is executed. OpenNebula needs to know in this case if there will be enough space in the datastore, and it assumes that if the mountpoint doesn't exist yet it will have the same space available as the parent folder, which is the datastore location: /var/lib/one/datastores. That's why we need to be checking that too.</span></div>
<div><span style="font-family:arial,sans-serif;font-size:13.333333969116211px"><br></span></div><div><font face="arial, sans-serif">Regards,<br>Jaime</font></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jun 6, 2014 at 2:57 PM, Armin Deliomini <span dir="ltr"><<a href="mailto:armin.deliomini@runtastic.com" target="_blank">armin.deliomini@runtastic.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div>Hi!<br><br></div><div>Platform: Ubuntu 12.04 + KVM<br></div><div>Opennebula 4.6.1<br>
</div><div><br>I recently upgraded from opennebula 4.2 to 4.6.1 ending up in hosts not beeing monitored anymore. After some investigation I found out, that there is a new remote monitoring script, monitor_ds.sh.<br>
<br></div>I have a shared storage environment (some central Netapps sharing NFS) and so my question is: the script is executed on all the hosts, so all hosts run a du -sLm on all datastores and the base_path (/var/lib/on/datastores) which can consume quite some storage performance unnecessarily.<br>
<br></div>In my case hosts haven't been monitored at all, because there was a driectory containing millions of small files in one of the datastores (used as a temp location without thinking of the consequences) and du would never finish in time.<br>
<br></div>There are two questions: <br><br> - is it needed to check the datastores from every single host (if it is a shared environment and the opennebula server itself has access to the datastores)?<br><br></div> - why is the base_path checked at all? isn't it enough to check the datastores? there is du run on all the directories twice and du is very resource consuming ...<br>
<br></div>Is there a reason for this I am missing?<br><br></div>Thx, Armin<br><div><div><div><div><div><br clear="all"><div><div><div><div><div><div><br>-- <br><div dir="ltr"><p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);line-height:13pt">
<span style="font-size:10pt;font-family:Arial,sans-serif;color:rgb(77,77,77);letter-spacing:0.5pt">runtastische Grüße · runtastic regards</span></p><p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);line-height:13pt">
<span style="font-size:5.5pt;color:rgb(64,64,64);letter-spacing:0.5pt"> </span></p><p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);line-height:13pt"><b><span style="font-size:10pt;font-family:Arial,sans-serif;color:rgb(0,105,209);letter-spacing:0.5pt">Armin Deliomini</span></b></p>
<p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><i><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(77,77,77);letter-spacing:0.5pt">Infrastructure engineer</span></i></p>
<p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><i><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(77,77,77);letter-spacing:0.5pt">Virtualization specialist<br>
</span></i></p><p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><i><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(77,77,77);letter-spacing:0.5pt">Storage engineer</span></i></p>
<p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)"><i><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(77,77,77);letter-spacing:0.5pt">DevOPS</span></i></p>
<p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">
<i><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(77,77,77);letter-spacing:0.5pt">OPS<br></span></i></p><p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">
<i><a href="http://www.runtastic.com" style="color:purple" target="_blank"><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(77,77,77);letter-spacing:0.5pt;text-decoration:initial"><img src="http://download.runtastic.com/signature/rt.png" alt="runtastic - makes sports funtastic" border="0" height="67" width="253"></span></a></i></p>
<p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);line-height:13pt"><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(127,127,127);letter-spacing:0.5pt">runtastic GmbH</span></p>
<p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);line-height:13pt"><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(127,127,127);letter-spacing:0.5pt">Pluskaufstraße 7 · Business Center</span></p>
<p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);line-height:13pt"><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(127,127,127);letter-spacing:0.5pt">4061 Pasching / Linz (Austria)</span></p>
<p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);line-height:13pt"><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(127,127,127);letter-spacing:0.5pt">Tel. <a href="tel:%2B43%280%2967689692450" value="+4367689692450" target="_blank">+43(0)67689692450</a><br>
</span></p><p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);line-height:13pt"><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(127,127,127);letter-spacing:0.5pt">Fax. <a href="tel:%2B43%280%297229.23327" value="+43722923327" target="_blank">+43(0)7229.23327</a></span></p>
<p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);line-height:13pt"><a href="mailto:armin.deliomini@runtastic.com" style="color:rgb(0,105,209);text-decoration:initial;font-size:9pt;font-family:Arial,sans-serif;letter-spacing:0.5pt" target="_blank"><span style="font-size:9pt;letter-spacing:0.5pt">armin.deliomini@runtastic.com</span></a></p>
<p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);line-height:13pt"><a href="http://www.runtastic.com" style="color:rgb(0,105,209);text-decoration:initial;font-size:9pt;font-family:Arial,sans-serif;letter-spacing:0.5pt" target="_blank"><span style="font-size:9pt;letter-spacing:0.5pt">www.runtastic.com</span></a><span style="font-size:9pt;font-family:Arial,sans-serif;color:rgb(64,64,64);letter-spacing:0.5pt"></span></p>
<p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0);line-height:13pt"><span style="color:rgb(64,64,64)"> </span></p><p style="margin:0cm 0cm 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif;color:rgb(0,0,0)">
<a href="http://www.facebook.com/runtastic" style="color:purple" target="_blank"><span style="font-size:10pt;font-family:Arial,sans-serif;color:rgb(77,77,77);letter-spacing:0.8pt;text-decoration:initial"><img src="http://download.runtastic.com/signature/fb.png" border="0" height="20" width="21"></span></a><span style="font-size:10pt;font-family:Arial,sans-serif;color:rgb(77,77,77);letter-spacing:0.8pt"> </span><a href="http://www.twitter.com/runtastic" style="color:purple" target="_blank"><span style="font-size:10pt;font-family:Arial,sans-serif;color:rgb(77,77,77);letter-spacing:0.8pt;text-decoration:initial"><img src="http://download.runtastic.com/signature/tw.png" border="0" height="20" width="20"></span></a><span style="font-size:10pt;font-family:Arial,sans-serif;color:rgb(77,77,77);letter-spacing:0.8pt"> </span><span style="color:rgb(77,77,77);font-size:10pt;font-family:Arial,sans-serif;letter-spacing:0.8pt;text-decoration:initial"><a href="http://www.youtube.com/user/runtastic" style="color:purple" target="_blank"><img src="http://download.runtastic.com/signature/yt.png" border="0" height="20" width="21"></a></span></p>
</div>
</div></div></div></div></div></div></div></div></div></div></div></div>
<br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><br>
<br></blockquote></div><br></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div>Jaime Melis<br>Project Engineer<br>OpenNebula - Flexible Enterprise Cloud Made Simple<br><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:jmelis@opennebula.org" target="_blank">jmelis@opennebula.org</a></div>
</div></div>