[one-users] monitor_ds.sh in a shared storage environement

[Armin Deliomini]

Thank you. Makes everything clearer for me.

I'll create the feature request and when I have the next free time slot,
I'll see if I find some alternative ways to get the data ...

Thanks, Armin


2014-06-11 12:33 GMT+02:00 Jaime Melis <jmelis at opennebula.org>:

> Hi Armin,
>
> apologies for the delay in the reponse.
>
> The problem here is that ideally we want to report the maximum size of the
> images if they are sparse images. If we have a 10G sparse image that is
> taking up only 1G in space, we would like to report the 10G amount, and not
> 1G, since we would like to guarantee available space in the datastore.
> That's why we cannot use `df`.
>
> In any case, the system now in place is not properly reporting the image
> size, since we would need to be using `du --aparent-size`. So as for now,
> switching from `du` to `df` should be a plausible workaround with no real
> impact, except better performance. On the other hand, as you have reported
> `du` generates a lot of I/O, so ideally we would need to be doing is to get
> the USED disk space directly from OpenNebula.
>
> To sum it up, we have these two related issues to improve this:
> http://dev.opennebula.org/issues/2783
> http://dev.opennebula.org/issues/2402
>
> As for your particular case, I think you should modify monitor_ds.sh with
> a call to `df` instead of `du`.
>
>  > - is it needed to check the datastores from every single host (if it is
> a shared environment and the opennebula server itself has access to the
> datastores)?
> The monitorization driver is very simple (in order to be easily adapted),
> and there is no way to orchestrate this. It would be a very nice thing to
> have though, feel free to open a feature request to implement this.
>
>  > - why is the base_path checked at all? isn't it enough to check the
> datastores? there is du run on all the directories twice and du is very
> resource consuming ...
> This is a tricky one. If the mountpoint does not exist
> (/var/lib/one/datastores/<id>) it can be created automatically by
> opennebula, but only when the first VM is executed. OpenNebula needs to
> know in this case if there will be enough space in the datastore, and it
> assumes that if the mountpoint doesn't exist yet it will have the same
> space available as the parent folder, which is the datastore location:
> /var/lib/one/datastores. That's why we need to be checking that too.
>
> Regards,
> Jaime
>
> On Fri, Jun 6, 2014 at 2:57 PM, Armin Deliomini <
> armin.deliomini at runtastic.com> wrote:
>
>> Hi!
>>
>> Platform: Ubuntu 12.04 + KVM
>> Opennebula 4.6.1
>>
>> I recently upgraded from opennebula 4.2 to 4.6.1 ending up in hosts not
>> beeing monitored anymore. After some investigation I found out, that there
>> is a new remote monitoring script, monitor_ds.sh.
>>
>> I have a shared storage environment (some central Netapps sharing NFS)
>> and so my question is: the script is executed on all the hosts, so all
>> hosts run a du -sLm on all datastores and the base_path
>> (/var/lib/on/datastores) which can consume quite some storage performance
>> unnecessarily.
>>
>> In my case hosts haven't been monitored at all, because there was a
>> driectory containing millions of small files in one of the datastores (used
>> as a temp location without thinking of the consequences) and du would never
>> finish in time.
>>
>> There are two questions:
>>
>>  - is it needed to check the datastores from every single host (if it is
>> a shared environment and the opennebula server itself has access to the
>> datastores)?
>>
>>  - why is the base_path checked at all? isn't it enough to check the
>> datastores? there is du run on all the directories twice and du is very
>> resource consuming ...
>>
>> Is there a reason for this I am missing?
>>
>> Thx, Armin
>>
>>
>> --
>>
>> runtastische Grüße · runtastic regards
>>
>>
>>
>> *Armin Deliomini*
>>
>> *Infrastructure engineer*
>>
>>
>> *Virtualization specialist *
>>
>> *Storage engineer*
>>
>> *DevOPS*
>>
>>
>> *OPS*
>>
>> *[image: runtastic - makes sports funtastic] <http://www.runtastic.com>*
>>
>> runtastic GmbH
>>
>> Pluskaufstraße 7 · Business Center
>>
>> 4061 Pasching / Linz (Austria)
>>
>> Tel. +43(0)67689692450
>>
>> Fax. +43(0)7229.23327
>>
>> armin.deliomini at runtastic.com
>>
>> www.runtastic.com
>>
>>
>>
>>  <http://www.facebook.com/runtastic>   <http://www.twitter.com/runtastic>
>>    <http://www.youtube.com/user/runtastic>
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opennebula.org
>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>
>>
>
>
> --
> Jaime Melis
> Project Engineer
> OpenNebula - Flexible Enterprise Cloud Made Simple
> www.OpenNebula.org | jmelis at opennebula.org
>



-- 

runtastische Grüße · runtastic regards



*Armin Deliomini*

*Infrastructure engineer*


*Virtualization specialist*

*Storage engineer*

*DevOPS*


*OPS*

*[image: runtastic - makes sports funtastic] <http://www.runtastic.com>*

runtastic GmbH

Pluskaufstraße 7 · Business Center

4061 Pasching / Linz (Austria)

Tel. +43(0)67689692450

Fax. +43(0)7229.23327

armin.deliomini at runtastic.com

www.runtastic.com



<http://www.facebook.com/runtastic>   <http://www.twitter.com/runtastic>
<http://www.youtube.com/user/runtastic>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20140612/9a775d25/attachment.htm>