[one-users] system datastores always have no space

Marius Rex MariusRex at venda.com
Fri May 16 06:40:07 PDT 2014


Thank you for your continued help in investigating this.

For some reason when I checked, the NFS volume on the node as blank.  It
was mounted, but the file system appeared empty.  I re-mounted and it is
now responding in line with what we would expect:

[root at kvm04 ~]# /var/tmp/one/im/kvm-probes.d/monitor_ds.sh
DS_LOCATION_USED_MB=1
DS_LOCATION_TOTAL_MB=1408656
DS_LOCATION_FREE_MB=1408080
DS = [
  ID = 0,
  USED_MB = 1,
  TOTAL_MB = 1408656,
  FREE_MB = 1408080
]
DS = [
  ID = 1,
  USED_MB = 1,
  TOTAL_MB = 1408656,
  FREE_MB = 1408080
]
DS = [
  ID = 2,
  USED_MB = 1,
  TOTAL_MB = 1408656,
  FREE_MB = 1408080
]

I noted that permissions were off when I remounted.  I had to force NFS
version 3 to get the onaeadmin user permissions I had set originally.

This seems to have resolved my current problem.  I am still having permission issues when I launch a VM, 
but I will look further into that issue later today.

But thank you very much for your help and support so far. 



On Fri, 2014-05-09 at 18:08 -0400, Ruben S. Montero wrote:
> Does this directory exist /var/lib/one//datastores/0?
> 
> 
> On Fri, May 9, 2014 at 11:55 PM, Marius Rex <MariusRex at venda.com>
> wrote:
>         
>         I am not sure I follow what you are trying to say.  What is
>         the source
>         of my problem?
>         
>         All of my datastores, which live on the same NFS mount, exist
>         on the the
>         monitored host.  They are mounted in /var/lib/one, which
>         matches the
>         BASE PATH for the datastore.  So I do not know how the NFS
>         mount is not
>         'properly mounted.'
>         
>         The output I get from the command you specify is:
>         
>         [root at kvm04 ~]# /var/tmp/one/im/kvm-probes.d/monitor_ds.sh
>         kvm /var/lib/one/datastores
>         DS_LOCATION_USED_MB=1
>         DS_LOCATION_TOTAL_MB=1409258
>         DS_LOCATION_FREE_MB=1408682
>         
>         So it looks the probe is finding available space.  But
>         according to the
>         output of 'onedatastore' there is no available space.  Thus no
>         VMs can
>         be created on my KVM host/node.
>         
>         I am not sure what step or procedure I am missing.  I do not
>         see
>         anything in
>         http://docs.opennebula.org/4.6/design_and_installation/quick_starts/qs_centos_kvm.html that I have skipped.
>         
>         Does it matter that the KVM node already has VMs running on
>         it?
>         Otherwise the node fits the guide above precisely. The plan is
>         to put
>         everything under OpenNebula management.  But since the system
>         DS reports
>         having no space, I cannot launch a single VM, and thus will
>         not be able
>         to spin any of those machines down to bring them up under
>         OpenNebula
>         management.
>         
>         -Marius
>         
>         
>         
>         On Fri, 2014-05-09 at 15:35 -0400, Ruben S. Montero wrote:
>         >
>         > On Fri, May 9, 2014 at 3:43 PM, Marius Rex
>         <MariusRex at venda.com>
>         > wrote:
>         >
>         >         The system DS is not supposed to show up with any
>         space until
>         >         a host is
>         >         configured?  That is a surprising because the
>         pre-configured
>         >         sandbox
>         >         instance I downloaded first for testing (a
>         Virtualbox
>         >         instance) has
>         >         space listed in the system DS even though that has
>         no hosts
>         >         connected to
>         >         it.
>         >
>         >
>         > I'm not sure about the sandbox configuration (e.g. there
>         were some
>         > data from the setup). Anyway, system DS is monitored from
>         the hosts.
>         >  Note that System DS is not need (not even accessible for
>         some storage
>         > backends) in the front-end, so the space is reported by the
>         hosts.
>         > Also System DS is used to run VMs from it, and mainly used
>         in the
>         > hosts.
>         >
>         >
>         >
>         >         In any case, there are no hosts connected to the
>         test instance
>         >         that I
>         >         gave the example on, but on some other instances
>         which I have
>         >         configured
>         >         elsewhere do have hosts configured and they have the
>         same
>         >         problem.  I
>         >         have /var/lib/one as a NFS mount that is mounted on
>         both the
>         >         frontend
>         >         and the node.  That instance comes up with no space
>         in the
>         >         system DS as
>         >         well.
>         >
>         >
>         > This may be a problem with the configuration or the sytem DS
>         not
>         > properly mounted in the hosts. The information is obtained
>         > with /var/tmp/one/im/kvm-probes.d, go to the host kvm04 and
>         execute:
>         >
>         >
>         >  /var/tmp/one/im/kvm-probes.d/monitor_ds.sh
>         > kvm /var/lib/one/datastores/
>         >
>         >
>         > The output should be similar to...
>         >
>         >
>         > DS_LOCATION_USED_MB=1
>         > DS_LOCATION_TOTAL_MB=9952
>         > DS_LOCATION_FREE_MB=6318
>         > DS = [
>         >   ID = 101,
>         >   USED_MB = 1,
>         >   TOTAL_MB = 9952,
>         >   FREE_MB = 6318
>         > ]
>         >
>         >
>         >
>         >         Here is some output from that showing the system DS
>         with no
>         >         space even
>         >         though the logs say the node is successfully
>         monitored:
>         >         http://pastebin.com/w6Wg909Y
>         >
>         >
>         >         -Marius
>         >         mariusrex at venda.com
>         >
>         >
>         >         On Fri, 2014-05-09 at 05:05 -0400, Carlos Martín
>         Sánchez
>         >         wrote:
>         >         > Hi,
>         >         >
>         >         >
>         >         > System DS are monitored with the host probes. Do
>         you have
>         >         any hosts
>         >         > configured?
>         >         >
>         >         > --
>         >         > Carlos Martín, MSc
>         >         > Project Engineer
>         >         > OpenNebula - Flexible Enterprise Cloud Made Simple
>         >         > www.OpenNebula.org | cmartin at opennebula.org |
>         @OpenNebula
>         >         >
>         >         >
>         >         > On Fri, May 9, 2014 at 12:02 AM, Marius Rex
>         >         <MariusRex at venda.com>
>         >         > wrote:
>         >         >         Platform: Centos 6.5
>         >         >         versions: opennebula-4.5.80 and
>         opennebula-4.6.0-1
>         >         >
>         >         >         I have been struggling with this for a
>         while with
>         >         various
>         >         >         configurations
>         >         >         and I continually run into the same
>         problem.
>         >         >
>         >         >         I cannot build out a VM as the system
>         volumes always
>         >         come up
>         >         >         with no
>         >         >         space.  I have been trying to reduce
>         variables and
>         >         now at a
>         >         >         very basic
>         >         >         setup: I installed a fresh Centos 6 server
>         and
>         >         pulled down the
>         >         >         latest
>         >         >         stable opennebula RPMs from the official
>         repo.  I am
>         >         just
>         >         >         using local
>         >         >         file systems for everything, and have done
>         nothing
>         >         beyond
>         >         >         installing the
>         >         >         RPM and starting opennebula.  My system
>         volume comes
>         >         up with
>         >         >         no
>         >         >         available room, seemingly no matter what I
>         do.
>         >         >
>         >         >
>         >         >         [oneadmin at trainging ~]$ onedatastore
>         >         >         list
>         >         >           ID NAME                SIZE AVAIL
>         CLUSTER
>         >          IMAGES TYPE
>         >         >         DS
>         >         >         TM
>         >         >            0 system                0M -     -
>         >         0 sys  -
>         >         >         shared
>         >         >            1 default             6.6G 77%   -
>         >         0 img
>         >         >          fs
>         >         >         shared
>         >         >            2 files               6.6G 77%   -
>         >         0 fil
>         >         >          fs
>         >         >         ssh
>         >         >
>         >         >
>         >         >
>         >         >         [oneadmin at trainging ~]$ onedatastore show
>         0
>         >         >         DATASTORE 0
>         >         >         INFORMATION
>         >         >         ID             :
>         >         >         0
>         >         >         NAME           :
>         >         >         system
>         >         >         USER           : oneadmin
>         >         >         GROUP          : oneadmin
>         >         >         CLUSTER        : -
>         >         >         TYPE           : SYSTEM
>         >         >         DS_MAD         : -
>         >         >         TM_MAD         : shared
>         >         >         BASE PATH
>          : /var/lib/one//datastores/0
>         >         >         DISK_TYPE      : FILE
>         >         >
>         >         >         DATASTORE
>         >         >         CAPACITY
>         >         >         TOTAL:         : 0M
>         >         >         FREE:          : 0M
>         >         >         USED:          : 0M
>         >         >         LIMIT:         : -
>         >         >
>         >         >         PERMISSIONS
>         >         >         OWNER          : um-
>         >         >         GROUP          : u--
>         >         >         OTHER          : ---
>         >         >
>         >         >         DATASTORE
>         >         >         TEMPLATE
>         >         >         BASE_PATH="/var/lib/one//datastores/"
>         >         >         SHARED="YES"
>         >         >         TM_MAD="shared"
>         >         >         TYPE="SYSTEM_DS"
>         >         >
>         >         >         IMAGES
>         >         >
>         >         >         It is the same filesystem the other
>         volumes are on,
>         >         yet they
>         >         >         have
>         >         >         available space.  I have tried two
>         different
>         >         versions of
>         >         >         OpenNebula.  I
>         >         >         have tried different base filesystems.   I
>         have
>         >         tried custom
>         >         >         RPMs.  It
>         >         >         comes up the same every time.
>         >         >
>         >         >         Is there a bug or am I missing something
>         very
>         >         obvious?
>         >         >
>         >         >
>         >         >         --
>         >         >         Marius Rex
>         >         >         mariusrex at venda.com
>         >         >
>         _______________________________________________
>         >         >         Users mailing list
>         >         >         Users at lists.opennebula.org
>         >         >
>         >
>         http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>         >
>         >
>         >
>         >         _______________________________________________
>         >         Users mailing list
>         >         Users at lists.opennebula.org
>         >
>         http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>         >
>         >
>         >
>         >
>         >
>         > --
>         > --
>         >
>         > Ruben S. Montero, PhD
>         > Project co-Lead and Chief Architect
>         > OpenNebula - Flexible Enterprise Cloud Made Simple
>         > www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula
>         
>         
> 
> 
> 
> 
> -- 
> -- 
> 
> Ruben S. Montero, PhD
> Project co-Lead and Chief Architect
> OpenNebula - Flexible Enterprise Cloud Made Simple
> www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula

-- 
Marius Rex
mariusrex at venda.com



More information about the Users mailing list