[one-users] system datastores always have no space
Marius Rex
MariusRex at venda.com
Fri May 16 06:40:07 PDT 2014
Thank you for your continued help in investigating this.
For some reason when I checked, the NFS volume on the node as blank. It
was mounted, but the file system appeared empty. I re-mounted and it is
now responding in line with what we would expect:
[root at kvm04 ~]# /var/tmp/one/im/kvm-probes.d/monitor_ds.sh
DS_LOCATION_USED_MB=1
DS_LOCATION_TOTAL_MB=1408656
DS_LOCATION_FREE_MB=1408080
DS = [
ID = 0,
USED_MB = 1,
TOTAL_MB = 1408656,
FREE_MB = 1408080
]
DS = [
ID = 1,
USED_MB = 1,
TOTAL_MB = 1408656,
FREE_MB = 1408080
]
DS = [
ID = 2,
USED_MB = 1,
TOTAL_MB = 1408656,
FREE_MB = 1408080
]
I noted that permissions were off when I remounted. I had to force NFS
version 3 to get the onaeadmin user permissions I had set originally.
This seems to have resolved my current problem. I am still having permission issues when I launch a VM,
but I will look further into that issue later today.
But thank you very much for your help and support so far.
On Fri, 2014-05-09 at 18:08 -0400, Ruben S. Montero wrote:
> Does this directory exist /var/lib/one//datastores/0?
>
>
> On Fri, May 9, 2014 at 11:55 PM, Marius Rex <MariusRex at venda.com>
> wrote:
>
> I am not sure I follow what you are trying to say. What is
> the source
> of my problem?
>
> All of my datastores, which live on the same NFS mount, exist
> on the the
> monitored host. They are mounted in /var/lib/one, which
> matches the
> BASE PATH for the datastore. So I do not know how the NFS
> mount is not
> 'properly mounted.'
>
> The output I get from the command you specify is:
>
> [root at kvm04 ~]# /var/tmp/one/im/kvm-probes.d/monitor_ds.sh
> kvm /var/lib/one/datastores
> DS_LOCATION_USED_MB=1
> DS_LOCATION_TOTAL_MB=1409258
> DS_LOCATION_FREE_MB=1408682
>
> So it looks the probe is finding available space. But
> according to the
> output of 'onedatastore' there is no available space. Thus no
> VMs can
> be created on my KVM host/node.
>
> I am not sure what step or procedure I am missing. I do not
> see
> anything in
> http://docs.opennebula.org/4.6/design_and_installation/quick_starts/qs_centos_kvm.html that I have skipped.
>
> Does it matter that the KVM node already has VMs running on
> it?
> Otherwise the node fits the guide above precisely. The plan is
> to put
> everything under OpenNebula management. But since the system
> DS reports
> having no space, I cannot launch a single VM, and thus will
> not be able
> to spin any of those machines down to bring them up under
> OpenNebula
> management.
>
> -Marius
>
>
>
> On Fri, 2014-05-09 at 15:35 -0400, Ruben S. Montero wrote:
> >
> > On Fri, May 9, 2014 at 3:43 PM, Marius Rex
> <MariusRex at venda.com>
> > wrote:
> >
> > The system DS is not supposed to show up with any
> space until
> > a host is
> > configured? That is a surprising because the
> pre-configured
> > sandbox
> > instance I downloaded first for testing (a
> Virtualbox
> > instance) has
> > space listed in the system DS even though that has
> no hosts
> > connected to
> > it.
> >
> >
> > I'm not sure about the sandbox configuration (e.g. there
> were some
> > data from the setup). Anyway, system DS is monitored from
> the hosts.
> > Note that System DS is not need (not even accessible for
> some storage
> > backends) in the front-end, so the space is reported by the
> hosts.
> > Also System DS is used to run VMs from it, and mainly used
> in the
> > hosts.
> >
> >
> >
> > In any case, there are no hosts connected to the
> test instance
> > that I
> > gave the example on, but on some other instances
> which I have
> > configured
> > elsewhere do have hosts configured and they have the
> same
> > problem. I
> > have /var/lib/one as a NFS mount that is mounted on
> both the
> > frontend
> > and the node. That instance comes up with no space
> in the
> > system DS as
> > well.
> >
> >
> > This may be a problem with the configuration or the sytem DS
> not
> > properly mounted in the hosts. The information is obtained
> > with /var/tmp/one/im/kvm-probes.d, go to the host kvm04 and
> execute:
> >
> >
> > /var/tmp/one/im/kvm-probes.d/monitor_ds.sh
> > kvm /var/lib/one/datastores/
> >
> >
> > The output should be similar to...
> >
> >
> > DS_LOCATION_USED_MB=1
> > DS_LOCATION_TOTAL_MB=9952
> > DS_LOCATION_FREE_MB=6318
> > DS = [
> > ID = 101,
> > USED_MB = 1,
> > TOTAL_MB = 9952,
> > FREE_MB = 6318
> > ]
> >
> >
> >
> > Here is some output from that showing the system DS
> with no
> > space even
> > though the logs say the node is successfully
> monitored:
> > http://pastebin.com/w6Wg909Y
> >
> >
> > -Marius
> > mariusrex at venda.com
> >
> >
> > On Fri, 2014-05-09 at 05:05 -0400, Carlos Martín
> Sánchez
> > wrote:
> > > Hi,
> > >
> > >
> > > System DS are monitored with the host probes. Do
> you have
> > any hosts
> > > configured?
> > >
> > > --
> > > Carlos Martín, MSc
> > > Project Engineer
> > > OpenNebula - Flexible Enterprise Cloud Made Simple
> > > www.OpenNebula.org | cmartin at opennebula.org |
> @OpenNebula
> > >
> > >
> > > On Fri, May 9, 2014 at 12:02 AM, Marius Rex
> > <MariusRex at venda.com>
> > > wrote:
> > > Platform: Centos 6.5
> > > versions: opennebula-4.5.80 and
> opennebula-4.6.0-1
> > >
> > > I have been struggling with this for a
> while with
> > various
> > > configurations
> > > and I continually run into the same
> problem.
> > >
> > > I cannot build out a VM as the system
> volumes always
> > come up
> > > with no
> > > space. I have been trying to reduce
> variables and
> > now at a
> > > very basic
> > > setup: I installed a fresh Centos 6 server
> and
> > pulled down the
> > > latest
> > > stable opennebula RPMs from the official
> repo. I am
> > just
> > > using local
> > > file systems for everything, and have done
> nothing
> > beyond
> > > installing the
> > > RPM and starting opennebula. My system
> volume comes
> > up with
> > > no
> > > available room, seemingly no matter what I
> do.
> > >
> > >
> > > [oneadmin at trainging ~]$ onedatastore
> > > list
> > > ID NAME SIZE AVAIL
> CLUSTER
> > IMAGES TYPE
> > > DS
> > > TM
> > > 0 system 0M - -
> > 0 sys -
> > > shared
> > > 1 default 6.6G 77% -
> > 0 img
> > > fs
> > > shared
> > > 2 files 6.6G 77% -
> > 0 fil
> > > fs
> > > ssh
> > >
> > >
> > >
> > > [oneadmin at trainging ~]$ onedatastore show
> 0
> > > DATASTORE 0
> > > INFORMATION
> > > ID :
> > > 0
> > > NAME :
> > > system
> > > USER : oneadmin
> > > GROUP : oneadmin
> > > CLUSTER : -
> > > TYPE : SYSTEM
> > > DS_MAD : -
> > > TM_MAD : shared
> > > BASE PATH
> : /var/lib/one//datastores/0
> > > DISK_TYPE : FILE
> > >
> > > DATASTORE
> > > CAPACITY
> > > TOTAL: : 0M
> > > FREE: : 0M
> > > USED: : 0M
> > > LIMIT: : -
> > >
> > > PERMISSIONS
> > > OWNER : um-
> > > GROUP : u--
> > > OTHER : ---
> > >
> > > DATASTORE
> > > TEMPLATE
> > > BASE_PATH="/var/lib/one//datastores/"
> > > SHARED="YES"
> > > TM_MAD="shared"
> > > TYPE="SYSTEM_DS"
> > >
> > > IMAGES
> > >
> > > It is the same filesystem the other
> volumes are on,
> > yet they
> > > have
> > > available space. I have tried two
> different
> > versions of
> > > OpenNebula. I
> > > have tried different base filesystems. I
> have
> > tried custom
> > > RPMs. It
> > > comes up the same every time.
> > >
> > > Is there a bug or am I missing something
> very
> > obvious?
> > >
> > >
> > > --
> > > Marius Rex
> > > mariusrex at venda.com
> > >
> _______________________________________________
> > > Users mailing list
> > > Users at lists.opennebula.org
> > >
> >
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users at lists.opennebula.org
> >
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
> >
> >
> >
> >
> >
> > --
> > --
> >
> > Ruben S. Montero, PhD
> > Project co-Lead and Chief Architect
> > OpenNebula - Flexible Enterprise Cloud Made Simple
> > www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula
>
>
>
>
>
>
> --
> --
>
> Ruben S. Montero, PhD
> Project co-Lead and Chief Architect
> OpenNebula - Flexible Enterprise Cloud Made Simple
> www.OpenNebula.org | rsmontero at opennebula.org | @OpenNebula
--
Marius Rex
mariusrex at venda.com
More information about the Users
mailing list