[one-users] ONE 4.8 + Gluster-3.4.5 on Centos 6.5 -- VMs stuck in BOOT

Amier Anis mymabma at gmail.com
Fri Sep 26 05:23:01 PDT 2014


*Dear Marco, *

Try this guide which i also get some clarification from Javier. I have try
and it work better.
Somehow, how you physical environment and network setup will give impact to
the performance.

I assume you fresh install and want to share "system" datastores 0 and
"images" datastore 1. If you need add additional both system for cluster
them just creat an empty template .... Datastore -> fill in name -> select
datastore type either images or system -> click save

----------- Begin --------------

1. Please ensure turn-off iptables to all nodes to ensure it works.
2. You share gluster and cluster node. but will slow down if you dont have
dedicated network card to it.

---------------------
Gluster Storage - (standalone)
---------------------

# wget -P /etc/yum.repos.d
http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo
# yum -y install glusterfs glusterfs-fuse glusterfs-server
# chkconfig glusterd on

# vi /etc/glusterfs/glusterd.vol

volume management
  option rpc-auth-allow-insecure on
  option bind-address 172.20.11.11  <----- change to your gluster storage
ip address
end-volume

# service glusterd start

We create glusterfs mount folder. Pls ensure the userid and groupid exist
in the gluster storage
# mkdir -p /mnt/ds0/datastores0
# chown oneadmin:oneadmin /mnt/ds0/datastores0
# chmod 775 /mnt/ds0/datastores0

# gluster volume create datastores0 172.20.11.11:/mnt/ds0/datastores0
# gluster volume start datastores0
# gluster volume set datastores0 server.allow-insecure on
# gluster volume set datastores0 storage.owner-uid 9869
# gluster volume set datastores0 storage.owner-gid 9869
# gluster volume set datastores0 auth.allow 172.20.*             <----
change to you ip subnet

# gluster volume info

------------------
Front Node
------------------

# wget -P /etc/yum.repos.d
http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo
# yum -y install glusterfs glusterfs-fuse

# su - oneadmin
$ cd datastores
$ rm -rf /var/lib/one/datastores/1
$ rm -rf /var/lib/one/datastores/2

$ ln -s /var/lib/one/datastores/0 /var/lib/one/datastores/1
$ ln -s /var/lib/one/datastores/0 /var/lib/one/datastores/2

Put permanent setting during boot
# vi /etc/fstab
172.20.11.11:/datastores0 /var/lib/one/datastores/0 glusterfs
defaults,_netdev 0 0

# mount -a
# df -h

hope no error.

Go sunstone n refresh datastore page. makesure you already add your first
cluster node.

in front end should mount all gluster datastores

------------------
Cluster Node
------------------
# wget -P /etc/yum.repos.d
http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo
# yum -y install glusterfs glusterfs-fuse

# su - oneadmin

We create default folder
$ mkdir -p /var/lib/one/datastores/0
$ ln -s /var/lib/one/datastores/0 /var/lib/one/datastores/1
$ ln -s /var/lib/one/datastores/0 /var/lib/one/datastores/2

$ exit

NOTE: Next two line if you create additional datastores as i mention
earlier. We only create system and images datastores in cluster node
$ mkdir -p /var/lib/one/datastores/100
$ ln -s /var/lib/one/datastores/100 /var/lib/one/datastores/101

$ exit

Put permanent setting during boot and point to system datastore. image just
a symlink.
# vi /etc/fstab
172.20.11.11:/datastores0 /var/lib/one/datastores/0 glusterfs
defaults,_netdev 0 0

# mount -a
# df -h

--------------- end ------------------

hope this will help you.

Regards & Best Wishes,


*.: Amier Anis :.*
Mobile: +6012-260-0819

On Wed, Sep 10, 2014 at 6:09 PM, Marco Aroldi <marco.aroldi at gmail.com>
wrote:

> Hi, thanks for the answer.
>
> The hypervisor hosts can telnet both clu100 and clu001 on port 24007
>
> I'm sorry, I'm a OpenNebual newbie, so it's not clear to me where to
> put the DISK_TYPE="GLUSTER" option
> So I've tired to put the option first in the image attributes, then in
> the template custom tags, but nothing has changed
> I don't get any message related to Gluster in logs
>
> I'm scratching my head.
> Any help is very appreciated. Thanks!
>
> Wed Sep 10 11:49:22 2014 [Z0][DiM][I]: New VM state is ACTIVE.
> Wed Sep 10 11:49:22 2014 [Z0][LCM][I]: New VM state is PROLOG.
> Wed Sep 10 11:49:46 2014 [Z0][LCM][I]: New VM state is BOOT
> Wed Sep 10 11:49:46 2014 [Z0][VMM][I]: Generating deployment file:
> /var/lib/one/vms/53/deployment.0
> Wed Sep 10 11:49:47 2014 [Z0][VMM][I]: ExitCode: 0
> Wed Sep 10 11:49:47 2014 [Z0][VMM][I]: Successfully execute network
> driver operation: pre.
>
> The deployment file:
>
> <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0
> '>
>         <name>one-53</name>
>         <cputune>
>                 <shares>1024</shares>
>         </cputune>
>         <memory>786432</memory>
>         <os>
>                 <type arch='x86_64'>hvm</type>
>                 <boot dev='hd'/>
>         </os>
>         <devices>
>                 <emulator>/usr/libexec/qemu-kvm</emulator>
>                 <disk type='network' device='disk'>
>                         <source protocol='gluster'
> name='sys-one/53/disk.0'>
>                                 <host name='clu100' port='24007'
> transport='tcp'/>
>                         </source>
>                         <target dev='hda'/>
>                         <driver name='qemu' type='qcow2' cache='none'/>
>                 </disk>
>                 <disk type='file' device='cdrom'>
>                         <source
> file='/var/lib/one//datastores/110/53/disk.1'/>
>                         <target dev='hdb'/>
>                         <readonly/>
>                         <driver name='qemu' type='raw'/>
>                 </disk>
>                 <interface type='bridge'>
>                         <source bridge='br0'/>
>                         <mac address='02:00:c0:a8:16:7e'/>
>                 </interface>
>                 <graphics type='vnc' listen='0.0.0.0' port='5953'/>
>         </devices>
>         <features>
>                 <acpi/>
>         </features>
> </domain>
>
>
> oneimage show:
>
> IMAGE 19 INFORMATION
> ID             : 19
> NAME           : CentOS-6.5-one-4.8_GLUSTER
> USER           : oneadmin
> GROUP          : oneadmin
> DATASTORE      : GLUSTER
> TYPE           : OS
> REGISTER TIME  : 08/30 10:41:12
> PERSISTENT     : No
> SOURCE         :
> /var/lib/one//datastores/108/c6780b5c1667ec829b9ed92f7853f934
> PATH           :
> http://marketplace.c12g.com/appliance/53e767ba8fb81d6a69000001/download/0
> SIZE           : 10G
> STATE          : used
> RUNNING_VMS    : 1
>
> PERMISSIONS
> OWNER          : um-
> GROUP          : ---
> OTHER          : ---
>
> IMAGE TEMPLATE
> DEV_PREFIX="hd"
> DISK_TYPE="GLUSTER"
> FROM_APP="53e767ba8fb81d6a69000001"
> FROM_APP_FILE="0"
> FROM_APP_NAME="CentOS 6.5 - KVM - OpenNebula 4.8"
> MD5="9d937b8fe70c403330c9284538f07cfc"
>
> VIRTUAL MACHINES
>
>     ID USER     GROUP    NAME            STAT UCPU    UMEM HOST
>  TIME
>     53 oneadmin oneadmin CentOS 6.5 - KV boot    0    768M pre-openne   0d
> 00h12
>
>
> onetemplate show
>
> TEMPLATE 8 INFORMATION
> ID             : 8
> NAME           : CentOS 6.5 - KVM - OpenNebula 4.8_GLSUTER
> USER           : oneadmin
> GROUP          : oneadmin
> REGISTER TIME  : 08/30 10:41:13
>
> PERMISSIONS
> OWNER          : um-
> GROUP          : ---
> OTHER          : ---
>
> TEMPLATE CONTENTS
> CONTEXT=[
>   NETWORK="YES",
>   SSH_PUBLIC_KEY="$USER[SSH_PUBLIC_KEY]" ]
> CPU="1"
> DISK=[
>   DRIVER="qcow2",
>   IMAGE="CentOS-6.5-one-4.8_GLUSTER",
>   IMAGE_UNAME="oneadmin" ]
> DISK_TYPE="GLUSTER"
> FROM_APP="53e767ba8fb81d6a69000001"
> FROM_APP_NAME="CentOS 6.5 - KVM - OpenNebula 4.8"
> GRAPHICS=[
>   LISTEN="0.0.0.0",
>   TYPE="vnc" ]
> MEMORY="768"
> NIC=[
>   NETWORK="private",
>   NETWORK_UNAME="oneadmin" ]
>
>
> 2014-09-09 12:38 GMT+02:00 Javier Fontan <jfontan at opennebula.org>:
> > That's right. Even if it is using GlusterFS the way of accessing the
> > files is using the fuse filesystem. This makes the IO performance
> > suffer.
> >
> > Do you get any errors in the log files related to gluster when you try
> > to boot a machine with DISK_TYPE="GLUSTER" is activated? It could be a
> > firewall or a permission problem. Make sure that the hypervisor host
> > can access clu100 port 24007.
> >
> > Also make sure that the server has the rpc-auth-allow-insecure option
> > configured and was restarted after the change.
> >
> > On Sat, Aug 30, 2014 at 11:44 AM, Marco Aroldi <marco.aroldi at gmail.com>
> wrote:
> >> Hi all,
> >> this is my first post to the list
> >>
> >> My goal is to get ONE 4.8 up and running using Gluster as datastore,
> >> everything on CentOS 6.5
> >> The problem: the VM remains stuck in BOOT status
> >> I've found a way to boot the machines (see below), but I think is not
> >> the correct way to manage this setup.
> >>
> >> First, let me describe what I've done until now:
> >> I've followed the docs at
> >> http://docs.opennebula.org/4.8/administration/storage/gluster_ds.html
> >> and the post on the blog by Javier Fontan
> >> http://opennebula.org/native-glusterfs-image-access-for-kvm-drivers/
> >>
> >> This is my Gluster volume:
> >> Volume Name: sys-one
> >> Type: Replicate
> >> Volume ID: f1bf1bcc-0280-46db-aab8-69fd34672263
> >> Status: Started
> >> Number of Bricks: 1 x 2 = 2
> >> Transport-type: tcp
> >> Bricks:
> >> Brick1: clu001:/one
> >> Brick2: clu100:/one
> >> Options Reconfigured:
> >> cluster.server-quorum-type: server
> >> cluster.quorum-type: auto
> >> network.remote-dio: enable
> >> cluster.eager-lock: enable
> >> performance.stat-prefetch: on
> >> performance.io-cache: off
> >> performance.read-ahead: off
> >> performance.quick-read: off
> >> storage.owner-gid: 9869
> >> storage.owner-uid: 9869
> >> server.allow-insecure: on
> >>
> >> And the datastores:
> >>   ID NAME                SIZE AVAIL CLUSTER      IMAGES TYPE DS       TM
> >>    1 default           230.7G 86%   -                 6 img  fs
>  shared
> >>    2 files             230.7G 86%   -                 0 fil  fs
>  ssh
> >>  108 GLUSTER              24G 52%   clussssss         2 img  fs
>  shared
> >>  110 new system           24G 52%   clussssss         0 sys  -
> shared
> >>
> >>
> >> DATASTORE 108 INFORMATION
> >> ID             : 108
> >> NAME           : GLUSTER
> >> USER           : oneadmin
> >> GROUP          : oneadmin
> >> CLUSTER        : clussssss
> >> TYPE           : IMAGE
> >> DS_MAD         : fs
> >> TM_MAD         : shared
> >> BASE PATH      : /var/lib/one//datastores/108
> >> DISK_TYPE      :
> >>
> >> DATASTORE CAPACITY
> >> TOTAL:         : 24G
> >> FREE:          : 12.5G
> >> USED:          : 6.3G
> >> LIMIT:         : 12.7G
> >>
> >> PERMISSIONS
> >> OWNER          : um-
> >> GROUP          : u--
> >> OTHER          : ---
> >>
> >> DATASTORE TEMPLATE
> >> BASE_PATH="/var/lib/one//datastores/"
> >> CLONE_TARGET="SYSTEM"
> >> DISK_TYPE="GLUSTER"
> >> DS_MAD="fs"
> >> GLUSTER_HOST="clu100:24007"
> >> GLUSTER_VOLUME="sys-one"
> >> LIMIT_MB="13000"
> >> LN_TARGET="NONE"
> >> TM_MAD="shared"
> >> TYPE="IMAGE_DS"
> >>
> >> DATASTORE 110 INFORMATION
> >> ID             : 110
> >> NAME           : new system
> >> USER           : oneadmin
> >> GROUP          : oneadmin
> >> CLUSTER        : clussssss
> >> TYPE           : SYSTEM
> >> DS_MAD         : -
> >> TM_MAD         : shared
> >> BASE PATH      : /var/lib/one//datastores/110
> >> DISK_TYPE      : FILE
> >>
> >> DATASTORE CAPACITY
> >> TOTAL:         : 24G
> >> FREE:          : 12.5G
> >> USED:          : 6.3G
> >> LIMIT:         : -
> >>
> >> PERMISSIONS
> >> OWNER          : um-
> >> GROUP          : u--
> >> OTHER          : ---
> >>
> >> DATASTORE TEMPLATE
> >> BASE_PATH="/var/lib/one//datastores/"
> >> SHARED="YES"
> >> TM_MAD="shared"
> >> TYPE="SYSTEM_DS"
> >>
> >>
> >> Here is the mounted glusterfs:
> >> clu100:/sys-one on /gluster type fuse.glusterfs
> >> (rw,default_permissions,allow_other,max_read=131072)
> >>
> >> And the symbolic links in the datastores directory:
> >> lrwxrwxrwx  1 oneadmin oneadmin    8 Aug 30 10:18 108 -> /gluster
> >> lrwxrwxrwx  1 oneadmin oneadmin    8 Aug 30 10:18 110 -> /gluster
> >>
> >> I've found the culprit in the system datastore:
> >> Created a new system datastore ON THE LOCAL FILESYSTEM:
> >>
> >> 111 system            230.7G 86%   -                 0 sys  -
> shared
> >>
> >> DATASTORE 111 INFORMATION
> >> ID             : 111
> >> NAME           : system
> >> USER           : oneadmin
> >> GROUP          : oneadmin
> >> CLUSTER        : -
> >> TYPE           : SYSTEM
> >> DS_MAD         : -
> >> TM_MAD         : shared
> >> BASE PATH      : /var/lib/one//datastores/111
> >> DISK_TYPE      : FILE
> >>
> >> DATASTORE CAPACITY
> >> TOTAL:         : 230.7G
> >> FREE:          : 199.2G
> >> USED:          : 1M
> >> LIMIT:         : -
> >>
> >> PERMISSIONS
> >> OWNER          : um-
> >> GROUP          : u--
> >> OTHER          : ---
> >>
> >> DATASTORE TEMPLATE
> >> BASE_PATH="/var/lib/one//datastores/"
> >> SHARED="YES"
> >> TM_MAD="shared"
> >> TYPE="SYSTEM_DS"
> >>
> >> Deploying now puts the VM in RUNNING status but, correct me if I'm
> >> wrong, this setup is not compliant, right?
> >> Thanks for the help
> >>
> >> Marco
> >> _______________________________________________
> >> Users mailing list
> >> Users at lists.opennebula.org
> >> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
> >
> >
> >
> > --
> > Javier Fontán Muiños
> > Developer
> > OpenNebula - Flexible Enterprise Cloud Made Simple
> > www.OpenNebula.org | @OpenNebula | github.com/jfontan
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20140926/fd1c413b/attachment-0001.htm>


More information about the Users mailing list