[one-users] ONE 4.8 + Gluster-3.4.5 on Centos 6.5 -- VMs stuck in BOOT

[Javier Fontan]

That's right. Even if it is using GlusterFS the way of accessing the
files is using the fuse filesystem. This makes the IO performance
suffer.

Do you get any errors in the log files related to gluster when you try
to boot a machine with DISK_TYPE="GLUSTER" is activated? It could be a
firewall or a permission problem. Make sure that the hypervisor host
can access clu100 port 24007.

Also make sure that the server has the rpc-auth-allow-insecure option
configured and was restarted after the change.

On Sat, Aug 30, 2014 at 11:44 AM, Marco Aroldi <marco.aroldi at gmail.com> wrote:
> Hi all,
> this is my first post to the list
>
> My goal is to get ONE 4.8 up and running using Gluster as datastore,
> everything on CentOS 6.5
> The problem: the VM remains stuck in BOOT status
> I've found a way to boot the machines (see below), but I think is not
> the correct way to manage this setup.
>
> First, let me describe what I've done until now:
> I've followed the docs at
> http://docs.opennebula.org/4.8/administration/storage/gluster_ds.html
> and the post on the blog by Javier Fontan
> http://opennebula.org/native-glusterfs-image-access-for-kvm-drivers/
>
> This is my Gluster volume:
> Volume Name: sys-one
> Type: Replicate
> Volume ID: f1bf1bcc-0280-46db-aab8-69fd34672263
> Status: Started
> Number of Bricks: 1 x 2 = 2
> Transport-type: tcp
> Bricks:
> Brick1: clu001:/one
> Brick2: clu100:/one
> Options Reconfigured:
> cluster.server-quorum-type: server
> cluster.quorum-type: auto
> network.remote-dio: enable
> cluster.eager-lock: enable
> performance.stat-prefetch: on
> performance.io-cache: off
> performance.read-ahead: off
> performance.quick-read: off
> storage.owner-gid: 9869
> storage.owner-uid: 9869
> server.allow-insecure: on
>
> And the datastores:
>   ID NAME                SIZE AVAIL CLUSTER      IMAGES TYPE DS       TM
>    1 default           230.7G 86%   -                 6 img  fs       shared
>    2 files             230.7G 86%   -                 0 fil  fs       ssh
>  108 GLUSTER              24G 52%   clussssss         2 img  fs       shared
>  110 new system           24G 52%   clussssss         0 sys  -        shared
>
>
> DATASTORE 108 INFORMATION
> ID             : 108
> NAME           : GLUSTER
> USER           : oneadmin
> GROUP          : oneadmin
> CLUSTER        : clussssss
> TYPE           : IMAGE
> DS_MAD         : fs
> TM_MAD         : shared
> BASE PATH      : /var/lib/one//datastores/108
> DISK_TYPE      :
>
> DATASTORE CAPACITY
> TOTAL:         : 24G
> FREE:          : 12.5G
> USED:          : 6.3G
> LIMIT:         : 12.7G
>
> PERMISSIONS
> OWNER          : um-
> GROUP          : u--
> OTHER          : ---
>
> DATASTORE TEMPLATE
> BASE_PATH="/var/lib/one//datastores/"
> CLONE_TARGET="SYSTEM"
> DISK_TYPE="GLUSTER"
> DS_MAD="fs"
> GLUSTER_HOST="clu100:24007"
> GLUSTER_VOLUME="sys-one"
> LIMIT_MB="13000"
> LN_TARGET="NONE"
> TM_MAD="shared"
> TYPE="IMAGE_DS"
>
> DATASTORE 110 INFORMATION
> ID             : 110
> NAME           : new system
> USER           : oneadmin
> GROUP          : oneadmin
> CLUSTER        : clussssss
> TYPE           : SYSTEM
> DS_MAD         : -
> TM_MAD         : shared
> BASE PATH      : /var/lib/one//datastores/110
> DISK_TYPE      : FILE
>
> DATASTORE CAPACITY
> TOTAL:         : 24G
> FREE:          : 12.5G
> USED:          : 6.3G
> LIMIT:         : -
>
> PERMISSIONS
> OWNER          : um-
> GROUP          : u--
> OTHER          : ---
>
> DATASTORE TEMPLATE
> BASE_PATH="/var/lib/one//datastores/"
> SHARED="YES"
> TM_MAD="shared"
> TYPE="SYSTEM_DS"
>
>
> Here is the mounted glusterfs:
> clu100:/sys-one on /gluster type fuse.glusterfs
> (rw,default_permissions,allow_other,max_read=131072)
>
> And the symbolic links in the datastores directory:
> lrwxrwxrwx  1 oneadmin oneadmin    8 Aug 30 10:18 108 -> /gluster
> lrwxrwxrwx  1 oneadmin oneadmin    8 Aug 30 10:18 110 -> /gluster
>
> I've found the culprit in the system datastore:
> Created a new system datastore ON THE LOCAL FILESYSTEM:
>
> 111 system            230.7G 86%   -                 0 sys  -        shared
>
> DATASTORE 111 INFORMATION
> ID             : 111
> NAME           : system
> USER           : oneadmin
> GROUP          : oneadmin
> CLUSTER        : -
> TYPE           : SYSTEM
> DS_MAD         : -
> TM_MAD         : shared
> BASE PATH      : /var/lib/one//datastores/111
> DISK_TYPE      : FILE
>
> DATASTORE CAPACITY
> TOTAL:         : 230.7G
> FREE:          : 199.2G
> USED:          : 1M
> LIMIT:         : -
>
> PERMISSIONS
> OWNER          : um-
> GROUP          : u--
> OTHER          : ---
>
> DATASTORE TEMPLATE
> BASE_PATH="/var/lib/one//datastores/"
> SHARED="YES"
> TM_MAD="shared"
> TYPE="SYSTEM_DS"
>
> Deploying now puts the VM in RUNNING status but, correct me if I'm
> wrong, this setup is not compliant, right?
> Thanks for the help
>
> Marco
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org



-- 
Javier Fontán Muiños
Developer
OpenNebula - Flexible Enterprise Cloud Made Simple
www.OpenNebula.org | @OpenNebula | github.com/jfontan