<div dir="ltr">Hi,<div><br></div><div>did this work for you?</div><div><br></div><div>cheers,<br>Jaime</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jul 14, 2014 at 2:38 AM, <a href="mailto:clm_tuan@hotmail.com">clm_tuan@hotmail.com</a> <span dir="ltr"><<a href="mailto:clm_tuan@hotmail.com" target="_blank">clm_tuan@hotmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>
<div><span></span><span style="color:rgb(51,51,51);font-family:arial;line-height:24px;background-color:rgb(251,252,254)">Thank you for your reply.</span></div>
<div><br></div>Regards.<br>
<blockquote style="margin-top:0px;margin-bottom:0px;margin-left:0.5em"><div> </div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"><div style="PADDING-RIGHT:8px;PADDING-LEFT:8px;FONT-SIZE:12px;FONT-FAMILY:tahoma;COLOR:#000000;BACKGROUND:#efefef;PADDING-BOTTOM:8px;PADDING-TOP:8px">
<div><b>From:</b> <a href="mailto:vincent@vanderkussen.org" target="_blank">vincent</a></div><div><b>Date:</b> 2014-07-12 18:30</div><div><b>To:</b> <a href="mailto:clm_tuan@hotmail.com" target="_blank">clm_tuan</a></div>
<div><b>CC:</b> <a href="mailto:users@lists.opennebula.org" target="_blank">users</a></div><div><b>Subject:</b> Re: [one-users] deploy Non-persistent vm on 4.6.2 base glusterfs no running</div></div></div><div><div class="h5">
<div><div>On 2014-07-11 10:17, <a href="mailto:clm_tuan@hotmail.com" target="_blank">clm_tuan@hotmail.com</a> wrote:</div>
<div>> Dear All,</div>
<div>> </div>
<div>> I tried to deploy opennebula 4.6.2 base glusterfs 3.5.1</div>
<div>> </div>
<div>> all three server:</div>
<div>> 1. one server,ip is 192.168.0.198;</div>
<div>> 2. node0,ip is 192.168.0.199;</div>
<div>> 3. node1,ip is 192.168.0.200;</div>
<div>> </div>
<div>> Datastores configuration:</div>
<div>> </div>
<div>> [oneadmin@fc-server ]$ onedatastore show 102</div>
<div>> DATASTORE 102 INFORMATION</div>
<div>> ID : 102</div>
<div>> NAME : sys</div>
<div>> USER : oneadmin</div>
<div>> GROUP : oneadmin</div>
<div>> CLUSTER : c1</div>
<div>> TYPE : SYSTEM</div>
<div>> DS_MAD : -</div>
<div>> TM_MAD : shared</div>
<div>> BASE PATH : /var/lib/one//datastores/102</div>
<div>> DISK_TYPE : FILE</div>
<div>> </div>
<div>> DATASTORE CAPACITY</div>
<div>> TOTAL: : 131.3G</div>
<div>> FREE: : 123.3G</div>
<div>> USED: : 41M</div>
<div>> LIMIT: : -</div>
<div>> </div>
<div>> PERMISSIONS</div>
<div>> OWNER : um-</div>
<div>> GROUP : u--</div>
<div>> OTHER : ---</div>
<div>> </div>
<div>> DATASTORE TEMPLATE</div>
<div>> BASE_PATH="/var/lib/one//datastores/"</div>
<div>> GLUSTER_HOST="<a href="http://192.168.0.199:24007" target="_blank">192.168.0.199:24007</a>"</div>
<div>> GLUSTER_VOLUME="fc-sys"</div>
<div>> SHARED="YES"</div>
<div>> TM_MAD="shared"</div>
<div>> TYPE="SYSTEM_DS"</div>
<div>> </div>
<div>> [oneadmin@fc-server ]$ onedatastore show 103</div>
<div>> DATASTORE 103 INFORMATION</div>
<div>> ID : 103</div>
<div>> NAME : img</div>
<div>> USER : oneadmin</div>
<div>> GROUP : oneadmin</div>
<div>> CLUSTER : c1</div>
<div>> TYPE : IMAGE</div>
<div>> DS_MAD : fs</div>
<div>> TM_MAD : shared</div>
<div>> BASE PATH : /var/lib/one//datastores/103</div>
<div>> DISK_TYPE :</div>
<div>> </div>
<div>> DATASTORE CAPACITY</div>
<div>> TOTAL: : 131.3G</div>
<div>> FREE: : 123.3G</div>
<div>> USED: : 41M</div>
<div>> LIMIT: : -</div>
<div>> </div>
<div>> PERMISSIONS</div>
<div>> OWNER : um-</div>
<div>> GROUP : u--</div>
<div>> OTHER : ---</div>
<div>> </div>
<div>> DATASTORE TEMPLATE</div>
<div>> BASE_PATH="/var/lib/one//datastores/"</div>
<div>> CLONE_TARGET="SYSTEM"</div>
<div>> DISK_TYPE="GLUSTER"</div>
<div>> DS_MAD="fs"</div>
<div>> GLUSTER_HOST="<a href="http://192.168.0.199:24007" target="_blank">192.168.0.199:24007</a>"</div>
<div>> GLUSTER_VOLUME="fc-img"</div>
<div>> LN_TARGET="NONE"</div>
<div>> TM_MAD="shared"</div>
<div>> TYPE="IMAGE_DS"</div>
<div>> </div>
<div>> 192.168.0.199:/fc-sys on /var/lib/one/datastores/102 type</div>
<div>> fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)</div>
<div>> 192.168.0.199:/fc-img on /var/lib/one/datastores/103 type</div>
<div>> fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)</div>
<div>> </div>
<div>> deploy image to PERSISTENT, it's ok.</div>
<div>> </div>
<div>> [oneadmin@fc-server ]$ oneimage show 1</div>
<div>> IMAGE 1 INFORMATION</div>
<div>> ID : 1</div>
<div>> NAME : ttylinux</div>
<div>> USER : oneadmin</div>
<div>> GROUP : oneadmin</div>
<div>> DATASTORE : img</div>
<div>> TYPE : OS</div>
<div>> REGISTER TIME : 07/11 21:35:11</div>
<div>> PERSISTENT : Yes</div>
<div>> SOURCE : /var/lib/one//datastores/103/4c40e4068ad2ff41c8767c40f85ae786</div>
<div>> </div>
<div>> PATH :</div>
<div>> <a href="http://marketplace.c12g.com/appliance/4fc76a938fb81d3517000003/download/0" target="_blank">http://marketplace.c12g.com/appliance/4fc76a938fb81d3517000003/download/0</a></div>
<div>> </div>
<div>> SIZE : 40M</div>
<div>> STATE : used</div>
<div>> RUNNING_VMS : 1</div>
<div>> </div>
<div>> PERMISSIONS</div>
<div>> OWNER : um-</div>
<div>> GROUP : ---</div>
<div>> OTHER : ---</div>
<div>> </div>
<div>> IMAGE TEMPLATE</div>
<div>> DEV_PREFIX="hd"</div>
<div>> FROM_APP="4fc76a938fb81d3517000003"</div>
<div>> FROM_APP_FILE="0"</div>
<div>> FROM_APP_NAME="ttylinux - kvm"</div>
<div>> MD5="04c7d00e88fa66d9aaa34d9cf8ad6aaa"</div>
<div>> </div>
<div>> VIRTUAL MACHINES</div>
<div>> </div>
<div>> ID USER GROUP NAME STAT UCPU UMEM HOST TIME</div>
<div>> 8 oneadmin oneadmin tt runn 0 512M 192.168.0. -1d 16h32</div>
<div>> [oneadmin@fc-server tm]$</div>
<div>> </div>
<div>> [oneadmin@fc-server 8]$ ll</div>
<div>> total 366</div>
<div>> -rw-rw-r-- 1 oneadmin oneadmin 992 Jul 11 13:44 deployment.0</div>
<div>> lrwxrwxrwx 1 oneadmin oneadmin 60 Jul 11 13:44 disk.0 -></div>
<div>> /var/lib/one/datastores/103/4c40e4068ad2ff41c8767c40f85ae786</div>
<div>> -rw-r--r-- 1 oneadmin oneadmin 372736 Jul 11 13:44 disk.1</div>
<div>> lrwxrwxrwx 1 oneadmin oneadmin 36 Jul 11 13:44 disk.1.iso -></div>
<div>> /var/lib/one/datastores/102/8/disk.1</div>
<div>> </div>
<div>> but deploy image to Non-persistent, it's no running.</div>
<div>> </div>
<div>> [oneadmin@fc-server 102]$ oneimage show 1</div>
<div>> IMAGE 1 INFORMATION</div>
<div>> ID : 1</div>
<div>> NAME : ttylinux</div>
<div>> USER : oneadmin</div>
<div>> GROUP : oneadmin</div>
<div>> DATASTORE : img</div>
<div>> TYPE : OS</div>
<div>> REGISTER TIME : 07/11 21:35:11</div>
<div>> PERSISTENT : No</div>
<div>> SOURCE : /var/lib/one//datastores/103/4c40e4068ad2ff41c8767c40f85ae786</div>
<div>> </div>
<div>> PATH :</div>
<div>> <a href="http://marketplace.c12g.com/appliance/4fc76a938fb81d3517000003/download/0" target="_blank">http://marketplace.c12g.com/appliance/4fc76a938fb81d3517000003/download/0</a></div>
<div>> </div>
<div>> SIZE : 40M</div>
<div>> STATE : used</div>
<div>> RUNNING_VMS : 1</div>
<div>> </div>
<div>> PERMISSIONS</div>
<div>> OWNER : um-</div>
<div>> GROUP : ---</div>
<div>> OTHER : ---</div>
<div>> </div>
<div>> IMAGE TEMPLATE</div>
<div>> DEV_PREFIX="hd"</div>
<div>> FROM_APP="4fc76a938fb81d3517000003"</div>
<div>> FROM_APP_FILE="0"</div>
<div>> FROM_APP_NAME="ttylinux - kvm"</div>
<div>> MD5="04c7d00e88fa66d9aaa34d9cf8ad6aaa"</div>
<div>> </div>
<div>> VIRTUAL MACHINES</div>
<div>> </div>
<div>> ID USER GROUP NAME STAT UCPU UMEM HOST TIME</div>
<div>> 10 oneadmin oneadmin tt fail 0 0K 0d 00h00</div>
<div>> </div>
<div>> [oneadmin@fc-server 102]$ ls</div>
<div>> 10</div>
<div>> [oneadmin@fc-server 102]$ cd 10</div>
<div>> [oneadmin@fc-server 10]$ ll</div>
<div>> total 41325</div>
<div>> -rw-rw-r-- 1 oneadmin oneadmin 971 Jul 11 14:35 deployment.0</div>
<div>> -rw-r--r-- 1 oneadmin oneadmin 41943040 Jul 11 14:35 disk.0</div>
<div>> -rw-r--r-- 1 oneadmin oneadmin 372736 Jul 11 14:35 disk.1</div>
<div>> lrwxrwxrwx 1 oneadmin oneadmin 37 Jul 11 14:35 disk.1.iso -></div>
<div>> /var/lib/one/datastores/102/10/disk.1</div>
<div>> </div>
<div>> show log :</div>
<div>> </div>
<div>> Fri Jul 11 14:35:26 2014 [DiM][D]: Deploying VM 10</div>
<div>> Fri Jul 11 14:35:26 2014 [ReM][D]: Req:8400 UID:0 VirtualMachineDeploy</div>
<div>> result SUCCESS, 10</div>
<div>> Fri Jul 11 14:35:31 2014 [TM][D]: Message received: TRANSFER SUCCESS</div>
<div>> 10 -</div>
<div>> </div>
<div>> Fri Jul 11 14:35:32 2014 [VMM][D]: Message received: LOG I 10</div>
<div>> ExitCode: 0</div>
<div>> </div>
<div>> Fri Jul 11 14:35:32 2014 [VMM][D]: Message received: LOG I 10</div>
<div>> Successfully execute network driver operation: pre.</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10 Command</div>
<div>> execution fail: cat << EOT | /var/tmp/one/vmm/kvm/deploy</div>
<div>> '/var/lib/one//datastores/102/10/deployment.0' '192.168.0.199' 10</div>
<div>> 192.168.0.199</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10 error:</div>
<div>> Failed to create domain from</div>
<div>> /var/lib/one//datastores/102/10/deployment.0</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10 error:</div>
<div>> internal error process exited while connecting to monitor: qemu-kvm:</div>
<div>> -drive</div>
<div>> file=gluster+tcp://<a href="http://192.168.0.199:24007/fc-img/10/disk.0,if=none,id=drive-ide0-0-0,format=raw,cache=none" target="_blank">192.168.0.199:24007/fc-img/10/disk.0,if=none,id=drive-ide0-0-0,format=raw,cache=none</a>:</div>
<div>> could not open disk image gluster+tcp://<a href="http://192.168.0.199:24007/fc-" target="_blank">192.168.0.199:24007/fc-</a></div>
<div>> img/10/disk.0: No such file or directory</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10</div>
<div>> [2014-07-11 06:35:36.105637] E [afr-common.c:4157:afr_notify]</div>
<div>> 0-fc-img-replicate-0: All subvolumes are down.</div>
<div>> Going offline until atleast one of them comes back up.</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG E 10 Could</div>
<div>> not create domain from /var/lib/one//datastores/102/10/deployment.0</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10</div>
<div>> ExitCode: 255</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10 Failed</div>
<div>> to execute virtualization driver operation: deploy.</div>
<div>> </div>
<div>> [oneadmin@fc-server 102]$ more /var/log/one/10.log</div>
<div>> Fri Jul 11 14:35:26 2014 [DiM][I]: New VM state is ACTIVE.</div>
<div>> Fri Jul 11 14:35:27 2014 [LCM][I]: New VM state is PROLOG.</div>
<div>> Fri Jul 11 14:35:32 2014 [LCM][I]: New VM state is BOOT</div>
<div>> Fri Jul 11 14:35:32 2014 [VMM][I]: Generating deployment file:</div>
<div>> /var/lib/one/vms/10/deployment.0</div>
<div>> Fri Jul 11 14:35:32 2014 [VMM][I]: ExitCode: 0</div>
<div>> Fri Jul 11 14:35:32 2014 [VMM][I]: Successfully execute network driver</div>
<div>> operation: pre.</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: Command execution fail: cat << EOT</div>
<div>> | /var/tmp/one/vmm/kvm/deploy</div>
<div>> '/var/lib/one//datastores/102/10/deployment.0' '192.168.0.199</div>
<div>> ' 10 192.168.0.199</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: error: Failed to create domain from</div>
<div>> /var/lib/one//datastores/102/10/deployment.0</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: error: internal error process</div>
<div>> exited while connecting to monitor: qemu-kvm: -drive</div>
<div>> file=gluster+tcp://<a href="http://192.168.0.199:24007/fc-i" target="_blank">192.168.0.199:24007/fc-i</a></div>
<div>> mg/10/disk.0,if=none,id=drive-ide0-0-0,format=raw,cache=none: could</div>
<div>> not open disk image</div>
<div>> gluster+tcp://<a href="http://192.168.0.199:24007/fc-img/10/disk.0" target="_blank">192.168.0.199:24007/fc-img/10/disk.0</a>: No such file or</div>
<div>> direc</div>
<div>> tory</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: [2014-07-11 06:35:36.105637] E</div>
<div>> [afr-common.c:4157:afr_notify] 0-fc-img-replicate-0: All subvolumes</div>
<div>> are down. Going offline unt</div>
<div>> il atleast one of them comes back up.</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]:</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][E]: Could not create domain from</div>
<div>> /var/lib/one//datastores/102/10/deployment.0</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: ExitCode: 255</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: Failed to execute virtualization</div>
<div>> driver operation: deploy.</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][E]: Error deploying virtual machine:</div>
<div>> Could not create domain from</div>
<div>> /var/lib/one//datastores/102/10/deployment.0</div>
<div>> Fri Jul 11 14:35:35 2014 [DiM][I]: New VM state is FAILED</div>
<div>> </div>
<div>> qemu-kvm: -drive</div>
<div>> file=gluster+tcp://<a href="http://192.168.0.199:24007/fc-img/10/disk.0" target="_blank">192.168.0.199:24007/fc-img/10/disk.0</a></div>
<div>> I think it should be</div>
<div>> gluster+tcp://<a href="http://192.168.0.199:24007/fc-sys/10/disk.0" target="_blank">192.168.0.199:24007/fc-sys/10/disk.0</a></div>
<div>> </div>
<div>> What advice do you have.Thanks.</div>
<div>> </div>
<div> </div>
<div>Hi,</div>
<div> </div>
<div>I've been setting up our Opennebula private cloud over the last few </div>
<div>weeks and I'm also</div>
<div>using Gluster for the storage backend. Not all info to make it work is </div>
<div>available in the</div>
<div>Opennebula docs and I was planning on submitting the necessary info.</div>
<div> </div>
<div>After reading your post I decided to no longer postpone it.</div>
<div> </div>
<div>have a look at this pull request : </div>
<div><a href="https://github.com/vincentvdk/docs/commit/98e70a83b40b58d84e00d3180023671263e2b787" target="_blank">https://github.com/vincentvdk/docs/commit/98e70a83b40b58d84e00d3180023671263e2b787</a></div>
<div> </div>
<div>Adding the 'virt' group should fix your problem.</div>
<div> </div>
<div>Regards,</div>
<div>Vincent</div>
<div> </div>
<div> </div>
<div> </div>
<div>> </div>
<div>> _______________________________________________</div>
<div>> Users mailing list</div>
<div>> <a href="mailto:Users@lists.opennebula.org" target="_blank">Users@lists.opennebula.org</a></div>
<div>> <a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a></div>
</div></div></div></blockquote>
</div><br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><br>
<br></blockquote></div><br></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div>Jaime Melis<br>Project Engineer<br>OpenNebula - Flexible Enterprise Cloud Made Simple<br><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:jmelis@opennebula.org" target="_blank">jmelis@opennebula.org</a></div>
</div>