<html><head><meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"><style>body { line-height: 1.5; }blockquote { margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em; }body { font-size: 10.5pt; font-family: Arial; color: rgb(0, 0, 0); line-height: 1.5; }</style></head><body>
<div><span></span><span style="color: rgb(51, 51, 51); font-family: arial; line-height: 24px; background-color: rgb(251, 252, 254);">Thank you for your reply.</span></div>
<div><br></div>Regards.<br>
<blockquote style="margin-top: 0px; margin-bottom: 0px; margin-left: 0.5em;"><div> </div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"><div style="PADDING-RIGHT: 8px; PADDING-LEFT: 8px; FONT-SIZE: 12px;FONT-FAMILY:tahoma;COLOR:#000000; BACKGROUND: #efefef; PADDING-BOTTOM: 8px; PADDING-TOP: 8px"><div><b>From:</b> <a href="mailto:vincent@vanderkussen.org">vincent</a></div><div><b>Date:</b> 2014-07-12 18:30</div><div><b>To:</b> <a href="mailto:clm_tuan@hotmail.com">clm_tuan</a></div><div><b>CC:</b> <a href="mailto:users@lists.opennebula.org">users</a></div><div><b>Subject:</b> Re: [one-users] deploy Non-persistent vm on 4.6.2 base glusterfs no running</div></div></div><div><div>On 2014-07-11 10:17, clm_tuan@hotmail.com wrote:</div>
<div>> Dear All,</div>
<div>> </div>
<div>> I tried to deploy opennebula 4.6.2 base glusterfs 3.5.1</div>
<div>> </div>
<div>> all three server:</div>
<div>> 1. one server,ip is 192.168.0.198;</div>
<div>> 2. node0,ip is 192.168.0.199;</div>
<div>> 3. node1,ip is 192.168.0.200;</div>
<div>> </div>
<div>> Datastores configuration:</div>
<div>> </div>
<div>> [oneadmin@fc-server ]$ onedatastore show 102</div>
<div>> DATASTORE 102 INFORMATION</div>
<div>> ID : 102</div>
<div>> NAME : sys</div>
<div>> USER : oneadmin</div>
<div>> GROUP : oneadmin</div>
<div>> CLUSTER : c1</div>
<div>> TYPE : SYSTEM</div>
<div>> DS_MAD : -</div>
<div>> TM_MAD : shared</div>
<div>> BASE PATH : /var/lib/one//datastores/102</div>
<div>> DISK_TYPE : FILE</div>
<div>> </div>
<div>> DATASTORE CAPACITY</div>
<div>> TOTAL: : 131.3G</div>
<div>> FREE: : 123.3G</div>
<div>> USED: : 41M</div>
<div>> LIMIT: : -</div>
<div>> </div>
<div>> PERMISSIONS</div>
<div>> OWNER : um-</div>
<div>> GROUP : u--</div>
<div>> OTHER : ---</div>
<div>> </div>
<div>> DATASTORE TEMPLATE</div>
<div>> BASE_PATH="/var/lib/one//datastores/"</div>
<div>> GLUSTER_HOST="192.168.0.199:24007"</div>
<div>> GLUSTER_VOLUME="fc-sys"</div>
<div>> SHARED="YES"</div>
<div>> TM_MAD="shared"</div>
<div>> TYPE="SYSTEM_DS"</div>
<div>> </div>
<div>> [oneadmin@fc-server ]$ onedatastore show 103</div>
<div>> DATASTORE 103 INFORMATION</div>
<div>> ID : 103</div>
<div>> NAME : img</div>
<div>> USER : oneadmin</div>
<div>> GROUP : oneadmin</div>
<div>> CLUSTER : c1</div>
<div>> TYPE : IMAGE</div>
<div>> DS_MAD : fs</div>
<div>> TM_MAD : shared</div>
<div>> BASE PATH : /var/lib/one//datastores/103</div>
<div>> DISK_TYPE :</div>
<div>> </div>
<div>> DATASTORE CAPACITY</div>
<div>> TOTAL: : 131.3G</div>
<div>> FREE: : 123.3G</div>
<div>> USED: : 41M</div>
<div>> LIMIT: : -</div>
<div>> </div>
<div>> PERMISSIONS</div>
<div>> OWNER : um-</div>
<div>> GROUP : u--</div>
<div>> OTHER : ---</div>
<div>> </div>
<div>> DATASTORE TEMPLATE</div>
<div>> BASE_PATH="/var/lib/one//datastores/"</div>
<div>> CLONE_TARGET="SYSTEM"</div>
<div>> DISK_TYPE="GLUSTER"</div>
<div>> DS_MAD="fs"</div>
<div>> GLUSTER_HOST="192.168.0.199:24007"</div>
<div>> GLUSTER_VOLUME="fc-img"</div>
<div>> LN_TARGET="NONE"</div>
<div>> TM_MAD="shared"</div>
<div>> TYPE="IMAGE_DS"</div>
<div>> </div>
<div>> 192.168.0.199:/fc-sys on /var/lib/one/datastores/102 type</div>
<div>> fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)</div>
<div>> 192.168.0.199:/fc-img on /var/lib/one/datastores/103 type</div>
<div>> fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072)</div>
<div>> </div>
<div>> deploy image to PERSISTENT, it's ok.</div>
<div>> </div>
<div>> [oneadmin@fc-server ]$ oneimage show 1</div>
<div>> IMAGE 1 INFORMATION</div>
<div>> ID : 1</div>
<div>> NAME : ttylinux</div>
<div>> USER : oneadmin</div>
<div>> GROUP : oneadmin</div>
<div>> DATASTORE : img</div>
<div>> TYPE : OS</div>
<div>> REGISTER TIME : 07/11 21:35:11</div>
<div>> PERSISTENT : Yes</div>
<div>> SOURCE : /var/lib/one//datastores/103/4c40e4068ad2ff41c8767c40f85ae786</div>
<div>> </div>
<div>> PATH :</div>
<div>> http://marketplace.c12g.com/appliance/4fc76a938fb81d3517000003/download/0</div>
<div>> </div>
<div>> SIZE : 40M</div>
<div>> STATE : used</div>
<div>> RUNNING_VMS : 1</div>
<div>> </div>
<div>> PERMISSIONS</div>
<div>> OWNER : um-</div>
<div>> GROUP : ---</div>
<div>> OTHER : ---</div>
<div>> </div>
<div>> IMAGE TEMPLATE</div>
<div>> DEV_PREFIX="hd"</div>
<div>> FROM_APP="4fc76a938fb81d3517000003"</div>
<div>> FROM_APP_FILE="0"</div>
<div>> FROM_APP_NAME="ttylinux - kvm"</div>
<div>> MD5="04c7d00e88fa66d9aaa34d9cf8ad6aaa"</div>
<div>> </div>
<div>> VIRTUAL MACHINES</div>
<div>> </div>
<div>> ID USER GROUP NAME STAT UCPU UMEM HOST TIME</div>
<div>> 8 oneadmin oneadmin tt runn 0 512M 192.168.0. -1d 16h32</div>
<div>> [oneadmin@fc-server tm]$</div>
<div>> </div>
<div>> [oneadmin@fc-server 8]$ ll</div>
<div>> total 366</div>
<div>> -rw-rw-r-- 1 oneadmin oneadmin 992 Jul 11 13:44 deployment.0</div>
<div>> lrwxrwxrwx 1 oneadmin oneadmin 60 Jul 11 13:44 disk.0 -></div>
<div>> /var/lib/one/datastores/103/4c40e4068ad2ff41c8767c40f85ae786</div>
<div>> -rw-r--r-- 1 oneadmin oneadmin 372736 Jul 11 13:44 disk.1</div>
<div>> lrwxrwxrwx 1 oneadmin oneadmin 36 Jul 11 13:44 disk.1.iso -></div>
<div>> /var/lib/one/datastores/102/8/disk.1</div>
<div>> </div>
<div>> but deploy image to Non-persistent, it's no running.</div>
<div>> </div>
<div>> [oneadmin@fc-server 102]$ oneimage show 1</div>
<div>> IMAGE 1 INFORMATION</div>
<div>> ID : 1</div>
<div>> NAME : ttylinux</div>
<div>> USER : oneadmin</div>
<div>> GROUP : oneadmin</div>
<div>> DATASTORE : img</div>
<div>> TYPE : OS</div>
<div>> REGISTER TIME : 07/11 21:35:11</div>
<div>> PERSISTENT : No</div>
<div>> SOURCE : /var/lib/one//datastores/103/4c40e4068ad2ff41c8767c40f85ae786</div>
<div>> </div>
<div>> PATH :</div>
<div>> http://marketplace.c12g.com/appliance/4fc76a938fb81d3517000003/download/0</div>
<div>> </div>
<div>> SIZE : 40M</div>
<div>> STATE : used</div>
<div>> RUNNING_VMS : 1</div>
<div>> </div>
<div>> PERMISSIONS</div>
<div>> OWNER : um-</div>
<div>> GROUP : ---</div>
<div>> OTHER : ---</div>
<div>> </div>
<div>> IMAGE TEMPLATE</div>
<div>> DEV_PREFIX="hd"</div>
<div>> FROM_APP="4fc76a938fb81d3517000003"</div>
<div>> FROM_APP_FILE="0"</div>
<div>> FROM_APP_NAME="ttylinux - kvm"</div>
<div>> MD5="04c7d00e88fa66d9aaa34d9cf8ad6aaa"</div>
<div>> </div>
<div>> VIRTUAL MACHINES</div>
<div>> </div>
<div>> ID USER GROUP NAME STAT UCPU UMEM HOST TIME</div>
<div>> 10 oneadmin oneadmin tt fail 0 0K 0d 00h00</div>
<div>> </div>
<div>> [oneadmin@fc-server 102]$ ls</div>
<div>> 10</div>
<div>> [oneadmin@fc-server 102]$ cd 10</div>
<div>> [oneadmin@fc-server 10]$ ll</div>
<div>> total 41325</div>
<div>> -rw-rw-r-- 1 oneadmin oneadmin 971 Jul 11 14:35 deployment.0</div>
<div>> -rw-r--r-- 1 oneadmin oneadmin 41943040 Jul 11 14:35 disk.0</div>
<div>> -rw-r--r-- 1 oneadmin oneadmin 372736 Jul 11 14:35 disk.1</div>
<div>> lrwxrwxrwx 1 oneadmin oneadmin 37 Jul 11 14:35 disk.1.iso -></div>
<div>> /var/lib/one/datastores/102/10/disk.1</div>
<div>> </div>
<div>> show log :</div>
<div>> </div>
<div>> Fri Jul 11 14:35:26 2014 [DiM][D]: Deploying VM 10</div>
<div>> Fri Jul 11 14:35:26 2014 [ReM][D]: Req:8400 UID:0 VirtualMachineDeploy</div>
<div>> result SUCCESS, 10</div>
<div>> Fri Jul 11 14:35:31 2014 [TM][D]: Message received: TRANSFER SUCCESS</div>
<div>> 10 -</div>
<div>> </div>
<div>> Fri Jul 11 14:35:32 2014 [VMM][D]: Message received: LOG I 10</div>
<div>> ExitCode: 0</div>
<div>> </div>
<div>> Fri Jul 11 14:35:32 2014 [VMM][D]: Message received: LOG I 10</div>
<div>> Successfully execute network driver operation: pre.</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10 Command</div>
<div>> execution fail: cat << EOT | /var/tmp/one/vmm/kvm/deploy</div>
<div>> '/var/lib/one//datastores/102/10/deployment.0' '192.168.0.199' 10</div>
<div>> 192.168.0.199</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10 error:</div>
<div>> Failed to create domain from</div>
<div>> /var/lib/one//datastores/102/10/deployment.0</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10 error:</div>
<div>> internal error process exited while connecting to monitor: qemu-kvm:</div>
<div>> -drive</div>
<div>> file=gluster+tcp://192.168.0.199:24007/fc-img/10/disk.0,if=none,id=drive-ide0-0-0,format=raw,cache=none:</div>
<div>> could not open disk image gluster+tcp://192.168.0.199:24007/fc-</div>
<div>> img/10/disk.0: No such file or directory</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10</div>
<div>> [2014-07-11 06:35:36.105637] E [afr-common.c:4157:afr_notify]</div>
<div>> 0-fc-img-replicate-0: All subvolumes are down.</div>
<div>> Going offline until atleast one of them comes back up.</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG E 10 Could</div>
<div>> not create domain from /var/lib/one//datastores/102/10/deployment.0</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10</div>
<div>> ExitCode: 255</div>
<div>> </div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][D]: Message received: LOG I 10 Failed</div>
<div>> to execute virtualization driver operation: deploy.</div>
<div>> </div>
<div>> [oneadmin@fc-server 102]$ more /var/log/one/10.log</div>
<div>> Fri Jul 11 14:35:26 2014 [DiM][I]: New VM state is ACTIVE.</div>
<div>> Fri Jul 11 14:35:27 2014 [LCM][I]: New VM state is PROLOG.</div>
<div>> Fri Jul 11 14:35:32 2014 [LCM][I]: New VM state is BOOT</div>
<div>> Fri Jul 11 14:35:32 2014 [VMM][I]: Generating deployment file:</div>
<div>> /var/lib/one/vms/10/deployment.0</div>
<div>> Fri Jul 11 14:35:32 2014 [VMM][I]: ExitCode: 0</div>
<div>> Fri Jul 11 14:35:32 2014 [VMM][I]: Successfully execute network driver</div>
<div>> operation: pre.</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: Command execution fail: cat << EOT</div>
<div>> | /var/tmp/one/vmm/kvm/deploy</div>
<div>> '/var/lib/one//datastores/102/10/deployment.0' '192.168.0.199</div>
<div>> ' 10 192.168.0.199</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: error: Failed to create domain from</div>
<div>> /var/lib/one//datastores/102/10/deployment.0</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: error: internal error process</div>
<div>> exited while connecting to monitor: qemu-kvm: -drive</div>
<div>> file=gluster+tcp://192.168.0.199:24007/fc-i</div>
<div>> mg/10/disk.0,if=none,id=drive-ide0-0-0,format=raw,cache=none: could</div>
<div>> not open disk image</div>
<div>> gluster+tcp://192.168.0.199:24007/fc-img/10/disk.0: No such file or</div>
<div>> direc</div>
<div>> tory</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: [2014-07-11 06:35:36.105637] E</div>
<div>> [afr-common.c:4157:afr_notify] 0-fc-img-replicate-0: All subvolumes</div>
<div>> are down. Going offline unt</div>
<div>> il atleast one of them comes back up.</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]:</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][E]: Could not create domain from</div>
<div>> /var/lib/one//datastores/102/10/deployment.0</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: ExitCode: 255</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][I]: Failed to execute virtualization</div>
<div>> driver operation: deploy.</div>
<div>> Fri Jul 11 14:35:33 2014 [VMM][E]: Error deploying virtual machine:</div>
<div>> Could not create domain from</div>
<div>> /var/lib/one//datastores/102/10/deployment.0</div>
<div>> Fri Jul 11 14:35:35 2014 [DiM][I]: New VM state is FAILED</div>
<div>> </div>
<div>> qemu-kvm: -drive</div>
<div>> file=gluster+tcp://192.168.0.199:24007/fc-img/10/disk.0</div>
<div>> I think it should be</div>
<div>> gluster+tcp://192.168.0.199:24007/fc-sys/10/disk.0</div>
<div>> </div>
<div>> What advice do you have.Thanks.</div>
<div>> </div>
<div> </div>
<div>Hi,</div>
<div> </div>
<div>I've been setting up our Opennebula private cloud over the last few </div>
<div>weeks and I'm also</div>
<div>using Gluster for the storage backend. Not all info to make it work is </div>
<div>available in the</div>
<div>Opennebula docs and I was planning on submitting the necessary info.</div>
<div> </div>
<div>After reading your post I decided to no longer postpone it.</div>
<div> </div>
<div>have a look at this pull request : </div>
<div>https://github.com/vincentvdk/docs/commit/98e70a83b40b58d84e00d3180023671263e2b787</div>
<div> </div>
<div>Adding the 'virt' group should fix your problem.</div>
<div> </div>
<div>Regards,</div>
<div>Vincent</div>
<div> </div>
<div> </div>
<div> </div>
<div>> </div>
<div>> _______________________________________________</div>
<div>> Users mailing list</div>
<div>> Users@lists.opennebula.org</div>
<div>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</div>
</div></blockquote>
</body></html>