<html><body><div style="font-family: tahoma,new york,times,serif; font-size: 10pt; color: #000000"><div>Which distribution are you using for the storage node that OpenNebula accesses? (this is the node where qemu-img would be run I do believe). This looks like a potential problem with that qemu-img binary.</div><div><br></div><hr id="zwchr"><div style="color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;" data-mce-style="color: #000; font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><b>From: </b>"Jon" <three18ti@gmail.com><br><b>Cc: </b>"Users OpenNebula" <users@lists.opennebula.org><br><b>Sent: </b>Wednesday, July 10, 2013 8:03:40 PM<br><b>Subject: </b>Re: [one-users] How to use Ceph/RBD for System Datastore<br><div><br></div>Hey Bill,<div><br></div><div>So let me ask you this. How do you create rbd images? I'm attempting to create a blank datastore, but qemu-img convert keeps segfaulting.</div><div>Any suggestions are appreciated.</div><div><div><br></div><div>root@red6:~# qemu-img convert -O rbd /var/tmp/61e14679af7dd1e0e1e09e230c89f82a rbd:one/one-17</div><div>Segmentation fault (core dumped)</div></div><div><br></div><div>The oned log indicates the same:</div><div><br></div><div><div>Wed Jul 10 17:52:30 2013 [ImM][I]: Creating disk at of 5120Mb (type: ext4)</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: Command execution fail: /var/lib/one/remotes/datastore/ceph/mkfs </div><div>Wed Jul 10 17:52:31 2013 [ImM][E]: mkfs: Command " set -e</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: </div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: # create and format</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: dd if=/dev/zero of=/var/tmp/61e14679af7dd1e0e1e09e230c89f82a bs=1 count=1 seek=5120M</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: mkfs -t ext4 -F /var/tmp/61e14679af7dd1e0e1e09e230c89f82a</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: </div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: # create rbd</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: qemu-img convert -O rbd /var/tmp/61e14679af7dd1e0e1e09e230c89f82a rbd:one/one-17</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: </div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: # remove original</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: rm -f /var/tmp/61e14679af7dd1e0e1e09e230c89f82a" failed: 1+0 records in</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: 1+0 records out</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: 1 byte (1 B) copied, 0.000232576 s, 4.3 kB/s</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: mke2fs 1.42.5 (29-Jul-2012)</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: Segmentation fault (core dumped)</div><div>Wed Jul 10 17:52:31 2013 [ImM][E]: Error registering one/one-17 in localhost</div><div>Wed Jul 10 17:52:31 2013 [ImM][I]: ExitCode: 139</div><div>Wed Jul 10 17:52:31 2013 [ImM][E]: Error creating datablock: Error registering one/one-17 in localhost</div><div><br></div><div>My image definition looks like this:</div><div><br></div><div><div>IMAGE 17 INFORMATION </div><div>ID : 17 </div><div>NAME : ubuntu-server-13.04-x86_64</div><div>USER : oneadmin </div><div>GROUP : oneadmin </div><div>DATASTORE : rbd1 </div><div>TYPE : DATABLOCK </div><div>REGISTER TIME : 07/10 17:52:30 </div><div>PERSISTENT : Yes </div><div>SOURCE : </div><div>FSTYPE : ext4 </div><div>SIZE : 5G </div><div>STATE : err </div><div>RUNNING_VMS : 0 </div><div><br></div><div>PERMISSIONS </div><div>OWNER : um- </div><div>GROUP : --- </div><div>OTHER : --- </div><div><br></div><div>IMAGE TEMPLATE </div><div>DESCRIPTION="ubuntu-server-13.04-x86_64"</div><div>DEV_PREFIX="hd"</div><div>ERROR="Wed Jul 10 17:52:31 2013 : Error creating datablock: Error registering one/one-17 in localhost"</div></div><div><br></div><div>Thanks,</div><div>Jon A</div><div><br></div><br><div class="gmail_quote">On Wed, Jul 10, 2013 at 5:14 PM, Jon <span dir="ltr"><<a href="mailto:three18ti@gmail.com" target="_blank" data-mce-href="mailto:three18ti@gmail.com">three18ti@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" data-mce-style="margin: 0 0 0 .8ex; border-left: 1px #ccc solid; padding-left: 1ex;">Hey Bill,<div><br></div><div>Thanks for this. This works perfectly!</div><div><br></div><div>Thanks,</div><div>Jon A<div><div class="h5"><br><div><br></div><div class="gmail_quote">On Wed, Jul 10, 2013 at 6:44 AM, Campbell, Bill <span dir="ltr"><<a href="mailto:bcampbell@axcess-financial.com" target="_blank" data-mce-href="mailto:bcampbell@axcess-financial.com">bcampbell@axcess-financial.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" data-mce-style="margin: 0 0 0 .8ex; border-left: 1px #ccc solid; padding-left: 1ex;"><div lang="EN-US"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">Not entirely. You shouldn’t need to manually create/mount an RBD for the system datastore. Since the system datastore holds the running VM deployment files (and not necessarily an RBD image, just a reference to it in the deployment file) then this directory does not necessarily need to be shared.<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">Here’s what we do:<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: Symbol; color: #1f497d;"><span>·<span style="font:7.0pt "Times New Roman"" data-mce-style="font: 7.0pt 'Times New Roman';"> </span></span></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">OpenNebula system configured with no special exports/shares.<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: Symbol; color: #1f497d;"><span>·<span style="font:7.0pt "Times New Roman"" data-mce-style="font: 7.0pt 'Times New Roman';"> </span></span></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">The System datastore is modified to use the SSH transfer manager<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span style="font-size:11.0pt;font-family:Symbol;color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: Symbol; color: #1f497d;"><span>·<span style="font:7.0pt "Times New Roman"" data-mce-style="font: 7.0pt 'Times New Roman';"> </span></span></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">We modify the SSH transfer manager pre/post migrate scripts (by default located in /var/lib/one/remotes/tm/ssh/) to copy files from the source host to the destination host prior to migration/delete files on source after successful migration.<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">Don’t worry about mapping/unmapping RBD volumes. When creating/importing images into the Ceph datastore the RBDs should be created at this point. So long as the Hypervisor nodes can see/interact with the Ceph cluster, when you deploy the VM it will use the RBD in the cluster for storage (no files copied/mapped locally, all handled by QEMU).<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">Here is the pre-migrate script we use (very simple):<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">#!/bin/bash<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">SRC=$1<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">DST=$2<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">REMDIR=$3<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">VMID=$4<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">DSID=$5<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">TEMPLATE=$6<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">ssh $DST mkdir -p /var/lib/one/datastores/0/$VMID<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">ssh $SRC scp /var/lib/one/datastores/0/$VMID/* $DST:/var/lib/one/datastores/0/$VMID/<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">exit 0<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">And the post-migrate script:<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">#!/bin/bash<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">SRC=$1<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">DST=$2<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">REMDIR=$3<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">VMID=$4<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">DSID=$5<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">TEMPLATE=$6<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">ssh $SRC rm -rf /var/lib/one/datastores/0/$VMID<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><i><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">exit 0<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></i></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;">Hope this helps!<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif'; color: #1f497d;"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif';">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"" data-mce-style="font-size: 11.0pt; font-family: 'Calibri','sans-serif';"> Jon [mailto:<a href="mailto:three18ti@gmail.com" target="_blank" data-mce-href="mailto:three18ti@gmail.com">three18ti@gmail.com</a>] <br> <b>Sent:</b> Wednesday, July 10, 2013 12:01 AM<br><b>To:</b> Campbell, Bill<br><b>Cc:</b> Users OpenNebula<br><b>Subject:</b> Re: [one-users] How to use Ceph/RBD for System Datastore<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><div><div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p><p class="MsoNormal">Hey Bill,<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p><div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">Thanks for getting back to me. <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">If I'm understanding you correctly, you're basically using the ssh transfer manager to perform live migrations?<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">Do you then create/mount one rbd per host?<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">E.g., <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">host1:<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">mount /dev/rbd/rbd/host1-one-system /var/lib/one/datastores/0<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">host2:<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">mount /dev/rbd/rbd/host2-one-system /var/lib/one/datastores/0<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">then use the modified ssh drivers to perform the migrations?<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">I would definitely be interested in learning how you accomplished that.<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">My other thought was to use CephFS for shared storage. This would eliminate the need for a NFS/GlusterFS/CLVM, which is an extra layer of complexity I would like to avoid. As I understand it though, CephFS isn't "ready for prime-time" which gives me pause... <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal">Thanks again,<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><div><p class="MsoNormal" style="margin-bottom:12.0pt" data-mce-style="margin-bottom: 12.0pt;">Jon A<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p><div><p class="MsoNormal">On Tue, Jul 9, 2013 at 7:55 PM, Campbell, Bill <<a href="mailto:bcampbell@axcess-financial.com" target="_blank" data-mce-href="mailto:bcampbell@axcess-financial.com">bcampbell@axcess-financial.com</a>> wrote:<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in" data-mce-style="border: none; border-left: solid #cccccc 1.0pt; padding: 0in 0in 0in 6.0pt; margin-left: 4.8pt; margin-right: 0in;"><div><div><div><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif';">Jon,<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif';">I think I understand what you are trying to do, but I think it doesn't quite work that way. Let me try to explain (and please let me know if I don't explain it well enough ;-))<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif';">I don't think that you can use Ceph directly as a system datastore. The way the Ceph datastore driver works for migrations is leveraging whatever transfer method you have for the system datastore to perform the migration. For example, if you use the 'shared' system datastore, then it will use that transfer manager's pre and post migration drivers. For 'ssh', the ssh drivers, and so on. The way the Ceph datastore is implemented is as Ceph Block Devices, so unfortunately there is not a way to use it as a simple shared volume.<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif';">There are 2 potential solutions for getting live migrations working for your Ceph datastore VMs:<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><ul type="disc"><li class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif';">Create a shared NFS volume (or other 'sharable' filesystem, like GFS2, OCFS2, etc., however these are much more complicated to configure and usually not worth the hassle) and have the shared volume mounted to the same location on each hypervisor node. In a previous test deployment, we just exported out the /var/lib/one/vms directory to the hypervisors. At this point, all of the hypervisors should be able to see the deployment files in the same location and you should be able to perform a migration.<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></li><li class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif';">Use SSH as the transfer manager for your system datastore, and modify the pre and post-migrate scripts to copy the deployment files from the current VM host to the target VM host. This is the method we use currently in our deployment, as it is one less configuration step that we have to worry about maintaining on each node, and makes expanding our cluster much quicker and easier. I can share with you the pre and post-migrate scripts we use if you like.<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></li></ul><div><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif';">Let me know if the above makes sense, and of course if you need any additional help please don't hesitate to bug me. I'm very familiar with the Ceph drivers ;-)<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div></div><div><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div class="MsoNormal" align="center" style="text-align:center" data-mce-style="text-align: center;"><hr size="2" width="100%" align="center"></div><div><p class="MsoNormal"><b><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">From: </span></b><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">"Jon" <<a href="mailto:three18ti@gmail.com" target="_blank" data-mce-href="mailto:three18ti@gmail.com">three18ti@gmail.com</a>><br> <b>To: </b>"Users OpenNebula" <<a href="mailto:users@lists.opennebula.org" target="_blank" data-mce-href="mailto:users@lists.opennebula.org">users@lists.opennebula.org</a>><br><b>Sent: </b>Tuesday, July 9, 2013 8:05:51 PM<br><b>Subject: </b>[one-users] How to use Ceph/RBD for System Datastore<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">Hello All,<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">I am using Ceph as my storage back end and would like to know how to configure the system datastore, such that I can live migrate vms.<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">Following the directions, I thought I could create a datastore, format it, and mount it at /var/lib/one/datastores/0 , however, I discovered, that isn't quite how things work.<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> <a href="http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-May/001913.html" target="_blank" data-mce-href="http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-May/001913.html">http://lists.ceph.com/pipermail/ceph-users-ceph.com/2013-May/001913.html</a><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">You can read more about that at the above link, but long story short, to mount a shared filesystem it has to be a "clustered" filesystem (I think CephFS is the "clustered filesystem", in this case).<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">I attempted to modify my system datastore config, however, I was unable to change the DS_MAD parameter, and vm creation errors out telling me there's no /var/lib/one/remotes/tm/ceph/mkswap driver (there isn't)<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> oneadmin@red6:~$ onedatastore show 0 <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> DATASTORE 0 INFORMATION <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> ID : 0 <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> NAME : system <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> USER : oneadmin <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> GROUP : oneadmin <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> CLUSTER : - <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> TYPE : SYSTEM <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> DS_MAD : - <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> TM_MAD : ceph <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> BASE PATH : /var/lib/one/datastores/0<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> DISK_TYPE : FILE <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> PERMISSIONS <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> OWNER : um- <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> GROUP : u-- <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> OTHER : --- <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> DATASTORE TEMPLATE <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> DISK_TYPE="rbd"<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> DS_MAD="-"<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> TM_MAD="ceph"<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> TYPE="SYSTEM_DS"<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">>> IMAGES <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">Maybe I'm just confused. Can anyone provide some guidance on setting ceph up as the system datastore?<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">Thanks,<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">Jon A<span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div></div><p class="MsoNormal"><span style="font-family:"Helvetica","sans-serif"" data-mce-style="font-family: 'Helvetica','sans-serif';">_______________________________________________<br>Users mailing list<br><a href="mailto:Users@lists.opennebula.org" target="_blank" data-mce-href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a><br> <a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank" data-mce-href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div><div><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"" data-mce-style="font-size: 10.0pt; font-family: 'Tahoma','sans-serif';"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></span></p></div></div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p><div><p class="MsoNormal"><b>NOTICE: Protect the information in this message in accordance with the company's security policies. If you received this message in error, immediately notify the sender and destroy all copies.</b><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div></blockquote></div><p class="MsoNormal"><span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span> <span data-mce-style="text-decoration: underline;" style="text-decoration: underline;"></span></p></div></div></div></div><div><div><br><div><b>NOTICE: Protect the information in this message in accordance with the company's security policies. If you received this message in error, immediately notify the sender and destroy all copies.</b></div><br></div></div></div></blockquote></div><br></div></div></div></blockquote></div><br></div><br>_______________________________________________<br>Users mailing list<br>Users@lists.opennebula.org<br>http://lists.opennebula.org/listinfo.cgi/users-opennebula.org<br></div><div><br></div></div>
<br><html><body><b>NOTICE: Protect the information in this message in accordance with the company's security policies. If you received this message in error, immediately notify the sender and destroy all copies.</b></body></html>
<br></body></html>