<div dir="ltr">Hi,<div><br></div><div>are you planning on using CephFS? if that's so you might find it easier to set it up like this:</div><div><br></div><div>- /var/lib/one -> not shared, only for frontend</div><div>- /var/lib/one/datastores -> lives in a cephFS share avaialble to the frontend and all the nodes</div><div><br></div><div>would this suit you?</div><div><br></div><div>cheers,<br>Jaime</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Nov 12, 2014 at 6:09 AM, Huynh Dac Nguyen <span dir="ltr"><<a href="mailto:ndhuynh@spsvietnam.vn" target="_blank">ndhuynh@spsvietnam.vn</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="margin-bottom:1px;line-height:normal;font-size:10pt;font-weight:normal;margin-left:4px;font-style:normal;margin-top:4px;font-variant:normal;margin-right:4px;font-family:Dialog">
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Dear Ruben, </font> </p>
<br>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">You mean:</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">- /var/lib/one is mounted from NAS server just for Front-end</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">- The datastore is mounted from Ceph for frond-end and node</font> </p>
<br>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">So </font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Front-end servers must be mounted NAS (/var/lib/one) and Ceph (/var/lib/one/datastores/[number]), 2 mounted points.</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Node servers must be mounted only Ceph (/var/lib/one/datastores/[number]), 1 mounted point</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">and the passwordless file must be updated manually</font> </p>
<br>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Is it correct? </font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">so why don't we use ceph for both of shared location? or just /var/lib/one? I really don't want to manage more device. </font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Could you show me the best solution? I'm confused to continue because of stuck in integrate Ceph and opennebula</font> </p>
<br> <br>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Regards, </font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Ndhuynh</font> </p>
<br> <br>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">----------------------------------------------------------------------</font> </p>
<br>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Message: 1</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Date: Tue, 11 Nov 2014 14:30:01 +0000</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">From: "Ruben S. Montero" <<a href="mailto:rsmontero@opennebula.org" target="_blank">rsmontero@opennebula.org</a>></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">To: Huynh Dac Nguyen <<a href="mailto:ndhuynh@spsvietnam.vn" target="_blank">ndhuynh@spsvietnam.vn</a>>,</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog"><a href="mailto:users@lists.opennebula.org" target="_blank">users@lists.opennebula.org</a></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Subject: Re: [one-users] Do we need to mount /var/lib/one with ceph</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Message-ID:</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog"><<a href="mailto:CAGi56tetpMozcEk%2BdDzLULj7y_YC-PY40av0rbGoPUDB7mcUCg@mail.gmail.com" target="_blank">CAGi56tetpMozcEk+dDzLULj7y_YC-PY40av0rbGoPUDB7mcUCg@mail.gmail.com</a>></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Content-Type: text/plain; charset="utf-8"</font> </p><span class="">
<br>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">The system datastore is accessed from the front-end to generate the context</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">iso. Note that you don't need to be it exported from the front-end, the</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">nodes and the front-end itself can mount it from an different NAS server.</font> </p>
<br>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">The /var/lib/one contents are needed in the front-end but not by the nodes.</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Just the system datastore directory.</font> </p>
<br>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">If the VM tries to access a disk from a device that is not mounted or the</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">NAS server is down you'll be in trouble. However, note that context ISO is</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">only accessed during the boot state and the main disks of the VM are in the</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">ceph pool, so you'll probably be fine (but if not tested it)</font> </p>
<br>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">Cheers</font> </p>
<br> <br>
</span><p style="margin-top:0;margin-bottom:0">
<font face="Dialog">On Mon Nov 10 2014 at 3:38:14 AM Huynh Dac Nguyen <<a href="mailto:ndhuynh@spsvietnam.vn" target="_blank">ndhuynh@spsvietnam.vn</a>></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">wrote:</font> </p>
<br><span class="">
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Dear Ruben,</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Thank you for replying</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> So what happens if the front-end opennebula is down</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> the /var/lib/one isn't mounted,only System Datastore is mounted,</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> then the VM can't work, right? (the vm requires image and additional</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> files)</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Can you explain why we don't need to export the whole /var/lib/one ?</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Regards,</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Ndhuynh</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> >>> "Ruben S. Montero" <<a href="mailto:rsmontero@opennebula.org" target="_blank">rsmontero@opennebula.org</a>> 11/7/2014 4:52 PM</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> >>></font> </p>
</span><p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Hi Ndhuynh</font> </p><div><div class="h5">
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Ceph storage in OpenNebula is handled as follows:</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> 1.- Image Datastores, hold disk images repository as well as images for</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> running VMs in a Ceph volume</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> 2.- System Datastore holds additional VM files, checkpoints, context</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> disks and the like.</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> If you need live-migration the easiest way is to have a shared</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> filesystem for the System Datastore. You don't need to export the whole</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> /var/lib/one, just the datastore directory, though.</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> If you do not need to live-migrate VMs you should be ok with a ssh</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> based system datastore....</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Cheers</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> On Wed Nov 05 2014 at 12:09:47 PM Huynh Dac Nguyen</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> <<a href="mailto:ndhuynh@spsvietnam.vn" target="_blank">ndhuynh@spsvietnam.vn</a>> wrote:</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Hi All,</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> I'm researching opennebula with ceph, i saw that most of guide focus on</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> using ceph as datastore - block device, right?</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Do we need to mount ceph to /var/lib/one as file system to prevent the</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> opennebula frontend down expectedly?</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> My script is:</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> 1) Make a ceph file system : one and mount to /var/lib/one in all</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> opennebula node (front-end and node)</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> 2) Create a ceph block device and add to opennebula as datastore</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Is this a right way?</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Regards,</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Ndhuynh</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> <a href="mailto:ndhuynh@spsvietnam.vn" target="_blank">ndhuynh@spsvietnam.vn</a></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> This e-mail message including any attachments is for the sole use of</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> the intended(s) and may contain privileged or confidential information.</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Any unauthorized review, use, disclosure or distribution is prohibited.</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> If you are not intended recipient, please immediately contact the sender</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> by reply e-mail and delete the original message and destroy all copies</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> thereof.</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> _______________________________________________</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> Users mailing list</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> <a href="mailto:Users@lists.opennebula.org" target="_blank">Users@lists.opennebula.org</a></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> </font><u><i><font face="Dialog" color="#0000ff"><a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a></font></i></u><font face="Dialog"></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> This e-mail message including any attachments is for the sole use of the</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> intended(s) and may contain privileged or confidential information. Any</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> unauthorized review, use, disclosure or distribution is prohibited. If</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> you are not intended recipient, please immediately contact the sender by</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> reply e-mail and delete the original message and destroy all copies</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">> thereof.</font> </p>
<p style="margin-top:0;margin-bottom:0">
<font face="Dialog">></font> </p>
<br>
This e-mail message including any attachments is for the sole use of the
intended(s) and may contain privileged or confidential information. Any
unauthorized review, use, disclosure or distribution is prohibited. If you
are not intended recipient, please immediately contact the sender by reply
e-mail and delete the original message and destroy all copies thereof.<br>
</div></div></div>
<br>_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@lists.opennebula.org">Users@lists.opennebula.org</a><br>
<a href="http://lists.opennebula.org/listinfo.cgi/users-opennebula.org" target="_blank">http://lists.opennebula.org/listinfo.cgi/users-opennebula.org</a><br>
<br></blockquote></div><br></div><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div>Jaime Melis<br>Project Engineer<br>OpenNebula - Flexible Enterprise Cloud Made Simple<br><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:jmelis@opennebula.org" target="_blank">jmelis@opennebula.org</a></div></div></div>