<div dir="ltr"><div><div><div><div><div>Hi,<br><br></div>I´m really sorry for the noise,<br>but I am able to reproduce the problem now by attaching a non-persistent image to a diskless vm.<br><br></div>Could somebody try to verify this, please?:<br><br></div>Setup:<br>ceph datastore, RBD_FORMAT=2 <br></div></div>opennebula 4.8 (Sorry, I was not able to update yet)<br><div><div><div>I run Ubuntu 12.04, kvm<br></div><div><br>Steps:<br>1) Create a disk on the ceph datastore (I used the marketplace), <br> - non-persistent<br>2) create a template with no disk attached.<br></div><div>3) instantiate a vm from 2)<br></div><div>4) attach disk from 1) -> (On my side, the original disk, not the clone is attached according to "virsh dumpxml")<br></div><div><br>5) write a file to the vm and sync or reboot<br></div><div>6) delete the vm and <br></div><div>7) recreate a new one with the disk from 1)<br></div><div>8) Look for file created in 5)<br></div><div><br></div><div><br></div><div><br></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Nov 12, 2014 at 1:20 PM, Joe Mauer <span dir="ltr"><<a href="mailto:aaskarg@gmail.com" target="_blank">aaskarg@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>Dear list,<br>sometimes it helps to write things down.<br></div>By looking at "libvirt dumxml"<br></div>######## vm 234 ########<br><div> <disk type='network' device='disk'><br> <driver name='qemu' type='raw' cache='writeback'/><br> <auth username='libvirt'><br> <secret type='ceph' uuid='2f146016-197e-4c94-ba2d-6be68d75034a'/><br> </auth><br> <source protocol='rbd' name='one/one-90'/><br> <target dev='vda' bus='virtio'/><br> <alias name='virtio-disk0'/><br> <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/><br> </disk><br><br>################<br><div><br>It seems like the vm has the wrong rbd image attached (one/one-90, not one-90-234-1), or am I wrong here?<br><br></div><div>vm 239 is fine:<br>##########<br> <disk type='network' device='disk'><br> <driver name='qemu' type='raw' cache='writeback'/><br> <auth username='libvirt'><br> <secret type='ceph' uuid='2f146016-197e-4c94-ba2d-6be68d75034a'/><br> </auth><br> <source protocol='rbd' name='one/one-90-239-0'/><br> <target dev='vda' bus='virtio'/><br> <alias name='virtio-disk0'/><br> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/><br> </disk><br><br></div><div><br></div><div>looking at the deployment files, it seems that in the first vm(234), i did attach the disk after creating the vm.<br> <br></div><div><br></div></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Nov 12, 2014 at 12:13 PM, Joe Mauer <span dir="ltr"><<a href="mailto:aaskarg@gmail.com" target="_blank">aaskarg@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Dear List,<br><br></div><div>I´m evaluating a one 4.8 setup, using a ceph 0.72.2 cluster, RBD_FORMAT=2<br><br></div><div>Using the clone option instead of copying images to be able to instantiating vms quickly does make this solution very interesting for us, but I ran into a rather big issue that I´m currently <br></div><div>not able to solve (I´m not a ceph or one expert ;-)).<br></div><div>I could not find anything similar on the net, so it might be a configuration issue, but I currently ran out of ideas to debug it.<br><br></div><div>In both test clouds, I instaniciated a virtual machine (ID 234 below) from a non-persistent image (e.g. Ubuntu 14.04 from the marketplace), did some changes (e.g. changed the hostname, thats why I discoverd the problem in the first place), updates ... software installation...<br></div><div><br></div><div>After a while, when starting a new vm (ID 239) from the (still non-persistent) image I get fsck messages when booting and have packages, and e.g. the hostname from the first vm.<br></div><div><br></div><div>Thank you in advance for taking a look.<br></div><div><br></div><div>############## first vm ##################<br></div><div> onevm show 234<br>VIRTUAL MACHINE 234 INFORMATION<br>ID : 234<br>NAME : jenkinssetup<br>USER : op<br>GROUP : opsadmin<br>STATE : ACTIVE<br>LCM_STATE : RUNNING<br>RESCHED : No<br>HOST : vnode6<br>CLUSTER ID : 102<br>CLUSTER : External<br>START TIME : 11/05 16:13:16<br>END TIME : -<br>DEPLOY ID : one-234<br><br>VM DISKS<br> ID TARGET IMAGE TYPE SAVE SAVE_AS<br> 1 vda Ubuntu-14.04-one-4.8 rbd NO -<br><br>VM NICS<br> ID NETWORK VLAN BRIDGE IP MAC<br> 0 Net_ext yes br0 10.10.10.10 02:00:5e:e6:38:8c<br><br>VIRTUAL MACHINE HISTORY<br>SEQ HOST ACTION DS START TIME PROLOG<br> 0 vnode6 none 103 11/05 16:13:42 6d 19h27m 0h00m01s<br><br>VIRTUAL MACHINE TEMPLATE<br>AUTOMATIC_REQUIREMENTS="CLUSTER_ID = 102 & !(PUBLIC_CLOUD = YES)"<br>CONTEXT=[<br> DISK_ID="0",<br> ETH0_DNS="10.10.10.1",<br> ETH0_GATEWAY="10.10.10.1",<br> ETH0_IP="10.10.10.10",<br> ETH0_MAC="02:00:5e:e6:38:8c",<br> ETH0_MASK="255.255.255.0",<br> ETH0_NETWORK="10.10.10.0",<br> HOSTNAME="jenkinssetup",<br> NETWORK="YES",<br> TARGET="hda" ]<br>CPU="2"<br>GRAPHICS=[<br> KEYMAP="de",<br> LISTEN="0.0.0.0",<br> PORT="6134",<br> TYPE="VNC" ]<br>MEMORY="2048"<br>TEMPLATE_ID="15"<br>VCPU="1"<br>VMID="234"<br><br><br></div><div>############# second vm #########<br></div><div>onevm show 239<br>VIRTUAL MACHINE 239 INFORMATION<br>ID : 239<br>NAME : Ubuntu 14.04 - KVM - OpenNebula 4.8-239<br>USER : oneadmin<br>GROUP : oneadmin<br>STATE : ACTIVE<br>LCM_STATE : RUNNING<br>RESCHED : No<br>HOST : vnode5<br>CLUSTER ID : 102<br>CLUSTER : External<br>START TIME : 11/12 10:35:27<br>END TIME : -<br>DEPLOY ID : one-239<br><br>VIRTUAL MACHINE MONITORING<br>USED CPU : 0<br>NET_RX : 959K<br>USED MEMORY : 779.5M<br>NET_TX : 55K<br><br>PERMISSIONS<br>OWNER : um-<br>GROUP : ---<br>OTHER : ---<br><br>VM DISKS<br> ID TARGET IMAGE TYPE SAVE SAVE_AS<br> 0 vda Ubuntu-14.04-one-4.8 rbd NO -<br><br>VM NICS<br> ID NETWORK VLAN BRIDGE IP MAC<br> 0 net-ext2 yes br0 10.10.10.105 02:00:5e:e6:35:69<br><br>VIRTUAL MACHINE HISTORY<br>SEQ HOST ACTION DS START TIME PROLOG<br> 0 vnode5 none 103 11/12 10:35:28 0d 01h16m 0h00m04s<br><br>VIRTUAL MACHINE TEMPLATE<br>AUTOMATIC_REQUIREMENTS="CLUSTER_ID = 102 & !(PUBLIC_CLOUD = YES)"<br>CONTEXT=[<br> DISK_ID="1",<br> ETH0_DNS="10.10.10.1",<br> ETH0_GATEWAY="10.10.10.1",<br> ETH0_IP="10.10.10.105",<br> ETH0_MAC="02:00:5e:e6:35:69",<br> ETH0_MASK="255.255.255.0",<br> ETH0_NETWORK="10.10.10.0",<br> NETWORK="YES",<br> TARGET="hda" ]<br>CPU="1"<br>GRAPHICS=[<br> LISTEN="0.0.0.0",<br> PORT="6139",<br> TYPE="vnc" ]<br>MEMORY="768"<br>OS=[<br> ARCH="x86_64" ]<br>TEMPLATE_ID="22"<br>VMID="239"<br><br></div><div><br>################################<div>oneimage show 90<br>IMAGE 90 INFORMATION<br>ID : 90<br>NAME : Ubuntu-14.04-one-4.8<br>USER : oneadmin<br>GROUP : oneadmin<br>DATASTORE : ceph<br>TYPE : OS<br>REGISTER TIME : 10/29 22:13:27<br>PERSISTENT : No<br>SOURCE : one/one-90<br>PATH : <a href="http://marketplace.c12g.com/appliance/53e7c1b28fb81d6a69000003/download/0" target="_blank">http://marketplace.c12g.com/appliance/53e7c1b28fb81d6a69000003/download/0</a><br>SIZE : 10G<br>STATE : used<br>RUNNING_VMS : 2<br><br>PERMISSIONS<br>OWNER : um-<br>GROUP : ---<br>OTHER : ---<br><br>IMAGE TEMPLATE<br>DEV_PREFIX="hd"<br>FROM_APP="53e7c1b28fb81d6a69000003"<br>FROM_APP_FILE="0"<br>FROM_APP_NAME="Ubuntu 14.04 - KVM - OpenNebula 4.8"<br>MD5="bdbd7fdced3ba5be52ba60e801c0b3e0"<br><br>VIRTUAL MACHINES<br><br> ID USER GROUP NAME STAT UCPU UMEM HOST TIME<br> 234 op opsadmin jenkinssetup runn 0 2.1G vnode6 6d 19h47<br> 239 oneadmin oneadmin Ubuntu 14.04 - runn 0 779.5M vnode5 0d 01h25<br><br></div><div>############ datastore ############<br></div><div>oneadmin@oned:~$ onedatastore show 102<br>DATASTORE 102 INFORMATION<br>ID : 102<br>NAME : ceph<br>USER : oneadmin<br>GROUP : opadmins<br>CLUSTER : -<br>TYPE : IMAGE<br>DS_MAD : ceph<br>TM_MAD : ceph<br>BASE PATH : /var/lib/one//datastores/102<br>DISK_TYPE : RBD<br><br>DATASTORE CAPACITY<br>TOTAL: : 798.1G<br>FREE: : 495G<br>USED: : 263.2G<br>LIMIT: : -<br><br>PERMISSIONS<br>OWNER : um-<br>GROUP : u--<br>OTHER : ---<br><br>DATASTORE TEMPLATE<br>BASE_PATH="/var/lib/one//datastores/"<br>BRIDGE_LIST="oned.test.cloud"<br>CEPH_SECRET="xxxxxxxxxxxxxxxxxxxxxxxx"<br>CEPH_USER="libvirt"<br>CLONE_TARGET="SELF"<br>DISK_TYPE="RBD"<br>DS_MAD="ceph"<br>LN_TARGET="NONE"<br>POOL_NAME="one"<br>RBD_FORMAT="2"<br>TM_MAD="ceph"<br>TYPE="IMAGE_DS"<br><br>IMAGES<br>....<br><br>90<br></div><br></div><div>############ ceph #############<br></div><div>rbd ls -l one| egrep 'NAME|one-90'<br>NAME SIZE PARENT FMT PROT LOCK<br>one-90 10240M 2<br>one-90@234-1 10240M 2 yes<br>one-90@239-0 10240M 2 yes<br>one-90-234-1 10240M one/one-90@234-1 2<br>one-90-239-0 10240M one/one-90@239-0 2<br><br></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>