[one-users] Ceph image format 2 issue for nonpersistent disks
Ruben S. Montero
rsmontero at opennebula.org
Wed Nov 12 09:11:57 PST 2014
Hi Joe
I can confirm this bug :( The problem is in the kvm/attach-disk plugin
script. It needs (as the core does) to look to the CLONE attribute for the
disk and use the snapshot instead of the base disk.
There should be no problem when attaching persistent disks.
Thank you very much for your detailed feedback!!
Ruben
for your reference: http://dev.opennebula.org/issues/3332
On Wed Nov 12 2014 at 4:27:46 PM Joe Mauer <aaskarg at gmail.com> wrote:
> Hi,
>
> I´m really sorry for the noise,
> but I am able to reproduce the problem now by attaching a non-persistent
> image to a diskless vm.
>
> Could somebody try to verify this, please?:
>
> Setup:
> ceph datastore, RBD_FORMAT=2
> opennebula 4.8 (Sorry, I was not able to update yet)
> I run Ubuntu 12.04, kvm
>
> Steps:
> 1) Create a disk on the ceph datastore (I used the marketplace),
> - non-persistent
> 2) create a template with no disk attached.
> 3) instantiate a vm from 2)
> 4) attach disk from 1) -> (On my side, the original disk, not the clone
> is attached according to "virsh dumpxml")
>
> 5) write a file to the vm and sync or reboot
> 6) delete the vm and
> 7) recreate a new one with the disk from 1)
> 8) Look for file created in 5)
>
>
>
>
> On Wed, Nov 12, 2014 at 1:20 PM, Joe Mauer <aaskarg at gmail.com> wrote:
>
>> Dear list,
>> sometimes it helps to write things down.
>> By looking at "libvirt dumxml"
>> ######## vm 234 ########
>> <disk type='network' device='disk'>
>> <driver name='qemu' type='raw' cache='writeback'/>
>> <auth username='libvirt'>
>> <secret type='ceph' uuid='2f146016-197e-4c94-ba2d-6be68d75034a'/>
>> </auth>
>> <source protocol='rbd' name='one/one-90'/>
>> <target dev='vda' bus='virtio'/>
>> <alias name='virtio-disk0'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
>> function='0x0'/>
>> </disk>
>>
>> ################
>>
>> It seems like the vm has the wrong rbd image attached (one/one-90, not
>> one-90-234-1), or am I wrong here?
>>
>> vm 239 is fine:
>> ##########
>> <disk type='network' device='disk'>
>> <driver name='qemu' type='raw' cache='writeback'/>
>> <auth username='libvirt'>
>> <secret type='ceph' uuid='2f146016-197e-4c94-ba2d-6be68d75034a'/>
>> </auth>
>> <source protocol='rbd' name='one/one-90-239-0'/>
>> <target dev='vda' bus='virtio'/>
>> <alias name='virtio-disk0'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
>> function='0x0'/>
>> </disk>
>>
>>
>> looking at the deployment files, it seems that in the first vm(234), i
>> did attach the disk after creating the vm.
>>
>>
>>
>> On Wed, Nov 12, 2014 at 12:13 PM, Joe Mauer <aaskarg at gmail.com> wrote:
>>
>>> Dear List,
>>>
>>> I´m evaluating a one 4.8 setup, using a ceph 0.72.2 cluster, RBD_FORMAT=2
>>>
>>> Using the clone option instead of copying images to be able to
>>> instantiating vms quickly does make this solution very interesting for us,
>>> but I ran into a rather big issue that I´m currently
>>> not able to solve (I´m not a ceph or one expert ;-)).
>>> I could not find anything similar on the net, so it might be a
>>> configuration issue, but I currently ran out of ideas to debug it.
>>>
>>> In both test clouds, I instaniciated a virtual machine (ID 234 below)
>>> from a non-persistent image (e.g. Ubuntu 14.04 from the marketplace), did
>>> some changes (e.g. changed the hostname, thats why I discoverd the problem
>>> in the first place), updates ... software installation...
>>>
>>> After a while, when starting a new vm (ID 239) from the (still
>>> non-persistent) image I get fsck messages when booting and have packages,
>>> and e.g. the hostname from the first vm.
>>>
>>> Thank you in advance for taking a look.
>>>
>>> ############## first vm ##################
>>> onevm show 234
>>> VIRTUAL MACHINE 234 INFORMATION
>>> ID : 234
>>> NAME : jenkinssetup
>>> USER : op
>>> GROUP : opsadmin
>>> STATE : ACTIVE
>>> LCM_STATE : RUNNING
>>> RESCHED : No
>>> HOST : vnode6
>>> CLUSTER ID : 102
>>> CLUSTER : External
>>> START TIME : 11/05 16:13:16
>>> END TIME : -
>>> DEPLOY ID : one-234
>>>
>>> VM DISKS
>>> ID TARGET IMAGE TYPE SAVE SAVE_AS
>>> 1 vda Ubuntu-14.04-one-4.8 rbd NO -
>>>
>>> VM NICS
>>> ID NETWORK VLAN BRIDGE IP MAC
>>> 0 Net_ext yes br0 10.10.10.10 02:00:5e:e6:38:8c
>>>
>>> VIRTUAL MACHINE HISTORY
>>> SEQ HOST ACTION DS START
>>> TIME PROLOG
>>> 0 vnode6 none 103 11/05 16:13:42 6d 19h27m
>>> 0h00m01s
>>>
>>> VIRTUAL MACHINE TEMPLATE
>>> AUTOMATIC_REQUIREMENTS="CLUSTER_ID = 102 & !(PUBLIC_CLOUD = YES)"
>>> CONTEXT=[
>>> DISK_ID="0",
>>> ETH0_DNS="10.10.10.1",
>>> ETH0_GATEWAY="10.10.10.1",
>>> ETH0_IP="10.10.10.10",
>>> ETH0_MAC="02:00:5e:e6:38:8c",
>>> ETH0_MASK="255.255.255.0",
>>> ETH0_NETWORK="10.10.10.0",
>>> HOSTNAME="jenkinssetup",
>>> NETWORK="YES",
>>> TARGET="hda" ]
>>> CPU="2"
>>> GRAPHICS=[
>>> KEYMAP="de",
>>> LISTEN="0.0.0.0",
>>> PORT="6134",
>>> TYPE="VNC" ]
>>> MEMORY="2048"
>>> TEMPLATE_ID="15"
>>> VCPU="1"
>>> VMID="234"
>>>
>>>
>>> ############# second vm #########
>>> onevm show 239
>>> VIRTUAL MACHINE 239 INFORMATION
>>> ID : 239
>>> NAME : Ubuntu 14.04 - KVM - OpenNebula 4.8-239
>>> USER : oneadmin
>>> GROUP : oneadmin
>>> STATE : ACTIVE
>>> LCM_STATE : RUNNING
>>> RESCHED : No
>>> HOST : vnode5
>>> CLUSTER ID : 102
>>> CLUSTER : External
>>> START TIME : 11/12 10:35:27
>>> END TIME : -
>>> DEPLOY ID : one-239
>>>
>>> VIRTUAL MACHINE MONITORING
>>> USED CPU : 0
>>> NET_RX : 959K
>>> USED MEMORY : 779.5M
>>> NET_TX : 55K
>>>
>>> PERMISSIONS
>>> OWNER : um-
>>> GROUP : ---
>>> OTHER : ---
>>>
>>> VM DISKS
>>> ID TARGET IMAGE TYPE SAVE SAVE_AS
>>> 0 vda Ubuntu-14.04-one-4.8 rbd NO -
>>>
>>> VM NICS
>>> ID NETWORK VLAN BRIDGE IP MAC
>>> 0 net-ext2 yes br0 10.10.10.105 02:00:5e:e6:35:69
>>>
>>> VIRTUAL MACHINE HISTORY
>>> SEQ HOST ACTION DS START
>>> TIME PROLOG
>>> 0 vnode5 none 103 11/12 10:35:28 0d 01h16m
>>> 0h00m04s
>>>
>>> VIRTUAL MACHINE TEMPLATE
>>> AUTOMATIC_REQUIREMENTS="CLUSTER_ID = 102 & !(PUBLIC_CLOUD = YES)"
>>> CONTEXT=[
>>> DISK_ID="1",
>>> ETH0_DNS="10.10.10.1",
>>> ETH0_GATEWAY="10.10.10.1",
>>> ETH0_IP="10.10.10.105",
>>> ETH0_MAC="02:00:5e:e6:35:69",
>>> ETH0_MASK="255.255.255.0",
>>> ETH0_NETWORK="10.10.10.0",
>>> NETWORK="YES",
>>> TARGET="hda" ]
>>> CPU="1"
>>> GRAPHICS=[
>>> LISTEN="0.0.0.0",
>>> PORT="6139",
>>> TYPE="vnc" ]
>>> MEMORY="768"
>>> OS=[
>>> ARCH="x86_64" ]
>>> TEMPLATE_ID="22"
>>> VMID="239"
>>>
>>>
>>> ################################
>>> oneimage show 90
>>> IMAGE 90 INFORMATION
>>> ID : 90
>>> NAME : Ubuntu-14.04-one-4.8
>>> USER : oneadmin
>>> GROUP : oneadmin
>>> DATASTORE : ceph
>>> TYPE : OS
>>> REGISTER TIME : 10/29 22:13:27
>>> PERSISTENT : No
>>> SOURCE : one/one-90
>>> PATH : http://marketplace.c12g.com/appliance/
>>> 53e7c1b28fb81d6a69000003/download/0
>>> SIZE : 10G
>>> STATE : used
>>> RUNNING_VMS : 2
>>>
>>> PERMISSIONS
>>> OWNER : um-
>>> GROUP : ---
>>> OTHER : ---
>>>
>>> IMAGE TEMPLATE
>>> DEV_PREFIX="hd"
>>> FROM_APP="53e7c1b28fb81d6a69000003"
>>> FROM_APP_FILE="0"
>>> FROM_APP_NAME="Ubuntu 14.04 - KVM - OpenNebula 4.8"
>>> MD5="bdbd7fdced3ba5be52ba60e801c0b3e0"
>>>
>>> VIRTUAL MACHINES
>>>
>>> ID USER GROUP NAME STAT UCPU UMEM
>>> HOST TIME
>>> 234 op opsadmin jenkinssetup runn 0 2.1G vnode6 6d
>>> 19h47
>>> 239 oneadmin oneadmin Ubuntu 14.04 - runn 0 779.5M vnode5
>>> 0d 01h25
>>>
>>> ############ datastore ############
>>> oneadmin at oned:~$ onedatastore show 102
>>> DATASTORE 102 INFORMATION
>>> ID : 102
>>> NAME : ceph
>>> USER : oneadmin
>>> GROUP : opadmins
>>> CLUSTER : -
>>> TYPE : IMAGE
>>> DS_MAD : ceph
>>> TM_MAD : ceph
>>> BASE PATH : /var/lib/one//datastores/102
>>> DISK_TYPE : RBD
>>>
>>> DATASTORE CAPACITY
>>> TOTAL: : 798.1G
>>> FREE: : 495G
>>> USED: : 263.2G
>>> LIMIT: : -
>>>
>>> PERMISSIONS
>>> OWNER : um-
>>> GROUP : u--
>>> OTHER : ---
>>>
>>> DATASTORE TEMPLATE
>>> BASE_PATH="/var/lib/one//datastores/"
>>> BRIDGE_LIST="oned.test.cloud"
>>> CEPH_SECRET="xxxxxxxxxxxxxxxxxxxxxxxx"
>>> CEPH_USER="libvirt"
>>> CLONE_TARGET="SELF"
>>> DISK_TYPE="RBD"
>>> DS_MAD="ceph"
>>> LN_TARGET="NONE"
>>> POOL_NAME="one"
>>> RBD_FORMAT="2"
>>> TM_MAD="ceph"
>>> TYPE="IMAGE_DS"
>>>
>>> IMAGES
>>> ....
>>>
>>> 90
>>>
>>> ############ ceph #############
>>> rbd ls -l one| egrep 'NAME|one-90'
>>> NAME SIZE PARENT FMT PROT LOCK
>>> one-90 10240M 2
>>> one-90 at 234-1 10240M 2 yes
>>> one-90 at 239-0 10240M 2 yes
>>> one-90-234-1 10240M one/one-90 at 234-1 2
>>> one-90-239-0 10240M one/one-90 at 239-0 2
>>>
>>>
>>
> _______________________________________________
> Users mailing list
> Users at lists.opennebula.org
> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20141112/ccb11a51/attachment-0001.htm>
More information about the Users
mailing list