[one-users] Ceph image format 2 issue for nonpersistent disks
Joe Mauer
aaskarg at gmail.com
Wed Nov 12 07:16:47 PST 2014
Hi,
I´m really sorry for the noise,
but I am able to reproduce the problem now by attaching a non-persistent
image to a diskless vm.
Could somebody try to verify this, please?:
Setup:
ceph datastore, RBD_FORMAT=2
opennebula 4.8 (Sorry, I was not able to update yet)
I run Ubuntu 12.04, kvm
Steps:
1) Create a disk on the ceph datastore (I used the marketplace),
- non-persistent
2) create a template with no disk attached.
3) instantiate a vm from 2)
4) attach disk from 1) -> (On my side, the original disk, not the clone
is attached according to "virsh dumpxml")
5) write a file to the vm and sync or reboot
6) delete the vm and
7) recreate a new one with the disk from 1)
8) Look for file created in 5)
On Wed, Nov 12, 2014 at 1:20 PM, Joe Mauer <aaskarg at gmail.com> wrote:
> Dear list,
> sometimes it helps to write things down.
> By looking at "libvirt dumxml"
> ######## vm 234 ########
> <disk type='network' device='disk'>
> <driver name='qemu' type='raw' cache='writeback'/>
> <auth username='libvirt'>
> <secret type='ceph' uuid='2f146016-197e-4c94-ba2d-6be68d75034a'/>
> </auth>
> <source protocol='rbd' name='one/one-90'/>
> <target dev='vda' bus='virtio'/>
> <alias name='virtio-disk0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
> function='0x0'/>
> </disk>
>
> ################
>
> It seems like the vm has the wrong rbd image attached (one/one-90, not
> one-90-234-1), or am I wrong here?
>
> vm 239 is fine:
> ##########
> <disk type='network' device='disk'>
> <driver name='qemu' type='raw' cache='writeback'/>
> <auth username='libvirt'>
> <secret type='ceph' uuid='2f146016-197e-4c94-ba2d-6be68d75034a'/>
> </auth>
> <source protocol='rbd' name='one/one-90-239-0'/>
> <target dev='vda' bus='virtio'/>
> <alias name='virtio-disk0'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
> </disk>
>
>
> looking at the deployment files, it seems that in the first vm(234), i did
> attach the disk after creating the vm.
>
>
>
> On Wed, Nov 12, 2014 at 12:13 PM, Joe Mauer <aaskarg at gmail.com> wrote:
>
>> Dear List,
>>
>> I´m evaluating a one 4.8 setup, using a ceph 0.72.2 cluster, RBD_FORMAT=2
>>
>> Using the clone option instead of copying images to be able to
>> instantiating vms quickly does make this solution very interesting for us,
>> but I ran into a rather big issue that I´m currently
>> not able to solve (I´m not a ceph or one expert ;-)).
>> I could not find anything similar on the net, so it might be a
>> configuration issue, but I currently ran out of ideas to debug it.
>>
>> In both test clouds, I instaniciated a virtual machine (ID 234 below)
>> from a non-persistent image (e.g. Ubuntu 14.04 from the marketplace), did
>> some changes (e.g. changed the hostname, thats why I discoverd the problem
>> in the first place), updates ... software installation...
>>
>> After a while, when starting a new vm (ID 239) from the (still
>> non-persistent) image I get fsck messages when booting and have packages,
>> and e.g. the hostname from the first vm.
>>
>> Thank you in advance for taking a look.
>>
>> ############## first vm ##################
>> onevm show 234
>> VIRTUAL MACHINE 234 INFORMATION
>> ID : 234
>> NAME : jenkinssetup
>> USER : op
>> GROUP : opsadmin
>> STATE : ACTIVE
>> LCM_STATE : RUNNING
>> RESCHED : No
>> HOST : vnode6
>> CLUSTER ID : 102
>> CLUSTER : External
>> START TIME : 11/05 16:13:16
>> END TIME : -
>> DEPLOY ID : one-234
>>
>> VM DISKS
>> ID TARGET IMAGE TYPE SAVE SAVE_AS
>> 1 vda Ubuntu-14.04-one-4.8 rbd NO -
>>
>> VM NICS
>> ID NETWORK VLAN BRIDGE IP MAC
>> 0 Net_ext yes br0 10.10.10.10 02:00:5e:e6:38:8c
>>
>> VIRTUAL MACHINE HISTORY
>> SEQ HOST ACTION DS START TIME
>> PROLOG
>> 0 vnode6 none 103 11/05 16:13:42 6d 19h27m
>> 0h00m01s
>>
>> VIRTUAL MACHINE TEMPLATE
>> AUTOMATIC_REQUIREMENTS="CLUSTER_ID = 102 & !(PUBLIC_CLOUD = YES)"
>> CONTEXT=[
>> DISK_ID="0",
>> ETH0_DNS="10.10.10.1",
>> ETH0_GATEWAY="10.10.10.1",
>> ETH0_IP="10.10.10.10",
>> ETH0_MAC="02:00:5e:e6:38:8c",
>> ETH0_MASK="255.255.255.0",
>> ETH0_NETWORK="10.10.10.0",
>> HOSTNAME="jenkinssetup",
>> NETWORK="YES",
>> TARGET="hda" ]
>> CPU="2"
>> GRAPHICS=[
>> KEYMAP="de",
>> LISTEN="0.0.0.0",
>> PORT="6134",
>> TYPE="VNC" ]
>> MEMORY="2048"
>> TEMPLATE_ID="15"
>> VCPU="1"
>> VMID="234"
>>
>>
>> ############# second vm #########
>> onevm show 239
>> VIRTUAL MACHINE 239 INFORMATION
>> ID : 239
>> NAME : Ubuntu 14.04 - KVM - OpenNebula 4.8-239
>> USER : oneadmin
>> GROUP : oneadmin
>> STATE : ACTIVE
>> LCM_STATE : RUNNING
>> RESCHED : No
>> HOST : vnode5
>> CLUSTER ID : 102
>> CLUSTER : External
>> START TIME : 11/12 10:35:27
>> END TIME : -
>> DEPLOY ID : one-239
>>
>> VIRTUAL MACHINE MONITORING
>> USED CPU : 0
>> NET_RX : 959K
>> USED MEMORY : 779.5M
>> NET_TX : 55K
>>
>> PERMISSIONS
>> OWNER : um-
>> GROUP : ---
>> OTHER : ---
>>
>> VM DISKS
>> ID TARGET IMAGE TYPE SAVE SAVE_AS
>> 0 vda Ubuntu-14.04-one-4.8 rbd NO -
>>
>> VM NICS
>> ID NETWORK VLAN BRIDGE IP MAC
>> 0 net-ext2 yes br0 10.10.10.105 02:00:5e:e6:35:69
>>
>> VIRTUAL MACHINE HISTORY
>> SEQ HOST ACTION DS START TIME
>> PROLOG
>> 0 vnode5 none 103 11/12 10:35:28 0d 01h16m
>> 0h00m04s
>>
>> VIRTUAL MACHINE TEMPLATE
>> AUTOMATIC_REQUIREMENTS="CLUSTER_ID = 102 & !(PUBLIC_CLOUD = YES)"
>> CONTEXT=[
>> DISK_ID="1",
>> ETH0_DNS="10.10.10.1",
>> ETH0_GATEWAY="10.10.10.1",
>> ETH0_IP="10.10.10.105",
>> ETH0_MAC="02:00:5e:e6:35:69",
>> ETH0_MASK="255.255.255.0",
>> ETH0_NETWORK="10.10.10.0",
>> NETWORK="YES",
>> TARGET="hda" ]
>> CPU="1"
>> GRAPHICS=[
>> LISTEN="0.0.0.0",
>> PORT="6139",
>> TYPE="vnc" ]
>> MEMORY="768"
>> OS=[
>> ARCH="x86_64" ]
>> TEMPLATE_ID="22"
>> VMID="239"
>>
>>
>> ################################
>> oneimage show 90
>> IMAGE 90 INFORMATION
>> ID : 90
>> NAME : Ubuntu-14.04-one-4.8
>> USER : oneadmin
>> GROUP : oneadmin
>> DATASTORE : ceph
>> TYPE : OS
>> REGISTER TIME : 10/29 22:13:27
>> PERSISTENT : No
>> SOURCE : one/one-90
>> PATH :
>> http://marketplace.c12g.com/appliance/53e7c1b28fb81d6a69000003/download/0
>> SIZE : 10G
>> STATE : used
>> RUNNING_VMS : 2
>>
>> PERMISSIONS
>> OWNER : um-
>> GROUP : ---
>> OTHER : ---
>>
>> IMAGE TEMPLATE
>> DEV_PREFIX="hd"
>> FROM_APP="53e7c1b28fb81d6a69000003"
>> FROM_APP_FILE="0"
>> FROM_APP_NAME="Ubuntu 14.04 - KVM - OpenNebula 4.8"
>> MD5="bdbd7fdced3ba5be52ba60e801c0b3e0"
>>
>> VIRTUAL MACHINES
>>
>> ID USER GROUP NAME STAT UCPU UMEM
>> HOST TIME
>> 234 op opsadmin jenkinssetup runn 0 2.1G vnode6 6d 19h47
>> 239 oneadmin oneadmin Ubuntu 14.04 - runn 0 779.5M vnode5
>> 0d 01h25
>>
>> ############ datastore ############
>> oneadmin at oned:~$ onedatastore show 102
>> DATASTORE 102 INFORMATION
>> ID : 102
>> NAME : ceph
>> USER : oneadmin
>> GROUP : opadmins
>> CLUSTER : -
>> TYPE : IMAGE
>> DS_MAD : ceph
>> TM_MAD : ceph
>> BASE PATH : /var/lib/one//datastores/102
>> DISK_TYPE : RBD
>>
>> DATASTORE CAPACITY
>> TOTAL: : 798.1G
>> FREE: : 495G
>> USED: : 263.2G
>> LIMIT: : -
>>
>> PERMISSIONS
>> OWNER : um-
>> GROUP : u--
>> OTHER : ---
>>
>> DATASTORE TEMPLATE
>> BASE_PATH="/var/lib/one//datastores/"
>> BRIDGE_LIST="oned.test.cloud"
>> CEPH_SECRET="xxxxxxxxxxxxxxxxxxxxxxxx"
>> CEPH_USER="libvirt"
>> CLONE_TARGET="SELF"
>> DISK_TYPE="RBD"
>> DS_MAD="ceph"
>> LN_TARGET="NONE"
>> POOL_NAME="one"
>> RBD_FORMAT="2"
>> TM_MAD="ceph"
>> TYPE="IMAGE_DS"
>>
>> IMAGES
>> ....
>>
>> 90
>>
>> ############ ceph #############
>> rbd ls -l one| egrep 'NAME|one-90'
>> NAME SIZE PARENT FMT PROT LOCK
>> one-90 10240M 2
>> one-90 at 234-1 10240M 2 yes
>> one-90 at 239-0 10240M 2 yes
>> one-90-234-1 10240M one/one-90 at 234-1 2
>> one-90-239-0 10240M one/one-90 at 239-0 2
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20141112/c1cb8082/attachment.htm>
More information about the Users
mailing list