[one-users] [OVS] Problem with OpenVSwitch and some questions about Ceph
Jon
three18ti at gmail.com
Sun Mar 31 12:58:46 PDT 2013
Hello Jamie,
Thanks for getting back to me, I apologize for the delayed response but
I've been in the middle of a move.
>> I'll separate each topic in different messages, so we can continue
separate sub-threads per topic
Thanks for that. I didn't want to spam the list with three separate
replies, this works.
>> Could you do the following right after the VM has been started?
>>sudo ovs-ofctl dump-ports ovsbr0 vnet1
sure, I wrapped that in a loop for easy copy paste, then I started this,
then attempted to create a new VM.
All I get is an error "ovs-ofctl: ovsbr0: couldn't find port `vnet1'"
repeatedly.
>> root at loki:~# perl -e 'while(1){ `ovs-ofctl dump-ports ovsbr0 vnet1`;
sleep(3)}'
>> ovs-ofctl: ovsbr0: couldn't find port `vnet1'
>> ovs-ofctl: ovsbr0: couldn't find port `vnet1'
>> ...
>> ovs-ofctl: ovsbr0: couldn't find port `vnet1'
Here's the log from the transaction,
>> Fri Mar 29 11:16:46 2013 [DiM][I]: New VM state is ACTIVE.
>> Fri Mar 29 11:16:46 2013 [LCM][I]: New VM state is PROLOG.
>> Fri Mar 29 11:16:46 2013 [VM][I]: Virtual Machine has no context
>> Fri Mar 29 11:18:14 2013 [LCM][E]: monitor_done_action, VM in a wrong
state
>> Fri Mar 29 11:23:20 2013 [LCM][E]: monitor_done_action, VM in a wrong
state
>> Fri Mar 29 11:28:29 2013 [LCM][E]: monitor_done_action, VM in a wrong
state
>> Fri Mar 29 11:29:06 2013 [TM][I]: clone: Cloning
/var/lib/one/datastores/1/68204b0b2e950587e337ebc7cbe4243e in
loki:/var/lib/one//datastores/0/35/disk.0
>> Fri Mar 29 11:29:06 2013 [TM][I]: ExitCode: 0
>> Fri Mar 29 11:29:06 2013 [LCM][I]: New VM state is BOOT
>> Fri Mar 29 11:29:06 2013 [VMM][I]: Generating deployment file:
/var/lib/one/vms/35/deployment.0
>> Fri Mar 29 11:29:09 2013 [VMM][I]: ExitCode: 0
>> Fri Mar 29 11:29:09 2013 [VMM][I]: Successfully execute network driver
operation: pre.
>> Fri Mar 29 11:29:12 2013 [VMM][I]: ExitCode: 0
>> Fri Mar 29 11:29:12 2013 [VMM][I]: Successfully execute virtualization
driver operation: deploy.
>> Fri Mar 29 11:29:12 2013 [VMM][I]: Command execution fail:
/var/tmp/one/vnm/ovswitch/post
PFZNPjxJRD4zNTwvSUQ+PFVJRD4wPC9VSUQ+PEdJRD4wPC9HSUQ+PFVOQU1FPm9uZWFkbWluPC9VTkFNRT48R05BTUU+b25lYWRtaW48L0dOQU1FPjxOQU1FPnVidW50dS1uZXR3b3JraW5nLTM1PC9OQU1FPjxQRVJNSVNTSU9OUz48T1dORVJfVT4xPC9PV05FUl9VPjxPV05FUl9NPjE8L09XTkVSX00+PE9XTkVSX0E+MDwvT1dORVJfQT48R1JPVVBfVT4wPC9HUk9VUF9VPjxHUk9VUF9NPjA8L0dST1VQX00+PEdST1VQX0E+MDwvR1JPVVBfQT48T1RIRVJfVT4wPC9PVEhFUl9VPjxPVEhFUl9NPjA8L09USEVSX00+PE9USEVSX0E+MDwvT1RIRVJfQT48L1BFUk1JU1NJT05TPjxMQVNUX1BPTEw+MDwvTEFTVF9QT0xMPjxTVEFURT4zPC9TVEFURT48TENNX1NUQVRFPjI8L0xDTV9TVEFURT48UkVTQ0hFRD4wPC9SRVNDSEVEPjxTVElNRT4xMzY0NTc3NDAwPC9TVElNRT48RVRJTUU+MDwvRVRJTUU+PERFUExPWV9JRC8+PE1FTU9SWT4wPC9NRU1PUlk+PENQVT4wPC9DUFU+PE5FVF9UWD4wPC9ORVRfVFg+PE5FVF9SWD4wPC9ORVRfUlg+PFRFTVBMQVRFPjxDUFU+PCFbQ0RBVEFbMV1dPjwvQ1BVPjxESVNLPjxDTE9ORT48IVtDREFUQVtZRVNdXT48L0NMT05FPjxEQVRBU1RPUkU+PCFbQ0RBVEFbZGVmYXVsdF1dPjwvREFUQVNUT1JFPjxEQVRBU1RPUkVfSUQ+PCFbQ0RBVEFbMV1dPjwvREFUQVNUT1JFX0lEPjxERVZfUFJFRklYPjwhW0NEQVRBW2hkXV0+PC9ERVZfUFJFRklYPjxESVNLX0lEPjwhW0NEQVRBWzBdXT48L0RJU0tfSUQ+PERSSVZFUj48IVtDREFUQVtyYXddXT48L0RSSVZFUj48SU1BR0U+PCFbQ0RBVEFbVWJ1bnR1IFNlcnZlciAxMi4wNCAoUHJlY2lzZSBQYW5nb2xpbikgLSBrdm1dXT48L0lNQUdFPjxJTUFHRV9JRD48IVtDREFUQVs1XV0+PC9JTUFHRV9JRD48UkVBRE9OTFk+PCFbQ0RBVEFbTk9dXT48L1JFQURPTkxZPjxTQVZFPjwhW0NEQVRBW05PXV0+PC9TQVZFPjxTT1VSQ0U+PCFbQ0RBVEFbL3Zhci9saWIvb25lL2RhdGFzdG9yZXMvMS82ODIwNGIwYjJlOTUwNTg3ZTMzN2ViYzdjYmU0MjQzZV1dPjwvU09VUkNFPjxUQVJHRVQ+PCFbQ0RBVEFbaGRhXV0+PC9UQVJHRVQ+PFRNX01BRD48IVtDREFUQVtzaGFyZWRdXT48L1RNX01BRD48VFlQRT48IVtDREFUQVtGSUxFXV0+PC9UWVBFPjwvRElTSz48R1JBUEhJQ1M+PExJU1RFTj48IVtDREFUQVswLjAuMC4wXV0+PC9MSVNURU4+PFBPUlQ+PCFbQ0RBVEFbNTkzNV1dPjwvUE9SVD48VFlQRT48IVtDREFUQVtWTkNdXT48L1RZUEU+PC9HUkFQSElDUz48TUVNT1JZPjwhW0NEQVRBWzEwMjRdXT48L01FTU9SWT48TklDPjxCUklER0U+PCFbQ0RBVEFbb3ZzYnIwXV0+PC9CUklER0U+PElQPjwhW0NEQVRBWzY4LjcxLjEzMS42N11dPjwvSVA+PElQNl9MSU5LPjwhW0NEQVRBW2ZlODA6OjQwMDo0NGZmOmZlNDc6ODM0M11dPjwvSVA2X0xJTks+PE1BQz48IVtDREFUQVswMjowMDo0NDo0Nzo4Mzo0M11dPjwvTUFDPjxORVRXT1JLPjwhW0NEQVRBW3Rlc3RuZXQxXV0+PC9ORVRXT1JLPjxORVRXT1JLX0lEPjwhW0NEQVRBWzBdXT48L05FVFdPUktfSUQ+PE5JQ19JRD48IVtDREFUQVswXV0+PC9OSUNfSUQ+PFZMQU4+PCFbQ0RBVEFbTk9dXT48L1ZMQU4+PC9OSUM+PE9TPjxBUkNIPjwhW0NEQVRBW3g4Nl82NF1dPjwvQVJDSD48L09TPjxURU1QTEFURV9JRD48IVtDREFUQVs3XV0+PC9URU1QTEFURV9JRD48VkNQVT48IVtDREFUQVsyXV0+PC9WQ1BVPjxWTUlEPjwhW0NEQVRBWzM1XV0+PC9WTUlEPjwvVEVNUExBVEU+PFVTRVJfVEVNUExBVEU+PFVOREVGSU5FRD48IVtDREFUQVsyLjAwXV0+PC9VTkRFRklORUQ+PC9VU0VSX1RFTVBMQVRFPjxISVNUT1JZX1JFQ09SRFM+PEhJU1RPUlk+PE9JRD4zNTwvT0lEPjxTRVE+MDwvU0VRPjxIT1NUTkFNRT5sb2tpPC9IT1NUTkFNRT48SElEPjA8L0hJRD48U1RJTUU+MTM2NDU3NzQwNjwvU1RJTUU+PEVUSU1FPjA8L0VUSU1FPjxWTU1NQUQ+a3ZtPC9WTU1NQUQ+PFZOTU1BRD5vdnN3aXRjaDwvVk5NTUFEPjxUTU1BRD5zaGFyZWQ8L1RNTUFEPjxEU19MT0NBVElPTj4vdmFyL2xpYi9vbmUvL2RhdGFzdG9yZXM8L0RTX0xPQ0FUSU9OPjxEU19JRD4wPC9EU19JRD48UFNUSU1FPjEzNjQ1Nzc0MDY8L1BTVElNRT48UEVUSU1FPjEzNjQ1NzgxNDY8L1BFVElNRT48UlNUSU1FPjEzNjQ1NzgxNDY8L1JTVElNRT48UkVUSU1FPjA8L1JFVElNRT48RVNUSU1FPjA8L0VTVElNRT48RUVUSU1FPjA8L0VFVElNRT48UkVBU09OPjA8L1JFQVNPTj48L0hJU1RPUlk+PC9ISVNUT1JZX1JFQ09SRFM+PC9WTT4=
one-35
>> Fri Mar 29 11:29:12 2013 [VMM][I]: sudo: no tty present and no askpass
program specified
>> Fri Mar 29 11:29:12 2013 [VMM][E]: post: Command "sudo
/usr/bin/ovs-ofctl add-flow ovsbr0
in_port=,dl_src=02:00:44:47:83:43,priority=40000,actions=normal" failed.
>> Fri Mar 29 11:29:12 2013 [VMM][E]: post: sudo: no tty present and no
askpass program specified
>> Fri Mar 29 11:29:12 2013 [VMM][E]: sudo: no tty present and no askpass
program specified
>> Fri Mar 29 11:29:12 2013 [VMM][E]:
>> Fri Mar 29 11:29:12 2013 [VMM][I]: ExitCode: 1
>> Fri Mar 29 11:29:24 2013 [VMM][I]: error: failed to get domain 'one-35'
>> Fri Mar 29 11:29:24 2013 [VMM][I]: error: Domain not found: no domain
with matching name 'one-35'
>> Fri Mar 29 11:29:24 2013 [VMM][I]: ExitCode: 0
>> Fri Mar 29 11:29:24 2013 [VMM][I]: Successfully execute virtualization
driver operation: cancel.
>> Fri Mar 29 11:29:24 2013 [VMM][I]: Failed to execute network driver
operation: post.
>> Fri Mar 29 11:29:24 2013 [VMM][E]: Error deploying virtual machine:
sudo: no tty present and no askpass program specified
>> Fri Mar 29 11:29:24 2013 [DiM][I]: New VM state is FAILED
The big base64 string decodes to:
>>
<VM><ID>35</ID><UID>0</UID><GID>0</GID><UNAME>oneadmin</UNAME><GNAME>oneadmin</GNAME><NAME>ubuntu-networking-35</NAME><PERMISSIONS><OWNER_U>1</OWNER_U><OWNER_M>1</OWNER_M><OWNER_A>0</OWNER_A><GROUP_U>0</GROUP_U><GROUP_M>0</GROUP_M><GROUP_A>0</GROUP_A><OTHER_U>0</OTHER_U><OTHER_M>0</OTHER_M><OTHER_A>0</OTHER_A></PERMISSIONS><LAST_POLL>0</LAST_POLL><STATE>3</STATE><LCM_STATE>2</LCM_STATE><RESCHED>0</RESCHED><STIME>1364577400</STIME><ETIME>0</ETIME><DEPLOY_ID/><MEMORY>0</MEMORY><CPU>0</CPU><NET_TX>0</NET_TX><NET_RX>0</NET_RX><TEMPLATE><CPU><![CDATA[1]]></CPU><DISK><CLONE><![CDATA[YES]]></CLONE><DATASTORE><![CDATA[default]]></DATASTORE><DATASTORE_ID><![CDATA[1]]></DATASTORE_ID><DEV_PREFIX><![CDATA[hd]]></DEV_PREFIX><DISK_ID><![CDATA[0]]></DISK_ID><DRIVER><![CDATA[raw]]></DRIVER><IMAGE><![CDATA[Ubuntu
Server 12.04 (Precise Pangolin) -
kvm]]></IMAGE><IMAGE_ID><![CDATA[5]]></IMAGE_ID><READONLY><![CDATA[NO]]></READONLY><SAVE><![CDATA[NO]]></SAVE><SOURCE><![CDATA[/var/lib/one/datastores/1/68204b0b2e950587e337ebc7cbe4243e]]></SOURCE><TARGET><![CDATA[hda]]></TARGET><TM_MAD><![CDATA[shared]]></TM_MAD><TYPE><![CDATA[FILE]]></TYPE></DISK><GRAPHICS><LISTEN><![CDATA[0.0.0.0]]></LISTEN><PORT><![CDATA[5935]]></PORT><TYPE><![CDATA[VNC]]></TYPE></GRAPHICS><MEMORY><![CDATA[1024]]></MEMORY><NIC><BRIDGE><![CDATA[ovsbr0]]></BRIDGE><IP><![CDATA[68.71.131.67]]></IP><IP6_LINK><![CDATA[fe80::400:44ff:fe47:8343]]></IP6_LINK><MAC><![CDATA[02:00:44:47:83:43]]></MAC><NETWORK><![CDATA[testnet1]]></NETWORK><NETWORK_ID><![CDATA[0]]></NETWORK_ID><NIC_ID><![CDATA[0]]></NIC_ID><VLAN><![CDATA[NO]]></VLAN></NIC><OS><ARCH><![CDATA[x86_64]]></ARCH></OS><TEMPLATE_ID><![CDATA[7]]></TEMPLATE_ID><VCPU><![CDATA[2]]></VCPU><VMID><![CDATA[35]]></VMID></TEMPLATE><USER_TEMPLATE><UNDEFINED><![CDATA[2.00]]></UNDEFINED></USER_TEMPLATE><HISTORY_RECORDS><HISTORY><OID>35</OID><SEQ>0</SEQ><HOSTNAME>loki</HOSTNAME><HID>0</HID><STIME>1364577406</STIME><ETIME>0</ETIME><VMMMAD>kvm</VMMMAD><VNMMAD>ovswitch</VNMMAD><TMMAD>shared</TMMAD><DS_LOCATION>/var/lib/one//datastores</DS_LOCATION><DS_ID>0</DS_ID><PSTIME>1364577406</PSTIME><PETIME>1364578146</PETIME><RSTIME>1364578146</RSTIME><RETIME>0</RETIME><ESTIME>0</ESTIME><EETIME>0</EETIME><REASON>0</REASON></HISTORY></HISTORY_RECORDS></VM>
Thanks,
Jon A
On Thu, Mar 28, 2013 at 2:45 PM, Jaime Melis <jmelis at opennebula.org> wrote:
> Hi Jon,
>
> thanks for your feedback! it really helps us. I'll separate each topic in
> different messages, so we can continue separate sub-threads per topic
>
> 1) OVS
>
> So, it's normal that you got this error:
> >> [TemplateInstantiate] Error allocating a new virtual machine. Cannot
> get IP/MAC lease from virtual network 0.
> if no LEASES where defined. An empty network is similar to a network where
> all the leases are taken, so this is expected behaviour.
>
> However, the other error message is a bit more worrying. It looks like
> there is a bug. The bit of code that's failing is this one [1], the "port"
> method. Apparently the dump-ports command is failing. Could you do the
> following right after the VM has been started?
>
> sudo ovs-ofctl dump-ports ovsbr0 vnet1
>
> and see what happens?
>
>
>
> On Thu, Mar 28, 2013 at 1:06 PM, Jon <three18ti at gmail.com> wrote:
>
>> Hello All,
>>
>> I've just installed OpenNebula 3.9.80 and I have to say this is
>> amazing. Everything works so smoothly.
>>
>> Anyway, down to business.
>>
>> OpenVSwitch:
>>
>> I've installed and configured OpenVSwitch and am able to manually add
>> the OVS config using libvirt, then launch a VM,
>>
>> >> <interface type='bridge'>
>> >> <source bridge='ovsbr0'/>
>> >> <virtualport type='openvswitch'>
>> >> </virtualport>
>> >> <model type='virtio'/>
>> >> <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
>> function='0x0'/>
>> >> </interface>
>>
>> Which creates the device in ovs:
>>
>> >> system at ovsbr0:
>> >> lookups: hit:1346333 missed:46007 lost:0
>> >> flows: 8
>> >> port 0: ovsbr0 (internal)
>> >> port 1: eth0
>> >> port 2: br0 (internal)
>> >> port 7: vnet0
>>
>>
>> However, when I attempt to create a virtual network without assigning
>> an IP and instantiate the template I get the error:
>>
>> >> [TemplateInstantiate] Error allocating a new virtual machine. Cannot
>> get IP/MAC lease from virtual network 0.
>>
>> The template of the virtual network is:
>>
>> >> oneadmin at loki:~$ onevnet show testnet1
>> >> VIRTUAL NETWORK 0 INFORMATION
>> >> ID : 0
>> >> NAME : testnet1
>> >> USER : oneadmin
>> >> GROUP : oneadmin
>> >> CLUSTER : -
>> >> TYPE : FIXED
>> >> BRIDGE : ovsbr0
>> >> VLAN : No
>> >> USED LEASES : 0
>> >>
>> >> PERMISSIONS
>> >> OWNER : um-
>> >> GROUP : ---
>> >> OTHER : ---
>> >>
>> >> VIRTUAL NETWORK TEMPLATE
>> >>
>> >>
>> >> VIRTUAL MACHINES
>> >>
>>
>> If I add an IP to the vnet, I get the following template and error
>> logs (full vm log attached, I think I've identified the relevant
>> line):
>>
>> >> Thu Mar 28 10:34:05 2013 [VMM][E]: post: Command "sudo
>> /usr/bin/ovs-ofctl add-flow ovsbr0
>> in_port=,dl_src=02:00:44:47:83:43,priority=40000,actions=normal" failed.
>>
>> >> oneadmin at loki:~$ onevnet show testnet1
>> >> VIRTUAL NETWORK 0 INFORMATION
>> >> ID : 0
>> >> NAME : testnet1
>> >> USER : oneadmin
>> >> GROUP : oneadmin
>> >> CLUSTER : -
>> >> TYPE : FIXED
>> >> BRIDGE : ovsbr0
>> >> VLAN : No
>> >> USED LEASES : 1
>> >>
>> >> PERMISSIONS
>> >> OWNER : um-
>> >> GROUP : ---
>> >> OTHER : ---
>> >>
>> >> VIRTUAL NETWORK TEMPLATE
>> >>
>> >>
>> >> USED LEASES
>> >> LEASE=[ MAC="02:00:44:47:83:43", IP="192.168.0.2",
>> IP6_LINK="fe80::400:44ff:fe47:8343", USED="1", VID="7" ]
>> >>
>> >> VIRTUAL MACHINES
>> >>
>> >> ID USER GROUP NAME STAT UCPU UMEM HOST
>> TIME
>> >> 7 oneadmin oneadmin template-4-7 fail 0 0K
>> 0d 00h00
>>
>> >> root at loki:~# cat /var/log/openvswitch/ovs-vswitchd.log
>> >> Mar 28 10:34:04|00081|bridge|INFO|created port vnet1 on bridge ovsbr0
>> >> Mar 28 10:34:07|00082|netdev_linux|WARN|ethtool command ETHTOOL_GSET
>> on network device vnet1 failed: No such device
>> >> Mar 28 10:34:07|00083|netdev_linux|INFO|ioctl(SIOCGIFHWADDR) on vnet1
>> device failed: No such device
>> >> Mar 28 10:34:07|00084|netdev|WARN|failed to get flags for network
>> device vnet1: No such device
>> >> Mar 28 10:34:07|00085|netdev|WARN|failed to retrieve MTU for network
>> device vnet1: No such device
>> >> Mar 28 10:34:07|00086|netdev|WARN|failed to get flags for network
>> device vnet1: No such device
>> >> Mar 28 10:34:07|00087|bridge|INFO|destroyed port vnet1 on bridge
>> ovsbr0
>>
>> I attempted to run the command but I never set a password for the
>> oneadmin user, but I don't think it's a permissions / sudo access
>> problem.
>>
>> Not really sure where to look next. Any ideas are appreciated.
>>
>> CEPH:
>>
>> I'm trying to use a Ceph datastore with a RBD instead of a cephFS
>> (it's an option).
>> When I try to create a Ceph datastore with a RBD type, I get a state
>> of "Error",
>> but I'm not sure where to look for relevant logs, oned.log didn't seem
>> to have anything,
>> or maybe I'm just grepping for the wrong string.
>>
>> As a work around, I have been creating the directory, creating the rbd
>> then manually mounting it,
>> this seems to work in my test environment, but doesn't seem very
>> scalable, how are others using CEPH?
>>
>> Documentation:
>>
>> I've noticed some errors in the documentation, namely the location of
>> the install scripts,
>>
>> The docs state they are in:
>> >> /usr/share/one/install_gems
>> >> /usr/share/one/sunstone/install_novnc.sh
>>
>> However, I found them in:
>> >> /usr/share/opennebula/install_gems
>> >> /usr/share/opennebula/install_novnc.sh
>> >> /usr/share/opennebula/sunstone/install_novnc.sh
>>
>> Is there some repository of the documentation somewhere that we can
>> contribute to?
>> It's a small thing, but when I'm going through the instructions, I
>> like to copy / paste;
>> I figured it out, but I know if it caused me problems, it might cause
>> others problems too.
>>
>> Thanks again, I can't wait for the final release of OpenNebula!
>>
>> Best Regards,
>> Jon A
>>
>> _______________________________________________
>> Users mailing list
>> Users at lists.opennebula.org
>> http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
>>
>>
>
>
> --
> Jaime Melis
> Project Engineer
> OpenNebula - The Open Source Toolkit for Cloud Computing
> www.OpenNebula.org | jmelis at opennebula.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20130331/9320ab94/attachment-0002.htm>
More information about the Users
mailing list