[one-users] [Docs] Problem with OpenVSwitch and some questions about Ceph
Jon
three18ti at gmail.com
Sat Apr 13 00:50:31 PDT 2013
Ok, I was able to turn up a VM "manually" based on the one xml template
The only difference is I don't have a similar stanza in mine:
>> <target dev='vnet0'/>
>> <alias name='net0'/>
dunno if that has anything to do with it, but the issue is decidedly not my
ovs configuration,
so I think the problem lies in the ovs post script.
Here's my xml config file I used to define the vm.
>> <domain type='kvm' id='4'>
>> <name>ubuntu-manual-test</name>
>> <memory unit='KiB'>1048576</memory>
>> <currentMemory unit='KiB'>1048576</currentMemory>
>> <vcpu placement='static'>2</vcpu>
>> <cputune>
>> <shares>512</shares>
>> </cputune>
>> <os>
>> <type arch='x86_64' machine='pc-1.2'>hvm</type>
>> <boot dev='hd'/>
>> </os>
>> <features>
>> <acpi/>
>> </features>
>> <clock offset='utc'/>
>> <on_poweroff>destroy</on_poweroff>
>> <on_reboot>restart</on_reboot>
>> <on_crash>destroy</on_crash>
>> <devices>
>> <emulator>/usr/bin/kvm</emulator>
>> <disk type='file' device='disk'>
>> <driver name='qemu' type='raw' cache='none'/>
>> <source
file='/var/lib/libvirt/images/vm_storage/ubuntu-manual-image.qcow2'/>
>> <target dev='hda' bus='ide'/>
>> <alias name='ide0-0-0'/>
>> <address type='drive' controller='0' bus='0' target='0' unit='0'/>
>> </disk>
>> <controller type='usb' index='0'>
>> <alias name='usb0'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x2'/>
>> </controller>
>> <controller type='ide' index='0'>
>> <alias name='ide0'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
function='0x1'/>
>> </controller>
>> <interface type='bridge'>
>> <source bridge='ovsbr0'/>
>> <virtualport type='openvswitch'>
>> </virtualport>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
function='0x0'/>
>> </interface>
>> <input type='mouse' bus='ps2'/>
>> <graphics type='vnc' port='5940' autoport='no' listen='0.0.0.0'>
>> <listen type='address' address='0.0.0.0'/>
>> </graphics>
>> <video>
>> <model type='cirrus' vram='9216' heads='1'/>
>> <alias name='video0'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
function='0x0'/>
>> </video>
>> <memballoon model='virtio'>
>> <alias name='balloon0'/>
>> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
function='0x0'/>
>> </memballoon>
>> </devices>
>> <seclabel type='dynamic' model='apparmor' relabel='yes'>
>> </seclabel>
>> </domain>
On Thu, Apr 11, 2013 at 1:09 PM, Jon <three18ti at gmail.com> wrote:
> Hello Jamie,
>
> Thanks for that... Sometimes the best place to hide the obvious answer, is
> in plain sight.
>
> This appears to run correctly, however, there is still no traffic leaving
> the host from the VM.
> Do you think this is related to disabling the post script for ovs or is
> this a problem with my configuration of ovs?
>
> It looks like the vm class definition is here:
>
> >>
> https://github.com/OpenNebula/one/blob/master/src/vnm_mad/remotes/OpenNebulaNetwork.rb#L45
>
> Maybe I can use that with the bit of code you indicated yesterday?
>
> Thanks for all your help.
> Jon A
>
>
>
>
> On Thu, Apr 11, 2013 at 7:37 AM, Jaime Melis <jmelis at opennebula.org>wrote:
>
>> Hi Jon,
>>
>> >> root at loki:~# ovs-ofctl dump-ports ovsbr0 vnet0
>> >> OFPST_PORT reply (xid=0x1): 1 ports
>> >> port 7: rx pkts=34, bytes=1644, drop=0, errs=0, frame=0, over=0,
>> crc=0
>> >> tx pkts=277, bytes=16622, drop=0, errs=0, coll=0
>>
>> the in_port we're looking for is '7' from the above snippet, so you
>> should be running:
>>
>> >> root at loki:~# ovs-ofctl add-flow ovsbr0
>> "in_port=7,dl_src=02:00:44:47:83:43,priority=40000,actions=normal"
>>
>> cheers,
>> Jaime
>>
>>
>> On Thu, Apr 11, 2013 at 1:14 AM, Jon <three18ti at gmail.com> wrote:
>>
>>> Hello Jamie,
>>>
>>> Well, I'm able to ssh from the host to the VM, so we're making progress
>>> at least :)
>>>
>>> When I add vnet0 to the in_port= I get the following error:
>>>
>>> >> root at loki:~# ovs-ofctl add-flow ovsbr0
>>> "in_port=vnet0,dl_src=02:00:44:47:83:43,priority=40000,actions=normal"
>>> >> ovs-ofctl: vnet0: bad syntax for in_port
>>>
>>> When I set in_pot=LOCAL, the command is accepted, but the server is
>>> unable to make outbound connections, so I don't think that's the solution:
>>>
>>> >> root at loki:~# ovs-ofctl add-flow ovsbr0
>>> "in_port=LOCAL,dl_src=02:00:44:47:83:43,priority=40000,actions=normal"
>>>
>>> >> root at loki:~# ovs-ofctl show ovsbr0
>>> >> OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:000000259030761a
>>> >> n_tables:255, n_buffers:256
>>> >> features: capabilities:0xc7, actions:0xfff
>>> >> 1(eth0): addr:00:25:90:30:76:1a
>>> >> config: 0
>>> >> state: 0
>>> >> current: 1GB-FD COPPER AUTO_NEG
>>> >> advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER
>>> AUTO_NEG
>>> >> supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER
>>> AUTO_NEG
>>> >> 2(br0): addr:22:de:1a:d5:63:74
>>> >> config: 0
>>> >> state: 0
>>> >> 7(vnet0): addr:fe:00:44:47:83:43
>>> >> config: 0
>>> >> state: 0
>>> >> current: 10MB-FD COPPER
>>> >> 8(vnet1): addr:fe:00:44:47:83:44
>>> >> config: 0
>>> >> state: 0
>>> >> current: 10MB-FD COPPER
>>> >> LOCAL(ovsbr0): addr:00:25:90:30:76:1a
>>> >> config: PORT_DOWN
>>> >> state: LINK_DOWN
>>> >> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0
>>>
>>> I guess I really don't understand what the correct syntax is to create a
>>> flow in this manner, so I'm kinda spinning my wheels. (I've been creating
>>> ports with `ovs-vsctl add-port`).
>>>
>>> Thanks for all your help,
>>> Jon A
>>>
>>>
>>> On Wed, Apr 10, 2013 at 4:16 PM, Jon <three18ti at gmail.com> wrote:
>>>
>>>> Hello Jamie,
>>>>
>>>> Thanks for getting back to me.
>>>>
>>>>
>>>> >> Yes, brcompat is now no longer required, and we recommend that if
>>>> you're using kvm/libvirt you shouldn't enable the brcompat module
>>>>
>>>> Awesome, thanks for clarifying that for me.
>>>>
>>>>
>>>> >> We recommend doing it like this:
>>>> >> oneadmin ALL=(ALL) NOPASSWD: /usr/bin/ovs-ofctl
>>>>
>>>> Gotcha, so ovs-ofctl is the only command that oneadmin needs to use.
>>>> That makes perfect sense, thanks.
>>>>
>>>> Ok, I'm reporting your steps out of order because I think the pertinent
>>>> information is this first bit.
>>>>
>>>> --5) finally, do $ sudo ovs_ofctl dump-ports ovsbr0 <vnet>
>>>>
>>>> Here we go:
>>>>
>>>> >> root at loki:~# ovs-ofctl dump-ports ovsbr0 vnet0
>>>> >> OFPST_PORT reply (xid=0x1): 1 ports
>>>> >> port 7: rx pkts=34, bytes=1644, drop=0, errs=0, frame=0, over=0,
>>>> crc=0
>>>> >> tx pkts=277, bytes=16622, drop=0, errs=0, coll=0
>>>>
>>>>
>>>> --3) do virsh dumpxml one-<vmid>
>>>>
>>>> Here's the relevant stanza:
>>>>
>>>> >> <interface type='bridge'>
>>>> >> <mac address='02:00:44:47:83:43'/>
>>>>
>>>> >> <source bridge='ovsbr0'/>
>>>> >> <virtualport type='openvswitch'>
>>>> >> <parameters
>>>> interfaceid='79c73248-4ebf-cc14-9f98-98be2d2a785b'/>
>>>> >> </virtualport>
>>>> >> <target dev='vnet0'/>
>>>> >> <alias name='net0'/>
>>>> >> <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
>>>> function='0x0'/>
>>>> >> </interface>
>>>>
>>>> -- 4) this bit of code should be able to extract the tap interface from
>>>> the previous output:
>>>> Honestly, my ruby is a bit rusty, what do I have to provide as "vm"? is
>>>> that just the virsh xml config?
>>>>
>>>> maybe something along the lines of:
>>>>
>>>> #!/usr/bin/ruby
>>>>
>>>> def get_tap(vm)
>>>> dumpxml = vm.vm_info[:dumpxml]
>>>> if dumpxml
>>>> dumpxml_root = REXML::Document.new(dumpxml).root
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> xpath = "devices/interface[@type='bridge']/"
>>>> xpath << "mac[@address='#{self[:mac]}']/../target"
>>>> tap = dumpxml_root.elements[xpath]
>>>> if tap
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> self[:tap] = tap.attributes['dev']
>>>> end
>>>> end
>>>> self
>>>> end
>>>>
>>>> get_tap(STDIN)
>>>>
>>>>
>>>> then I could:
>>>>
>>>> >> root at loki:~# virsh dumpxml one-37 | ruby interface.rb
>>>>
>>>> This results in the following error:
>>>>
>>>> >> interface.rb:5:in `get_tap': undefined method `vm_info' for
>>>> #<IO:<STDIN>> (NoMethodError)
>>>> >> from interface.rb:18:in `<main>'
>>>>
>>>> so I think my input is not valid... I -think- vm should be some sort
>>>> of object so maybe the problem is getting from XML to an object of "vm"
>>>> type?
>>>>
>>>> Thanks for the assistance,
>>>> Jon A
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, Apr 10, 2013 at 5:01 AM, Jaime Melis <jmelis at opennebula.org>wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>>
>>>>>> I know that previous versions of opennebula required
>>>>>> openvswitch-brcompat
>>>>>>
>>>>>> >>
>>>>>> http://opennebula.org/documentation:rel3.8:openvswitch#hosts_configuration
>>>>>>
>>>>>> but the 4.0 docs seem to indicate that with KVM brcompat is not
>>>>>> required:
>>>>>>
>>>>>> >>
>>>>>> http://opennebula.org/documentation:rel4.0:openvswitch#ovswitch_and_ovswitch_brcompat
>>>>>>
>>>>>> Anyway, I did install the openvswitch-brcompat package, but got the
>>>>>> same error.
>>>>>> (looking at the docs now, it seems that you have to tell the
>>>>>> OpenNebula to use a different openvswitch driver,
>>>>>> specifically openvswitch_brcompat, so I think this test may have been
>>>>>> incomplete)
>>>>>>
>>>>>>
>>>>> Yes, brcompat is now no longer required, and we recommend that if
>>>>> you're using kvm/libvirt you shouldn't enable the brcompat module
>>>>>
>>>>>
>>>>>
>>>>>> Then I attempted to run the ovs command at the command line, when I
>>>>>> do so, I get the following error:
>>>>>>
>>>>>> >> root at loki:~# sudo /usr/bin/ovs-ofctl add-flow ovsbr0
>>>>>> in_port=,dl_src=02:00:44:47:83:44,priority=40000,actions=normal
>>>>>> >> ovs-ofctl: dl_src=02:00:44:47:83:44: bad syntax for in_port
>>>>>>
>>>>>> I think this might be the root cause of the issue? Where is
>>>>>> "in_port" defined?
>>>>>> I don't see it in the onevnet template, but that doesn't mean I
>>>>>> didn't just miss it.
>>>>>>
>>>>>
>>>>> Exactly, that's the root cause, the in_port is not being found
>>>>> correctly. I propose you do this to figure things out:
>>>>>
>>>>> 1) (in the worker node i.e. 'loki')
>>>>> disable /var/lib/one/remotes/vnm/ovswitch/post by adding exit 0 at line 2
>>>>> of that script
>>>>> 2) launch the virtual machine, it should get to running
>>>>> 3) do virsh dumpxml one-<vmid>
>>>>> 4) this bit of code should be able to extract the tap interface from
>>>>> the previous output:
>>>>>
>>>>> https://github.com/OpenNebula/one/blob/master/src/vnm_mad/remotes/OpenNebulaNic.rb#L59 (get_tap
>>>>> for the NicKVM class)
>>>>> can you try so manually extracting that bit of code and providing the
>>>>> info it needs see if it's working correctly?
>>>>> Otherwise, you can figure it out manually, we're looking for something
>>>>> like vnetX (let's call that <vnet>)
>>>>> 5) finally, do
>>>>> $ sudo ovs_ofctl dump-ports ovsbr0 <vnet>
>>>>>
>>>>>
>>>>>> But this seems less than ideal. Also, I think the incorrect syntax
>>>>>> is the root cause of the issue, if there are other issues.
>>>>>>
>>>>>
>>>>> We recommend doing it like this:
>>>>> oneadmin ALL=(ALL) NOPASSWD: /usr/bin/ovs-ofctl
>>>>>
>>>>> cheers,
>>>>> Jaime
>>>>>
>>>>> --
>>>>> Jaime Melis
>>>>> Project Engineer
>>>>> OpenNebula - The Open Source Toolkit for Cloud Computing
>>>>> www.OpenNebula.org | jmelis at opennebula.org
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Jaime Melis
>> Project Engineer
>> OpenNebula - The Open Source Toolkit for Cloud Computing
>> www.OpenNebula.org | jmelis at opennebula.org
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20130413/3fefc804/attachment-0002.htm>
More information about the Users
mailing list