<div dir="ltr">Ok, I was able to turn up a VM "manually" based on the one xml template<div><br></div><div>The only difference is I don't have a similar stanza in mine:</div><div><br></div><div>>> <target dev='vnet0'/><div>
>> <alias name='net0'/></div><div><br></div><div style>dunno if that has anything to do with it, but the issue is decidedly not my ovs configuration,</div><div style>so I think the problem lies in the ovs post script.</div>
<div><br></div><div style>Here's my xml config file I used to define the vm.</div><div><br></div><div><div>>> <domain type='kvm' id='4'></div><div>>> <name>ubuntu-manual-test</name></div>
<div>>> <memory unit='KiB'>1048576</memory></div><div>>> <currentMemory unit='KiB'>1048576</currentMemory></div><div>>> <vcpu placement='static'>2</vcpu></div>
<div>>> <cputune></div><div>>> <shares>512</shares></div><div>>> </cputune></div><div>>> <os></div><div>>> <type arch='x86_64' machine='pc-1.2'>hvm</type></div>
<div>>> <boot dev='hd'/></div><div>>> </os></div><div>>> <features></div><div>>> <acpi/></div><div>>> </features></div><div>>> <clock offset='utc'/></div>
<div>>> <on_poweroff>destroy</on_poweroff></div><div>>> <on_reboot>restart</on_reboot></div><div>>> <on_crash>destroy</on_crash></div><div>>> <devices></div>
<div>>> <emulator>/usr/bin/kvm</emulator></div><div>>> <disk type='file' device='disk'></div><div>>> <driver name='qemu' type='raw' cache='none'/></div>
<div>>> <source file='/var/lib/libvirt/images/vm_storage/ubuntu-manual-image.qcow2'/></div><div>>> <target dev='hda' bus='ide'/></div><div>>> <alias name='ide0-0-0'/></div>
<div>>> <address type='drive' controller='0' bus='0' target='0' unit='0'/></div><div>>> </disk></div><div>>> <controller type='usb' index='0'></div>
<div>>> <alias name='usb0'/></div><div>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/></div><div>>> </controller></div>
<div>>> <controller type='ide' index='0'></div><div>>> <alias name='ide0'/></div><div>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/></div>
<div>>> </controller></div><div>>> <interface type='bridge'></div><div>>> <source bridge='ovsbr0'/></div><div>>> <virtualport type='openvswitch'></div>
<div>>> </virtualport></div><div>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/></div><div>>> </interface></div>
<div>>> <input type='mouse' bus='ps2'/></div><div>>> <graphics type='vnc' port='5940' autoport='no' listen='0.0.0.0'></div><div>>> <listen type='address' address='0.0.0.0'/></div>
<div>>> </graphics></div><div>>> <video></div><div>>> <model type='cirrus' vram='9216' heads='1'/></div><div>>> <alias name='video0'/></div>
<div>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/></div><div>>> </video></div><div>>> <memballoon model='virtio'></div>
<div>>> <alias name='balloon0'/></div><div>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/></div><div>>> </memballoon></div>
<div>>> </devices></div><div>>> <seclabel type='dynamic' model='apparmor' relabel='yes'></div><div>>> </seclabel></div><div>>> </domain></div></div>
</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Apr 11, 2013 at 1:09 PM, Jon <span dir="ltr"><<a href="mailto:three18ti@gmail.com" target="_blank">three18ti@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello Jamie,<div><br></div><div>Thanks for that... Sometimes the best place to hide the obvious answer, is in plain sight.</div>
<div><br></div><div>This appears to run correctly, however, there is still no traffic leaving the host from the VM.</div>
<div>Do you think this is related to disabling the post script for ovs or is this a problem with my configuration of ovs?</div><div><br></div><div>It looks like the vm class definition is here:</div><div>
<br></div><div>>> <a href="https://github.com/OpenNebula/one/blob/master/src/vnm_mad/remotes/OpenNebulaNetwork.rb#L45" target="_blank">https://github.com/OpenNebula/one/blob/master/src/vnm_mad/remotes/OpenNebulaNetwork.rb#L45</a></div>
<div><br></div><div>Maybe I can use that with the bit of code you indicated yesterday?</div><div><br></div><div>Thanks for all your help.</div><div>Jon A</div><div><br></div><div><br></div></div><div class="HOEnZb"><div class="h5">
<div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Apr 11, 2013 at 7:37 AM, Jaime Melis <span dir="ltr"><<a href="mailto:jmelis@opennebula.org" target="_blank">jmelis@opennebula.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Jon,<div><div><br></div><div><div>>> root@loki:~# ovs-ofctl dump-ports ovsbr0 vnet0</div>
<div>>> OFPST_PORT reply (xid=0x1): 1 ports</div><div>>> port 7: rx pkts=34, bytes=1644, drop=0, errs=0, frame=0, over=0, crc=0</div>
<div>>> tx pkts=277, bytes=16622, drop=0, errs=0, coll=0</div></div><div><br></div></div><div>the in_port we're looking for is '7' from the above snippet, so you should be running:</div><div>
<br></div><div><div>>> root@loki:~# ovs-ofctl add-flow ovsbr0 "in_port=7,dl_src=02:00:44:47:83:43,priority=40000,actions=normal"</div><div><br></div><div>cheers,<br>Jaime</div></div></div><div>
<div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Thu, Apr 11, 2013 at 1:14 AM, Jon <span dir="ltr"><<a href="mailto:three18ti@gmail.com" target="_blank">three18ti@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div>Hello Jamie,<br><br></div><div>Well, I'm able to ssh from the host to the VM, so we're making progress at least :)<br></div><div><br></div>When I add vnet0 to the in_port= I get the following error:<br>
<div><br>>> root@loki:~# ovs-ofctl add-flow ovsbr0 "in_port=vnet0,dl_src=02:00:44:47:83:43,priority=40000,actions=normal"<br>>> ovs-ofctl: vnet0: bad syntax for in_port<br><br></div><div>When I set in_pot=LOCAL, the command is accepted, but the server is unable to make outbound connections, so I don't think that's the solution:<br>
<br>>> root@loki:~# ovs-ofctl add-flow ovsbr0 "in_port=LOCAL,dl_src=02:00:44:47:83:43,priority=40000,actions=normal"<br><br>>> root@loki:~# ovs-ofctl show ovsbr0<br>>> OFPT_FEATURES_REPLY (xid=0x1): ver:0x1, dpid:000000259030761a<br>
>> n_tables:255, n_buffers:256<br>>> features: capabilities:0xc7, actions:0xfff<br>>> 1(eth0): addr:00:25:90:30:76:1a<br>>> config: 0<br>>> state: 0<br>>> current: 1GB-FD COPPER AUTO_NEG<br>
>> advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG<br>>> supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG<br>>> 2(br0): addr:22:de:1a:d5:63:74<br>>> config: 0<br>
>> state: 0<br>>> 7(vnet0): addr:fe:00:44:47:83:43<br>>> config: 0<br>>> state: 0<br>>> current: 10MB-FD COPPER<br>>> 8(vnet1): addr:fe:00:44:47:83:44<br>
>> config: 0<br>>> state: 0<br>>> current: 10MB-FD COPPER<br>>> LOCAL(ovsbr0): addr:00:25:90:30:76:1a<br>>> config: PORT_DOWN<br>>> state: LINK_DOWN<br>
>> OFPT_GET_CONFIG_REPLY (xid=0x3): frags=normal miss_send_len=0<br><br></div><div>I guess I really don't understand what the correct syntax is to create a flow in this manner, so I'm kinda spinning my wheels. (I've been creating ports with `ovs-vsctl add-port`).<br>
<br></div><div>Thanks for all your help,<br></div><div>Jon A<br></div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 10, 2013 at 4:16 PM, Jon <span dir="ltr"><<a href="mailto:three18ti@gmail.com" target="_blank">three18ti@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hello Jamie,<br><br></div>Thanks for getting back to me.<div><br><br>>> Yes, brcompat is now no longer required, and we recommend that if you're
using kvm/libvirt you shouldn't enable the brcompat module<br><br></div></div>Awesome, thanks for clarifying that for me.<div><br><br>>> We recommend doing it like this:
<div>>> oneadmin ALL=(ALL) NOPASSWD: /usr/bin/ovs-ofctl<br><br></div></div><div>Gotcha, so ovs-ofctl is the only command that oneadmin needs to use.<br></div>That makes perfect sense, thanks.<br><br></div>Ok, I'm reporting your steps out of order because I think the pertinent information is this first bit.<br>
<div><br>--5) finally, do $ sudo ovs_ofctl dump-ports ovsbr0 <vnet><br><div><br>Here we go:<br><br>>> root@loki:~# ovs-ofctl dump-ports ovsbr0 vnet0<br>>> OFPST_PORT reply (xid=0x1): 1 ports<br>>> port 7: rx pkts=34, bytes=1644, drop=0, errs=0, frame=0, over=0, crc=0<br>
>> tx pkts=277, bytes=16622, drop=0, errs=0, coll=0<br><br><br>--3) do virsh dumpxml one-<vmid><br><br></div><div>Here's the relevant stanza:<br><br>>> <interface type='bridge'><br>
>> <mac address='02:00:44:47:83:43'/><div><br>>> <source bridge='ovsbr0'/><br>>> <virtualport type='openvswitch'><br></div>>> <parameters interfaceid='79c73248-4ebf-cc14-9f98-98be2d2a785b'/><br>
>> </virtualport><br>>> <target dev='vnet0'/><br>>> <alias name='net0'/><br>>> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/><br>
>> </interface><br><br>-- 4) this bit of code should be able to extract the tap interface from the previous output:<br></div><div>Honestly, my ruby is a bit rusty, what do I have to provide as "vm"? is that just the virsh xml config?<br>
<br></div><div>maybe something along the lines of:<br><pre>#!/usr/bin/ruby<br><br>def get_tap(vm)<br> dumpxml = vm.vm_info[:dumpxml]<br> if dumpxml<br> dumpxml_root = REXML::Document.new(dumpxml).root<br>
xpath = "devices/interface[@type='bridge']/"<br> xpath << "mac[@address='#{self[:mac]}']/../target"<br> tap = dumpxml_root.elements[xpath]<br> if tap<br>
self[:tap] = tap.attributes['dev']<br> end<br> end<br> self<br>end<br><br>get_tap(STDIN)<br></pre><br></div><div>then I could:<br><br>>> root@loki:~# virsh dumpxml one-37 | ruby interface.rb <br>
<br></div><div>This results in the following error:<br><br>>> interface.rb:5:in `get_tap': undefined method `vm_info' for #<IO:<STDIN>> (NoMethodError)<br>>> from interface.rb:18:in `<main>'<br>
<br></div><div>so I think my input is not valid... I -think- vm should be some sort of object so maybe the problem is getting from XML to an object of "vm" type?<br><br></div><div>Thanks for the assistance,<br>
</div><div>Jon A<br></div><br><div><br></div><br></div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Apr 10, 2013 at 5:01 AM, Jaime Melis <span dir="ltr"><<a href="mailto:jmelis@opennebula.org" target="_blank">jmelis@opennebula.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div> <div class="gmail_extra"><div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div>I know that previous versions of opennebula required openvswitch-brcompat </div><div><br></div><div>>> <a href="http://opennebula.org/documentation:rel3.8:openvswitch#hosts_configuration" target="_blank">http://opennebula.org/documentation:rel3.8:openvswitch#hosts_configuration</a></div>
<div><br></div><div>but the 4.0 docs seem to indicate that with KVM brcompat is not required:</div><div><br></div><div>>> <a href="http://opennebula.org/documentation:rel4.0:openvswitch#ovswitch_and_ovswitch_brcompat" target="_blank">http://opennebula.org/documentation:rel4.0:openvswitch#ovswitch_and_ovswitch_brcompat</a></div>
<div><br></div><div>Anyway, I did install the openvswitch-brcompat package, but got the same error.</div><div>(looking at the docs now, it seems that you have to tell the OpenNebula to use a different openvswitch driver, </div>
<div>specifically openvswitch_brcompat, so I think this test may have been incomplete)</div><div><br></div></div></blockquote><div><br></div></div><div>Yes, brcompat is now no longer required, and we recommend that if you're using kvm/libvirt you shouldn't enable the brcompat module</div>
<div>
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>
</div><div>Then I attempted to run the ovs command at the command line, when I do so, I get the following error:</div>
<div><br></div><div>>> root@loki:~# sudo /usr/bin/ovs-ofctl add-flow ovsbr0 in_port=,dl_src=02:00:44:47:83:44,priority=40000,actions=normal</div><div>>> ovs-ofctl: dl_src=02:00:44:47:83:44: bad syntax for in_port</div>
<div><br></div><div>I think this might be the root cause of the issue? Where is "in_port" defined? </div><div>I don't see it in the onevnet template, but that doesn't mean I didn't just miss it.</div>
</div></blockquote><div><br></div></div><div>Exactly, that's the root cause, the in_port is not being found correctly. I propose you do this to figure things out:</div><div><br></div><div>1) (in the worker node i.e. 'loki') disable /var/lib/one/remotes/vnm/ovswitch/post by adding exit 0 at line 2 of that script</div>
<div>2) launch the virtual machine, it should get to running</div><div>3) do virsh dumpxml one-<vmid></div><div>4) this bit of code should be able to extract the tap interface from the previous output:</div>
<div><a href="https://github.com/OpenNebula/one/blob/master/src/vnm_mad/remotes/OpenNebulaNic.rb#L59" target="_blank">https://github.com/OpenNebula/one/blob/master/src/vnm_mad/remotes/OpenNebulaNic.rb#L59</a> (get_tap for the NicKVM class)<br>
</div><div>can you try so manually extracting that bit of code and providing the info it needs see if it's working correctly?</div><div>Otherwise, you can figure it out manually, we're looking for something like vnetX (let's call that <vnet>)</div>
<div>5) finally, do</div><div>$ sudo ovs_ofctl dump-ports ovsbr0 <vnet></div><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div>But this seems less than ideal. Also, I think the incorrect syntax is the root cause of the issue, if there are other issues.</div></div></blockquote><div><br></div></div><div>We recommend doing it like this:</div>
<div>oneadmin ALL=(ALL) NOPASSWD: /usr/bin/ovs-ofctl</div><div><br></div><div>cheers,<br>Jaime</div></div><div><div><br></div>-- <br>Jaime Melis<br>Project Engineer<br>OpenNebula - The Open Source Toolkit for Cloud Computing<br>
<a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:jmelis@opennebula.org" target="_blank">jmelis@opennebula.org</a>
</div></div></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Jaime Melis<br>Project Engineer<br>OpenNebula - The Open Source Toolkit for Cloud Computing<br><a href="http://www.OpenNebula.org" target="_blank">www.OpenNebula.org</a> | <a href="mailto:jmelis@opennebula.org" target="_blank">jmelis@opennebula.org</a>
</div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>