[one-users] (RESEND) need to create Flows? for openvswitch-based ONE (4.2) setup -- (passed on ebtables)
Mark Biggers
mbiggers at ine.com
Thu Nov 21 11:01:17 PST 2013
Hello Valentin, thanks for the reply.
On 11/21/2013 03:30 AM, Valentin Bud wrote:
Hello Mark,
Before pointing you to the problem I think your config has you should also
check that you have routing enabled in the machine
Are you speaking of "ip route" routes? Or some sysconf variable? Or, route(s) on the VMs themselves??
I have attempted "ip route", to route to the VMs 10.0.0.0/24 network.... Where's 10.0.0.1 supposed be -- on the vbr0?
and if you want Internet connectivity for VMs also NAT vbr0 over eth0.
I want (1) internet connectivity to the VMs (VLANs) and connectivity "out" of the VMs network. Not sure how to get this going, though it appears the OVSwitch has all the MAC-addr info, for the VMs...
This might not be the case if your router has routes to 10/24 network through your laptop's
eth0 interface.
Currently, it does not. Just attempted, and can't ping to 10.0.0.3 VM. (output, below).
Reading once again your config I see you've inserted eth0 in the vbr0 OvS bridge and that it has an IP
address from the 192.168.1.0/24<http://192.168.1.0/24> network. I suspect that is your local network.
Yes, 192.168.1.0/24 is my "external" (laptop) network, including a Netgear router at 192.168.1.1.
Do you have connectivity between your VMs using this setup? You should from what your setup tells me.
The VMs, at 10.0.0.3 and 10.0.0.4 can ping each other, and ssh works between them just fine. They can only "see" the 10.0.0.0/24 network, and can't ping 10.0.0.1.
thank you,
----mark
root at sealion.ine.corp:~<mailto:root at sealion.ine.corp:~> # route add -net 10.0.0.0/24 gw 192.168.1.250 dev eth0
SIOCADDRT: Network is unreachable
root at sealion.ine.corp:~<mailto:root at sealion.ine.corp:~> # route add -net 10.0.0.0/24 gw 192.168.1.250 dev vbr0
root at sealion.ine.corp:~<mailto:root at sealion.ine.corp:~> # ping 10.0.0.3
PING 10.0.0.3 (10.0.0.3) 56(84) bytes of data.
>From 192.168.1.250 icmp_seq=1 Destination Host Unreachable
>From 192.168.1.250 icmp_seq=2 Destination Host Unreachable
>From 192.168.1.250 icmp_seq=3 Destination Host Unreachable
>From 192.168.1.250 icmp_seq=4 Destination Host Unreachable
^C
--- 10.0.0.3 ping statistics ---
4 packets transmitted, 0 received, +4 errors, 100% packet loss, time 2999ms
pipe 4
root at sealion.ine.corp:~<mailto:root at sealion.ine.corp:~> # netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 vbr0
10.0.0.0 192.168.1.250 255.255.255.0 UG 0 0 0 vbr0
67.139.46.149 192.168.1.1 255.255.255.255 UGH 0 0 0 vbr0
127.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 lo
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 vbr0
root at sealion.ine.corp:~<mailto:root at sealion.ine.corp:~> #
-------------------------------------------------------------------------------------------------------------------------------------------------
On Thu, Nov 21, 2013 at 12:31 AM, Mark Biggers <mbiggers at ine.com<mailto:mbiggers at ine.com>> wrote:
Hello ONE team, (RESENT -- dropped in bad "onevm show NN" data, earlier -- corrected, below!)
I have "passed" on the ebtables configuration for networking in 4.2 ONE. We'll need OpenVSwitch anyways to manage the VMs VLANS, so I have moved on.
I *think* I have an almost working OpenVSwitch configuration. Must I manually create "flows" for each VM/MAC-addr to enable IP traffic, across the OVS "vbridge" (vbr0), in this case?
No, you shouldn't have to manually add flows for IP traffic to flow between your machines and/or Internet.
The info on my new (OVS networking) setup, is included, at the end of this message. Thank you. (The platform is still openSUSE 12.3 on a Thinkpad W530...)
On 11/19/2013 05:43 AM, Jaime Melis wrote:
Hi Mark,
I have the feeling the NAT policies are interfering with this. Can you try without applying NAT rules?
On Wed, Nov 13, 2013 at 9:08 PM, Mark Biggers <mbiggers at ine.com<mailto:mbiggers at ine.com>> wrote:
The subject says it all. I am available on IRC -- see my signature, and Google chat.
I can get no "networking across a bridge" working, for the ONE "ebtables" model.
=============== edited out....
--
Jaime Melis
Project Engineer
OpenNebula - Flexible Enterprise Cloud Made Simple
www.OpenNebula.org<http://www.OpenNebula.org> | jmelis at opennebula.org<mailto:jmelis at opennebula.org>
Script started on Wed Nov 20 16:27:05 2013
root at sealion.ine.corp:one<mailto:root at sealion.ine.corp:one> # netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 vbr0
67.139.46.149 192.168.1.1 255.255.255.255 UGH 0 0 0 vbr0
127.0.0.0 0.0.0.0 255.255.255.0 U 0 0 0 lo
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 vbr0
root at sealion.ine.corp:one<mailto:root at sealion.ine.corp:one> # ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8<http://127.0.0.1/8> brd 127.255.255.255 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 3c:97:0e:ab:0a:de brd ff:ff:ff:ff:ff:ff
3: wlan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
link/ether 6c:88:14:da:0b:44 brd ff:ff:ff:ff:ff:ff
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether 0a:0e:fd:bb:5a:8a brd ff:ff:ff:ff:ff:ff
7: vbr0: <BROADCAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 3c:97:0e:ab:0a:de brd ff:ff:ff:ff:ff:ff
inet 192.168.1.250/24<http://192.168.1.250/24> scope global vbr0
12: vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
link/ether fe:00:0a:00:00:03 brd ff:ff:ff:ff:ff:ff
13: vnet1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UNKNOWN qlen 500
link/ether fe:00:0a:00:00:04 brd ff:ff:ff:ff:ff:ff
root at sealion.ine.corp:one<mailto:root at sealion.ine.corp:one> # BRIDGE_DEV=vbr0
root at sealion.ine.corp:one<mailto:root at sealion.ine.corp:one> # sudo ovs-ofctl dump-desc $BRIDGE_DEV
OFPST_DESC reply (xid=0x2):
Manufacturer: Nicira, Inc.
Hardware: Open vSwitch
Software: 1.11.0
Serial Num: None
DP Description: None
root at sealion.ine.corp:one<mailto:root at sealion.ine.corp:one> # sudo ovs-vsctl show
001119d6-32d7-4db8-8015-229b271cca6a
Bridge "vbr0"
Controller "ptcp:"
fail_mode: standalone
Port "vnet0"
tag: 0
Interface "vnet0"
Port "vnet1"
tag: 0
Interface "vnet1"
Port "eth0"
Interface "eth0"
Port "vbr0"
Interface "vbr0"
type: internal
ovs_version: "1.11.0"
Open vSwitch is a multilayer switch meaning that each VLAN has to have a Layer 3 interface with IP address from the
network you plan to use in order to connect to other networks.
Your VLAN 0 doesn't have such an interface.
Port "vbr0"
*tag: 0*
Interface "vbr0"
type: internal
Also vbr0 should have an IP address from 10/24 network, .1 to be exact so your VMs can have Internet connectivity.
Another thing I would like to point out is the use of VLAN 0. That is a special kind of VLAN, accepts and processes tagged
and un-tagged traffic. I would advise you to use another VLAN and keep the eth0 interface out of that bridge.
In that case your machine would act as a router between 10/24 and the Internet via eth0.
I don't know if it works but you could also try to add another internal interface to your bridge and assign it
10.0.0.1/24<http://10.0.0.1/24> and tag 0. Then you'd have to MASQUERADE over vbr0 interface.
The following set of commands would get your OvS networks up and running.
# ovs-vsctl del-port vbr0 eth0
# dhclient eth0 ( or set its IP address manually)
# ovs-vsctl set Port vbr0 tag=0
# ip addr add 10.0.0.1/24<http://10.0.0.1/24> dev vbr0
# iptables -A INPUT -o eth0 -j MASQUERADE
You should also have a running resolver if you want your machines to resolve DNS names because
you've passed 10.0.0.1 as DNS in the network defined inside OpenNebula.
I hop the above makes sense and helps you to some extent.
Cheers and Goodwill,
root at sealion.ine.corp:one<mailto:root at sealion.ine.corp:one> # sudo ovs-ofctl show $BRIDGE_DEV
OFPT_FEATURES_REPLY (xid=0x2): dpid:00003c970eab0ade
n_tables:254, n_buffers:256
capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
actions: OUTPUT SET_VLAN_VID SET_VLAN_PCP STRIP_VLAN SET_DL_SRC SET_DL_DST SET_NW_SRC SET_NW_DST SET_NW_TOS SET_TP_SRC SET_TP_DST ENQUEUE
1(eth0): addr:3c:97:0e:ab:0a:de
config: 0
state: STP_FORWARD
current: 1GB-FD COPPER AUTO_NEG
advertised: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
supported: 10MB-HD 10MB-FD 100MB-HD 100MB-FD 1GB-FD COPPER AUTO_NEG
speed: 1000 Mbps now, 1000 Mbps max
2(vnet0): addr:fe:00:0a:00:00:03
config: 0
state: 0
current: 10MB-FD COPPER
speed: 10 Mbps now, 0 Mbps max
3(vnet1): addr:fe:00:0a:00:00:04
config: 0
state: 0
current: 10MB-FD COPPER
speed: 10 Mbps now, 0 Mbps max
LOCAL(vbr0): addr:3c:97:0e:ab:0a:de
config: 0
state: 0
speed: 0 Mbps now, 0 Mbps max
OFPT_GET_CONFIG_REPLY (xid=0x4): frags=normal miss_send_len=0
root at sealion.ine.corp:one<mailto:root at sealion.ine.corp:one> # sudo ovs-ofctl dump-flows $BRIDGE_DEV
NXST_FLOW reply (xid=0x4):
cookie=0x0, duration=8382.092s, table=0, n_packets=4, n_bytes=240, idle_age=8381, priority=40000,in_port=2,dl_src=02:00:0a:00:00:03 actions=NORMAL
cookie=0x0, duration=6882.175s, table=0, n_packets=4, n_bytes=240, idle_age=6881, priority=40000,in_port=3,dl_src=02:00:0a:00:00:04 actions=NORMAL
cookie=0x0, duration=6882.159s, table=0, n_packets=0, n_bytes=0, idle_age=6882, priority=39000,in_port=3 actions=drop
cookie=0x0, duration=8382.072s, table=0, n_packets=0, n_bytes=0, idle_age=8382, priority=39000,in_port=2 actions=drop
cookie=0x0, duration=13024.731s, table=0, n_packets=157158, n_bytes=44610731, idle_age=0, priority=0 actions=NORMAL
root at sealion.ine.corp:one<mailto:root at sealion.ine.corp:one> # sudo -u oneadmin onehost show 7
HOST 7 INFORMATION
ID : 7
NAME : sealion.ine.corp
CLUSTER : -
STATE : MONITORED
IM_MAD : kvm
VM_MAD : kvm
VN_MAD : ovswitch
LAST MONITORING TIME : 11/20 16:39:02
HOST SHARES
TOTAL MEM : 31G
USED MEM (REAL) : 1.6G
USED MEM (ALLOCATED) : 1.5G
TOTAL CPU : 800
USED CPU (REAL) : 7
USED CPU (ALLOCATED) : 200
RUNNING VMS : 2
MONITORING INFORMATION
ARCH="x86_64"
CPUSPEED="3000"
FREECPU="792.8"
FREEMEMORY="30893828"
HOSTNAME="sealion.ine.corp"
HYPERVISOR="kvm"
MODELNAME="Intel(R) Core(TM) i7-3940XM CPU @ 3.00GHz"
NETRX="0"
NETTX="0"
TOTALCPU="800"
TOTALMEMORY="32557228"
USEDCPU="7.2000000000000455"
USEDMEMORY="1663400"
VIRTUAL MACHINES
ID USER GROUP NAME STAT UCPU UMEM HOST TIME
42 oneadmin oneadmin vyatta-router 32mrunn 0 768M sealion.in<http://sealion.in> 0d 16h48
43 oneadmin oneadmin vyatta-router-0 32mrunn 0 768M sealion.in<http://sealion.in> 0d 16h48
oneadmin at sealion:~ > onevm list
ID USER GROUP NAME STAT UCPU UMEM HOST TIME
41 oneadmin oneadmin one-vr42 stop 1 768M 6d 00h24
42 oneadmin oneadmin vyatta-router runn 0 768M sealion.in<http://sealion.in> 0d 16h48
43 oneadmin oneadmin vyatta-router-0 runn 0 768M sealion.in<http://sealion.in> 0d 16h48
oneadmin at sealion:~ > onevnet show 9
VIRTUAL NETWORK 9 INFORMATION
ID : 9
NAME : ovsnet_0_0
USER : oneadmin
GROUP : oneadmin
CLUSTER : -
TYPE : RANGED
BRIDGE : vbr0
VLAN : Yes
VLAN ID : 0
USED LEASES : 3
PERMISSIONS
OWNER : um-
GROUP : ---
OTHER : ---
VIRTUAL NETWORK TEMPLATE
DNS="10.0.0.1"
GATEWAY="10.0.0.1"
NETWORK_ADDRESS="10.0.0.0/24<http://10.0.0.0/24>"
NETWORK_MASK="255.255.255.0"
RANGE
IP_START : 10.0.0.1
IP_END : 10.0.0.254
USED LEASES
LEASE=[ MAC="02:00:0a:00:00:02", IP="10.0.0.2", IP6_LINK="fe80::400:aff:fe00:2", USED="1", VID="41" ]
LEASE=[ MAC="02:00:0a:00:00:03", IP="10.0.0.3", IP6_LINK="fe80::400:aff:fe00:3", USED="1", VID="42" ]
LEASE=[ MAC="02:00:0a:00:00:04", IP="10.0.0.4", IP6_LINK="fe80::400:aff:fe00:4", USED="1", VID="43" ]
VIRTUAL MACHINES
ID USER GROUP NAME STAT UCPU UMEM HOST TIME
41 oneadmin oneadmin one-vr42 stop 1 768M 6d 00h26
42 oneadmin oneadmin vyatta-router runn 0 768M sealion.in<http://sealion.in> 0d 16h48
43 oneadmin oneadmin vyatta-router-0 runn 0 768M sealion.in<http://sealion.in> 0d 16h48
Script done on Wed 20 Nov 2013 04:59:17 PM EST
Script started on Wed 20 Nov 2013 05:23:22 PM EST
oneadmin at sealion:~ > onevm show 42
VIRTUAL MACHINE 42 INFORMATION
ID : 42
NAME : vyatta-router
USER : oneadmin
GROUP : oneadmin
STATE : ACTIVE
LCM_STATE : RUNNING
RESCHED : No
HOST : sealion.ine.corp
START TIME : 11/14 16:55:09
END TIME : 11/15 09:43:24
DEPLOY ID : one-42
VIRTUAL MACHINE MONITORING
USED MEMORY : 768M
USED CPU : 0
NET_TX : 0K
NET_RX : 533K
PERMISSIONS
OWNER : um-
GROUP : ---
OTHER : ---
VM DISKS
ID TARGET IMAGE TYPE SAVE SAVE_AS
0 vda Vyatta Core 6.5R1 - kvm file NO -
VM NICS
ID NETWORK VLAN BRIDGE IP MAC
0 ovsnet_0_0 yes vbr0 10.0.0.3 02:00:0a:00:00:03
fe80::400:aff:fe00:3
VIRTUAL MACHINE HISTORY
SEQ HOST ACTION REAS START TIME PROLOG
0 sealion.ine.cor stop user 11/14 16:55:10 0d 00h14m 0h00m23s
1 sealion.ine.cor none erro 11/15 09:37:31 0d 00h00m 0h00m00s
2 sealion.ine.cor none erro 11/15 09:43:01 0d 00h00m 0h00m23s
3 sealion.ine.cor stop user 11/15 14:16:01 0d 03h15m 0h00m22s
4 sealion.ine.cor stop user 11/20 11:27:59 0d 02h40m 0h00m00s
5 sealion.ine.cor none none 11/20 14:08:59 0d 03h14m 0h00m00s
USER TEMPLATE
ERROR="Fri Nov 15 09:43:24 2013 : Error executing image transfer script: Error creating ISO symbolic link"
VIRTUAL MACHINE TEMPLATE
CONTEXT=[
DISK_ID="1",
HOSTNAME="MAINHOST",
IMAGE_UNAME="oneadmin",
IP_GEN="192.168.122.42",
TARGET="vdb" ]
CPU="1"
GRAPHICS=[
LISTEN="0.0.0.0",
PORT="5942",
TYPE="vnc" ]
MEMORY="768"
OS=[
ARCH="i686" ]
TEMPLATE_ID="44"
VMID="42"
oneadmin at sealion:~ > onevm show 43
VIRTUAL MACHINE 43 INFORMATION
ID : 43
NAME : vyatta-router-02
USER : oneadmin
GROUP : oneadmin
STATE : ACTIVE
LCM_STATE : RUNNING
RESCHED : No
HOST : sealion.ine.corp
START TIME : 11/14 16:55:54
END TIME : 11/15 09:43:54
DEPLOY ID : one-43
VIRTUAL MACHINE MONITORING
USED MEMORY : 768M
USED CPU : 0
NET_TX : 0K
NET_RX : 464K
PERMISSIONS
OWNER : um-
GROUP : ---
OTHER : ---
VM DISKS
ID TARGET IMAGE TYPE SAVE SAVE_AS
0 vda Vyatta Core 6.5R1 - kvm file NO -
VM NICS
ID NETWORK VLAN BRIDGE IP MAC
0 ovsnet_0_0 yes vbr0 10.0.0.4 02:00:0a:00:00:04
fe80::400:aff:fe00:4
VIRTUAL MACHINE HISTORY
SEQ HOST ACTION REAS START TIME PROLOG
0 sealion.ine.cor stop user 11/14 16:56:10 0d 00h14m 0h00m21s
1 sealion.ine.cor none erro 11/15 09:38:01 0d 00h00m 0h00m00s
2 sealion.ine.cor none erro 11/15 09:43:31 0d 00h00m 0h00m22s
3 sealion.ine.cor stop user 11/15 14:17:01 0d 03h14m 0h00m24s
4 sealion.ine.cor stop user 11/20 11:28:29 0d 02h39m 0h00m00s
5 sealion.ine.cor none none 11/20 14:33:59 0d 02h49m 0h00m00s
USER TEMPLATE
ERROR="Fri Nov 15 09:43:53 2013 : Error executing image transfer script: Error creating ISO symbolic link"
VIRTUAL MACHINE TEMPLATE
CONTEXT=[
DISK_ID="1",
HOSTNAME="MAINHOST",
IMAGE_UNAME="oneadmin",
IP_GEN="192.168.122.43",
TARGET="vdb" ]
CPU="1"
GRAPHICS=[
LISTEN="0.0.0.0",
PORT="5943",
TYPE="vnc" ]
MEMORY="768"
OS=[
ARCH="i686" ]
TEMPLATE_ID="44"
VMID="43"
oneadmin at sealion:~ > exit
exit
Script done on Wed 20 Nov 2013 05:23:33 PM EST
1
_______________________________________________
Users mailing list
Users at lists.opennebula.org<mailto:Users at lists.opennebula.org>
http://lists.opennebula.org/listinfo.cgi/users-opennebula.org
--
Valentin Bud
http://databus.pro | valentin at databus.pro<mailto:valentin at databus.pro>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opennebula.org/pipermail/users-opennebula.org/attachments/20131121/055fe7ad/attachment-0002.htm>
More information about the Users
mailing list