seteuid0's blog
Themed by Diary.
Archive Categories Tags RSS Feed
Diary Rise
Makito's Journal.

© This is a customized copyright.
Archive Categories Tags RSS Feed
seteuid0's blog
Themed by Diary.
[ZZ]openvswitch学习笔记
label openflow   openvswitch   openvswitch  

场景： 创建一个Virtual Switch,支持VLAN，支持MAC-Learning 包含下面四个Port：

  • P1, truck port
  • P2, VLAN 20
  • P3, P4 VLAN 30

包含五个flow table:``` Table 0: Admission control.

Table 1: VLAN input processing.

Table 2: Learn source MAC and VLAN for ingress port.

Table 3: Look up learned port for destination MAC and VLAN.

Table 4: Output processing

首先创建一个bridge sudo ovs-vsctl add-br helloworld -- set bridge helloworld fail-mode=secure 然后我们查看这个bridge $ sudo ovs-vsctl show c24322e6-8453-402a-afaf-64757ef231e9 Bridge helloworld fail\_mode: secure Port helloworld Interface helloworld type: internal ovs\_version: "2.0.1" $ sudo ovs-ofctl show helloworld OFPT\_FEATURES\_REPLY (xid=0x2): dpid:00003ad44a48c646 n\_tables:254, n\_buffers:256 capabilities: FLOW\_STATS TABLE\_STATS PORT\_STATS QUEUE\_STATS ARP\_MATCH\_IP actions: OUTPUT SET\_VLAN\_VID SET\_VLAN\_PCP STRIP\_VLAN SET\_DL\_SRC SET\_DL\_DST SET\_NW\_SRC SET\_NW\_DST SET\_NW\_TOS SET\_TP\_SRC SET\_TP\_DST ENQUEUE LOCAL(helloworld): addr:3a:d4:4a:48:c6:46 config:     0 state:      0 speed: 0 Mbps now, 0 Mbps max OFPT\_GET\_CONFIG\_REPLY (xid=0x4): frags=normal miss\_send\_len=0 $ sudo ovs-ofctl dump-flows helloworld NXST\_FLOW reply (xid=0x4): 如果设为fail-secure mode，则初始情况下flow table是空的，否则会有normal $ sudo ovs-vsctl add-br helloworld1 $ sudo ovs-ofctl show helloworld1 OFPT\_FEATURES\_REPLY (xid=0x2): dpid:00008a2f1d184941 n\_tables:254, n\_buffers:256 capabilities: FLOW\_STATS TABLE\_STATS PORT\_STATS QUEUE\_STATS ARP\_MATCH\_IP actions: OUTPUT SET\_VLAN\_VID SET\_VLAN\_PCP STRIP\_VLAN SET\_DL\_SRC SET\_DL\_DST SET\_NW\_SRC SET\_NW\_DST SET\_NW\_TOS SET\_TP\_SRC SET\_TP\_DST ENQUEUE LOCAL(helloworld1): addr:8a:2f:1d:18:49:41 config:     0 state:      0 speed: 0 Mbps now, 0 Mbps max OFPT\_GET\_CONFIG\_REPLY (xid=0x4): frags=normal miss\_send\_len=0 $ sudo ovs-ofctl dump-flows helloworld1 NXST\_FLOW reply (xid=0x4): cookie=0x0, duration=31.467s, table=0, n\_packets=8, n\_bytes=648, idle\_age=21, priority=0 actions=NORMAL 接下来，创建四个veth pair sudo ip link add first\_br type veth peer name first\_if sudo ip link add second\_br type veth peer name second\_if sudo ip link add third\_br type veth peer name third\_if sudo ip link add forth\_br type veth peer name forth\_if xxx\_br将是添加到bridge上的。 我们添加四个端口port sudo ovs-vsctl add-port helloworld first\_br -- set Interface first\_br ofport\_request=1 sudo ovs-vsctl add-port helloworld second\_br -- set Interface second\_br ofport\_request=2 sudo ovs-vsctl add-port helloworld third\_br -- set Interface third\_br ofport\_request=3 sudo ovs-vsctl add-port helloworld forth\_br -- set Interface forth\_br ofport\_request=4 ofport\_request是指定端口号 新添加的port都是出于DOWN的状态 $ ip addr 22: first\_if: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether ca:d4:fd:47:a6:ce brd ff:ff:ff:ff:ff:ff 23: first\_br: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master ovs-system state DOWN group default qlen 1000 link/ether f2:ac:70:72:49:61 brd ff:ff:ff:ff:ff:ff 24: second\_if: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 76:fa:16:61:d7:0e brd ff:ff:ff:ff:ff:ff 25: second\_br: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master ovs-system state DOWN group default qlen 1000 link/ether 66:63:74:a9:0e:f2 brd ff:ff:ff:ff:ff:ff 26: third\_if: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 1e:6c:0e:6f:8c:cb brd ff:ff:ff:ff:ff:ff 27: third\_br: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master ovs-system state DOWN group default qlen 1000 link/ether 7e:4e:87:28:33:93 brd ff:ff:ff:ff:ff:ff 28: forth\_if: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 9a:0d:44:fc:6b:51 brd ff:ff:ff:ff:ff:ff 29: forth\_br: <BROADCAST,MULTICAST> mtu 1500 qdisc noop master ovs-system state DOWN group default qlen 1000 link/ether ea:01:d1:6a:2a:07 brd ff:ff:ff:ff:ff:ff 30: helloworld: <BROADCAST,UP,LOWER\_UP> mtu 1500 qdisc noqueue state UNKNOWN group default link/ether ee:04:95:bd:a3:4b brd ff:ff:ff:ff:ff:ff inet6 fe80::b42d:a4ff:fe49:ba75/64 scope link valid\_lft forever preferred\_lft forever $ sudo ovs-ofctl show helloworld OFPT\_FEATURES\_REPLY (xid=0x2): dpid:0000ee0495bda34b n\_tables:254, n\_buffers:256 capabilities: FLOW\_STATS TABLE\_STATS PORT\_STATS QUEUE\_STATS ARP\_MATCH\_IP actions: OUTPUT SET\_VLAN\_VID SET\_VLAN\_PCP STRIP\_VLAN SET\_DL\_SRC SET\_DL\_DST SET\_NW\_SRC SET\_NW\_DST SET\_NW\_TOS SET\_TP\_SRC SET\_TP\_DST ENQUEUE 1(first\_br): addr:f2:ac:70:72:49:61 config:     PORT\_DOWN state:      LINK\_DOWN current:    10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max 2(second\_br): addr:66:63:74:a9:0e:f2 config:     PORT\_DOWN state:      LINK\_DOWN current:    10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max 3(third\_br): addr:7e:4e:87:28:33:93 config:     PORT\_DOWN state:      LINK\_DOWN current:    10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max 4(forth\_br): addr:ea:01:d1:6a:2a:07 config:     PORT\_DOWN state:      LINK\_DOWN current:    10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max LOCAL(helloworld): addr:ee:04:95:bd:a3:4b config:     0 state:      0 speed: 0 Mbps now, 0 Mbps max OFPT\_GET\_CONFIG\_REPLY (xid=0x4): frags=normal miss\_send\_len=0 把它们设为UP sudo ip link set first\_if up sudo ip link set first\_br up sudo ip link set second\_br up sudo ip link set second\_if up sudo ip link set third\_if up sudo ip link set third\_br up sudo ip link set forth\_br up sudo ip link set forth\_if up 也可以用下面的命令 ovs-ofctl mod-port helloworld first\_br up $ sudo ip addr 22: first\_if: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc pfifo\_fast state UP group default qlen 1000 link/ether ca:d4:fd:47:a6:ce brd ff:ff:ff:ff:ff:ff inet6 fe80::c8d4:fdff:fe47:a6ce/64 scope link valid\_lft forever preferred\_lft forever 23: first\_br: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc pfifo\_fast master ovs-system state UP group default qlen 1000 link/ether f2:ac:70:72:49:61 brd ff:ff:ff:ff:ff:ff inet6 fe80::f0ac:70ff:fe72:4961/64 scope link valid\_lft forever preferred\_lft forever 24: second\_if: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc pfifo\_fast state UP group default qlen 1000 link/ether 76:fa:16:61:d7:0e brd ff:ff:ff:ff:ff:ff inet6 fe80::74fa:16ff:fe61:d70e/64 scope link valid\_lft forever preferred\_lft forever 25: second\_br: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc pfifo\_fast master ovs-system state UP group default qlen 1000 link/ether 66:63:74:a9:0e:f2 brd ff:ff:ff:ff:ff:ff inet6 fe80::6463:74ff:fea9:ef2/64 scope link valid\_lft forever preferred\_lft forever 26: third\_if: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc pfifo\_fast state UP group default qlen 1000 link/ether 1e:6c:0e:6f:8c:cb brd ff:ff:ff:ff:ff:ff inet6 fe80::1c6c:eff:fe6f:8ccb/64 scope link valid\_lft forever preferred\_lft forever 27: third\_br: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc pfifo\_fast master ovs-system state UP group default qlen 1000 link/ether 7e:4e:87:28:33:93 brd ff:ff:ff:ff:ff:ff inet6 fe80::7c4e:87ff:fe28:3393/64 scope link valid\_lft forever preferred\_lft forever 28: forth\_if: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc pfifo\_fast state UP group default qlen 1000 link/ether 9a:0d:44:fc:6b:51 brd ff:ff:ff:ff:ff:ff inet6 fe80::980d:44ff:fefc:6b51/64 scope link valid\_lft forever preferred\_lft forever 29: forth\_br: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc pfifo\_fast master ovs-system state UP group default qlen 1000 link/ether ea:01:d1:6a:2a:07 brd ff:ff:ff:ff:ff:ff inet6 fe80::e801:d1ff:fe6a:2a07/64 scope link valid\_lft forever preferred\_lft forever 30: helloworld: <BROADCAST,UP,LOWER\_UP> mtu 1500 qdisc noqueue state UNKNOWN group default link/ether ee:04:95:bd:a3:4b brd ff:ff:ff:ff:ff:ff inet6 fe80::b42d:a4ff:fe49:ba75/64 scope link valid\_lft forever preferred\_lft forever $ sudo ovs-ofctl show helloworld OFPT\_FEATURES\_REPLY (xid=0x2): dpid:0000ee0495bda34b n\_tables:254, n\_buffers:256 capabilities: FLOW\_STATS TABLE\_STATS PORT\_STATS QUEUE\_STATS ARP\_MATCH\_IP actions: OUTPUT SET\_VLAN\_VID SET\_VLAN\_PCP STRIP\_VLAN SET\_DL\_SRC SET\_DL\_DST SET\_NW\_SRC SET\_NW\_DST SET\_NW\_TOS SET\_TP\_SRC SET\_TP\_DST ENQUEUE 1(first\_br): addr:f2:ac:70:72:49:61 config:     0 state:      0 current:    10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max 2(second\_br): addr:66:63:74:a9:0e:f2 config:     0 state:      0 current:    10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max 3(third\_br): addr:7e:4e:87:28:33:93 config:     0 state:      0 current:    10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max 4(forth\_br): addr:ea:01:d1:6a:2a:07 config:     0 state:      0 current:    10GB-FD COPPER speed: 10000 Mbps now, 0 Mbps max LOCAL(helloworld): addr:ee:04:95:bd:a3:4b config:     0 state:      0 speed: 0 Mbps now, 0 Mbps max OFPT\_GET\_CONFIG\_REPLY (xid=0x4): frags=normal miss\_send\_len=0 实现第一个Table 0，Admission control 包进入vswitch的时候首先进入Table 0，我们在这里可以设定规则，控制那些包可以进入，那些包不可以进入。 比如，如果source address是multicast的就不允许进入。 01:00:00:00:00:00/01:00:00:00:00:00是广播地址

00:00:00:00:00:00/01:00:00:00:00:00是单播地址

这种表示形式类似CIDR

于是我们添加下面的规则：

sudo ovs-ofctl add-flow helloworld “table=0, dl_src=01:00:00:00:00:00/01:00:00:00:00:00, actions=drop”

STP的也不接受

sudo ovs-ofctl add-flow helloworld “table=0, dl_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0, actions=drop”

我们在添加最后一个flow，这个flow的priority低于default，如果上面两个不匹配，则我们进入table 1

sudo ovs-ofctl add-flow helloworld “table=0, priority=0, actions=resubmit(,1)”

我们查看一下所有的flow
```$ sudo ovs-ofctl dump-flows helloworld NXST\_FLOW reply (xid=0x4): cookie=0x0, duration=42.162s, table=0, n\_packets=0, n\_bytes=0, idle\_age=42, priority=0 actions=resubmit(,1) cookie=0x0, duration=232.121s, table=0, n\_packets=0, n\_bytes=0, idle\_age=232, dl\_src=01:00:00:00:00:00/01:00:00:00:00:00 actions=drop cookie=0x0, duration=167.636s, table=0, n\_packets=0, n\_bytes=0, idle\_age=167, dl\_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0 actions=drop 测试Table 0 有个很好的工具ovs-appctl ofproto/trace 不满足条件DROP $ sudo ovs-appctl ofproto/trace helloworld in\_port=1,dl\_dst=01:80:c2:00:00:05 Flow: metadata=0,in\_port=1,vlan\_tci=0x0000,dl\_src=00:00:00:00:00:00,dl\_dst=01:80:c2:00:00:05,dl\_type=0x0000 Rule: table=0 cookie=0 dl\_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0 OpenFlow actions=drop Final flow: unchanged Relevant fields: skb\_priority=0,in\_port=1,dl\_src=00:00:00:00:00:00/01:00:00:00:00:00,dl\_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0,dl\_type=0x0000,nw\_frag=no Datapath actions: drop 满足条件RESUBMIT $ sudo ovs-appctl ofproto/trace helloworld in\_port=1,dl\_dst=01:80:c2:00:00:10 Flow: metadata=0,in\_port=1,vlan\_tci=0x0000,dl\_src=00:00:00:00:00:00,dl\_dst=01:80:c2:00:00:10,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop No match Final flow: unchanged Relevant fields: skb\_priority=0,in\_port=1,dl\_src=00:00:00:00:00:00/01:00:00:00:00:00,dl\_dst=01:80:c2:00:00:10/ff:ff:ff:ff:ff:f0,dl\_type=0x0000,nw\_frag=no Datapath actions: drop 实现第二个Table 1：VLAN Input Processing 首先添加一个最低优先级的DROP的规则 sudo ovs-ofctl add-flow helloworld "table=1, priority=0, actions=drop" 对于port 1，是trunk口，无论有没有VLAN Header都接受。 sudo ovs-ofctl add-flow helloworld "table=1, priority=99, in\_port=1, actions=resubmit(,2)" 对于port 2, 3, 4, 我们希望没有VLAN Tag，然后我们给打上VLAN Tag $ sudo ovs-ofctl add-flows helloworld - <<'EOF' table=1, priority=99, in\_port=2, vlan\_tci=0, actions=mod\_vlan\_vid:20, resubmit(,2) table=1, priority=99, in\_port=3, vlan\_tci=0, actions=mod\_vlan\_vid:30, resubmit(,2) table=1, priority=99, in\_port=4, vlan\_tci=0, actions=mod\_vlan\_vid:30, resubmit(,2) EOF $ sudo ovs-ofctl dump-flows helloworld NXST\_FLOW reply (xid=0x4): cookie=0x0, duration=4478.582s, table=0, n\_packets=0, n\_bytes=0, idle\_age=4478, priority=0 actions=resubmit(,1) cookie=0x0, duration=4668.541s, table=0, n\_packets=0, n\_bytes=0, idle\_age=4668, dl\_src=01:00:00:00:00:00/01:00:00:00:00:00 actions=drop cookie=0x0, duration=4604.056s, table=0, n\_packets=0, n\_bytes=0, idle\_age=4604, dl\_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0 actions=drop cookie=0x0, duration=89.273s, table=1, n\_packets=0, n\_bytes=0, idle\_age=89, priority=99,in\_port=2,vlan\_tci=0x0000 actions=mod\_vlan\_vid:20,resubmit(,2) cookie=0x0, duration=89.273s, table=1, n\_packets=0, n\_bytes=0, idle\_age=89, priority=99,in\_port=4,vlan\_tci=0x0000 actions=mod\_vlan\_vid:30,resubmit(,2) cookie=0x0, duration=89.273s, table=1, n\_packets=0, n\_bytes=0, idle\_age=89, priority=99,in\_port=3,vlan\_tci=0x0000 actions=mod\_vlan\_vid:30,resubmit(,2) cookie=0x0, duration=220.318s, table=1, n\_packets=0, n\_bytes=0, idle\_age=220, priority=99,in\_port=1 actions=resubmit(,2) cookie=0x0, duration=298.739s, table=1, n\_packets=0, n\_bytes=0, idle\_age=298, priority=0 actions=drop 测试一个从port 1进入，tag为5的 $ sudo ovs-appctl ofproto/trace helloworld in\_port=1,vlan\_tci=5 Flow: metadata=0,in\_port=1,vlan\_tci=0x0005,dl\_src=00:00:00:00:00:00,dl\_dst=00:00:00:00:00:00,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=1 OpenFlow actions=resubmit(,2) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop No match Final flow: unchanged Relevant fields: skb\_priority=0,in\_port=1,dl\_src=00:00:00:00:00:00/01:00:00:00:00:00,dl\_dst=00:00:00:00:00:00/ff:ff:ff:ff:ff:f0,dl\_type=0x0000,nw\_frag=no Datapath actions: drop 测试二，从port 2进入，没有打Tag的 $ sudo ovs-appctl ofproto/trace helloworld in\_port=2 Flow: metadata=0,in\_port=2,vlan\_tci=0x0000,dl\_src=00:00:00:00:00:00,dl\_dst=00:00:00:00:00:00,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=2,vlan\_tci=0x0000 OpenFlow actions=mod\_vlan\_vid:20,resubmit(,2) Resubmitted flow: metadata=0,in\_port=2,dl\_vlan=20这里被打上了Tag,dl\_vlan\_pcp=0,dl\_src=00:00:00:00:00:00,dl\_dst=00:00:00:00:00:00,dl\_type=0x0000 Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop No match Final flow: unchanged Relevant fields: skb\_priority=0,in\_port=2,vlan\_tci=0x0000,dl\_src=00:00:00:00:00:00/01:00:00:00:00:00,dl\_dst=00:00:00:00:00:00/ff:ff:ff:ff:ff:f0,dl\_type=0x0000,nw\_frag=no Datapath actions: drop 测试三：从port进入，带Tag 5的 $ sudo ovs-appctl ofproto/trace helloworld in\_port=2,vlan\_tci=5 Flow: metadata=0,in\_port=2,vlan\_tci=0x0005,dl\_src=00:00:00:00:00:00,dl\_dst=00:00:00:00:00:00,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=0 OpenFlow actions=drop Final flow: unchanged Relevant fields: skb\_priority=0,in\_port=2,vlan\_tci=0x0005,dl\_src=00:00:00:00:00:00/01:00:00:00:00:00,dl\_dst=00:00:00:00:00:00/ff:ff:ff:ff:ff:f0,dl\_type=0x0000,nw\_frag=no Datapath actions: drop 实现第三个Table 2: MAC, VLAN learning for ingress port 对于普通的switch，都会有这个学习的过程，当一个包到来的时候，由于包里面有MAC，VLAN Tag，以及从哪个口进来的这个信息。于是switch学习后，维护了一个表格port –> MAC –> VLAN Tag。 这样以后如果有需要发给这个MAC的包，不用ARP，switch自然之道应该发给哪个port，应该打什么VLAN Tag。 OVS也要学习这个，并维护三个之间的mapping关系。 在我们的例子中，无论是从port进来的本身就带Tag的，还是从port 2, 3, 4进来的后来被打上Tag的，都需要学习。 sudo ovs-ofctl add-flow helloworld "table=2 actions=learn(table=10, NXM\_OF\_VLAN\_TCI\[0..11\], NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\], load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]), resubmit(,3)" 这一句比较难理解。 learn表示这是一个学习的action table 10，这是一个MAC learning table，学习的结果会放在这个table中。 NXM\_OF\_VLAN\_TCI这个是VLAN Tag，在MAC Learning table中，每一个entry都是仅仅对某一个VLAN来说的，不同VLAN的learning table是分开的。在学习的结果的entry中，会标出这个entry是对于哪个VLAN的。 NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\]这个的意思是当前包里面的MAC Source Address会被放在学习结果的entry里面的dl\_dst里面。这是因为每个switch都是通过Ingress包来学习，某个MAC从某个port进来，switch就应该记住以后发往这个MAC的包要从这个port出去，因而MAC source address就被放在了Mac destination address里面，因为这是为发送用的。 NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0将portf放入register. 一般对于学习的entry还需要有hard\_timeout，这是的每个学习结果都会expire，需要重新学习。 我们再来分析一个实践中，openstack中使用openvswitch的情况，这是br-tun上的规则。 **cookie=0x0, duration=802188.071s, table=10, n\_packets=4885, n\_bytes=347789, idle\_age=730, hard\_age=65534, priority=1 actions=learn(table=20,hard\_timeout=300,priority=1,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:0->NXM\_OF\_VLAN\_TCI\[\],load:NXM\_NX\_TUN\_ID\[\]->NXM\_NX\_TUN\_ID\[\],output:NXM\_OF\_IN\_PORT\[\]),output:1** cookie=0x0, duration=802187.786s, table=20, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=0 actions=resubmit(,21) **cookie=0x0, duration=802038.514s, table=20, n\_packets=1239, n\_bytes=83620, idle\_age=735, hard\_age=65534, priority=2,dl\_vlan=1,dl\_dst=fa:16:3e:7e:ab:cc actions=strip\_vlan,set\_tunnel:0x3e9,output:2** cookie=0x0, duration=802187.653s, table=21, n\_packets=17, n\_bytes=1426, idle\_age=65534, hard\_age=65534, priority=0 actions=drop cookie=0x0, duration=802055.878s, table=21, n\_packets=40, n\_bytes=1736, idle\_age=65534, hard\_age=65534, dl\_vlan=1 actions=strip\_vlan,set\_tunnel:0x3e9,output:2 这里table 10是用来学习的。table 20是learning table。如果table 20是空的，也即还没有学到什么，则会通过priority=0的规则resubmit到table 21. table 21是发送规则，将br-int上的vlan tag消除，然后打上gre tunnel的id。 上面的情况中，table 20不是空的，也即发送给dl\_dst=fa:16:3e:7e:ab:cc的包不用走默认规则，直接通过table 20就发送出去了。 table 20的规则是通过table 10学习得到的，table 10是一个接受规则。最终output 1，发送给了br-int NXM\_OF\_VLAN\_TCI\[0..11\]是记录vlan tag，所以学习结果中有dl\_vlan=1 NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\]是将mac source address记录，所以结果中有dl\_dst=fa:16:3e:7e:ab:cc load:0->NXM\_OF\_VLAN\_TCI\[\]意思是发送出去的时候，vlan tag设为0，所以结果中有actions=strip\_vlan load:NXM\_NX\_TUN\_ID\[\]->NXM\_NX\_TUN\_ID\[\]意思是发出去的时候，设置tunnul id，所以结果中有set\_tunnel:0x3e9 output:NXM\_OF\_IN\_PORT\[\]意思是发送给哪个port，由于是从port2进来的，因而结果中有output:2 测试一：从port 1来一个vlan为20的mac为50:00:00:00:00:01的包 $ sudo ovs-appctl ofproto/trace helloworld in\_port=1,vlan\_tci=20,dl\_src=50:00:00:00:00:01 -generate Flow: metadata=0,in\_port=1,vlan\_tci=0x0014,dl\_src=50:00:00:00:00:01,dl\_dst=00:00:00:00:00:00,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=1 OpenFlow actions=resubmit(,2) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=2 cookie=0 OpenFlow actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop No match Final flow: unchanged Relevant fields: skb\_priority=0,in\_port=1,vlan\_tci=0x0014/0x0fff,dl\_src=50:00:00:00:00:01,dl\_dst=00:00:00:00:00:00/ff:ff:ff:ff:ff:f0,dl\_type=0x0000,nw\_frag=no Datapath actions: drop $ sudo ovs-ofctl dump-flows helloworld NXST\_FLOW reply (xid=0x4): cookie=0x0, duration=90537.25s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=0 actions=resubmit(,1) cookie=0x0, duration=90727.209s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, dl\_src=01:00:00:00:00:00/01:00:00:00:00:00 actions=drop cookie=0x0, duration=90662.724s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, dl\_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0 actions=drop cookie=0x0, duration=86147.941s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=2,vlan\_tci=0x0000 actions=mod\_vlan\_vid:20,resubmit(,2) cookie=0x0, duration=86147.941s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=4,vlan\_tci=0x0000 actions=mod\_vlan\_vid:30,resubmit(,2) cookie=0x0, duration=86147.941s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=3,vlan\_tci=0x0000 actions=mod\_vlan\_vid:30,resubmit(,2) cookie=0x0, duration=86278.986s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=1 actions=resubmit(,2) cookie=0x0, duration=86357.407s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=0 actions=drop cookie=0x0, duration=83587.281s, table=2, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) **cookie=0x0, duration=31.258s, table=10, n\_packets=0, n\_bytes=0, idle\_age=31, vlan\_tci=0x0014/0x0fff,dl\_dst=50:00:00:00:00:01 actions=load:0x1->NXM\_NX\_REG0\[0..15\]** table 10多了一条，vlan为20，dl\_dst为50:00:00:00:00:01，发送的时候从port 1出去。 测试二：从port 2进来，被打上了vlan 20，mac为50:00:00:00:00:02 $ sudo ovs-appctl ofproto/trace helloworld in\_port=2,dl\_src=50:00:00:00:00:02 -generate Flow: metadata=0,in\_port=2,vlan\_tci=0x0000,dl\_src=50:00:00:00:00:02,dl\_dst=00:00:00:00:00:00,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=2,vlan\_tci=0x0000 OpenFlow actions=mod\_vlan\_vid:20,resubmit(,2) Resubmitted flow: metadata=0,in\_port=2,dl\_vlan=20,dl\_vlan\_pcp=0,dl\_src=50:00:00:00:00:02,dl\_dst=00:00:00:00:00:00,dl\_type=0x0000 Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=2 cookie=0 OpenFlow actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop No match Final flow: unchanged Relevant fields: skb\_priority=0,in\_port=2,vlan\_tci=0x0000,dl\_src=50:00:00:00:00:02,dl\_dst=00:00:00:00:00:00/ff:ff:ff:ff:ff:f0,dl\_type=0x0000,nw\_frag=no Datapath actions: drop $ sudo ovs-ofctl dump-flows helloworld NXST\_FLOW reply (xid=0x4): cookie=0x0, duration=90823.14s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=0 actions=resubmit(,1) cookie=0x0, duration=91013.099s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, dl\_src=01:00:00:00:00:00/01:00:00:00:00:00 actions=drop cookie=0x0, duration=90948.614s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, dl\_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0 actions=drop cookie=0x0, duration=86433.831s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=2,vlan\_tci=0x0000 actions=mod\_vlan\_vid:20,resubmit(,2) cookie=0x0, duration=86433.831s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=4,vlan\_tci=0x0000 actions=mod\_vlan\_vid:30,resubmit(,2) cookie=0x0, duration=86433.831s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=3,vlan\_tci=0x0000 actions=mod\_vlan\_vid:30,resubmit(,2) cookie=0x0, duration=86564.876s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=1 actions=resubmit(,2) cookie=0x0, duration=86643.297s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=0 actions=drop cookie=0x0, duration=83873.171s, table=2, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) **cookie=0x0, duration=4.472s, table=10, n\_packets=0, n\_bytes=0, idle\_age=4, vlan\_tci=0x0014/0x0fff,dl\_dst=50:00:00:00:00:02 actions=load:0x2->NXM\_NX\_REG0\[0..15\]** cookie=0x0, duration=317.148s, table=10, n\_packets=0, n\_bytes=0, idle\_age=317, vlan\_tci=0x0014/0x0fff,dl\_dst=50:00:00:00:00:01 actions=load:0x1->NXM\_NX\_REG0\[0..15\] 实现第四个table 3: Look Up Destination Port 在table 2中，vswtich通过进入的包，学习了vlanid –> mac –> port的映射后，对于要发送的包，可以根据学习到的table 10里面的内容，根据destination mac和vlan，来找到相应的port发送出去，而不用每次都flood sudo ovs-ofctl add-flow helloworld "table=3 priority=50 actions=resubmit(,10), resubmit(,4)" 添加这条规则，首先到table 10中查找learn table entry，如果找不到则到table 4 如果包本身就是multicast的或者broadcast的，则不用去table 10里面取查找。 sudo ovs-ofctl add-flow helloworld "table=3 priority=99 dl\_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,4)" 我们进行一项测试 $ sudo ovs-appctl ofproto/trace helloworld in\_port=1,dl\_vlan=20,dl\_src=f0:00:00:00:00:01,dl\_dst=90:00:00:00:00:01 -generate Flow: metadata=0,in\_port=1,dl\_vlan=20,dl\_vlan\_pcp=0,dl\_src=f0:00:00:00:00:01,dl\_dst=90:00:00:00:00:01,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=1 OpenFlow actions=resubmit(,2) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=2 cookie=0 OpenFlow actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=3 cookie=0 priority=50 OpenFlow actions=resubmit(,10),resubmit(,4) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop No match Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop No match Final flow: unchanged Relevant fields: skb\_priority=0,in\_port=1,vlan\_tci=0x0014/0x0fff,dl\_src=f0:00:00:00:00:01,dl\_dst=90:00:00:00:00:01,dl\_type=0x0000,nw\_frag=no Datapath actions: drop 由于目标地址f0:00:00:00:00:01没有在table 10中找到，因而到达table 4. 但是这次测试使得table 10中学习到了mac地址90:00:00:00:00:01 $ sudo ovs-ofctl dump-flows helloworld NXST\_FLOW reply (xid=0x4): cookie=0x0, duration=91588.452s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=0 actions=resubmit(,1) cookie=0x0, duration=91778.411s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, dl\_src=01:00:00:00:00:00/01:00:00:00:00:00 actions=drop cookie=0x0, duration=91713.926s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, dl\_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0 actions=drop cookie=0x0, duration=87199.143s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=2,vlan\_tci=0x0000 actions=mod\_vlan\_vid:20,resubmit(,2) cookie=0x0, duration=87199.143s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=4,vlan\_tci=0x0000 actions=mod\_vlan\_vid:30,resubmit(,2) cookie=0x0, duration=87199.143s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=3,vlan\_tci=0x0000 actions=mod\_vlan\_vid:30,resubmit(,2) cookie=0x0, duration=87330.188s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=1 actions=resubmit(,2) cookie=0x0, duration=87408.609s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=0 actions=drop cookie=0x0, duration=84638.483s, table=2, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) cookie=0x0, duration=352.841s, table=3, n\_packets=0, n\_bytes=0, idle\_age=352, priority=50 actions=resubmit(,10),resubmit(,4) cookie=0x0, duration=212.704s, table=3, n\_packets=0, n\_bytes=0, idle\_age=212, priority=99,dl\_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,4) **cookie=0x0, duration=117.364s, table=10, n\_packets=0, n\_bytes=0, idle\_age=117, vlan\_tci=0x0014/0x0fff,dl\_dst=f0:00:00:00:00:01 actions=load:0x1->NXM\_NX\_REG0\[0..15\]** cookie=0x0, duration=769.784s, table=10, n\_packets=0, n\_bytes=0, idle\_age=769, vlan\_tci=0x0014/0x0fff,dl\_dst=50:00:00:00:00:02 actions=load:0x2->NXM\_NX\_REG0\[0..15\] cookie=0x0, duration=1082.46s, table=10, n\_packets=0, n\_bytes=0, idle\_age=1082, vlan\_tci=0x0014/0x0fff,dl\_dst=50:00:00:00:00:01 actions=load:0x1->NXM\_NX\_REG0\[0..15\] 下面我们进行另一个测试 $ sudo ovs-appctl ofproto/trace helloworld in\_port=2,dl\_src=90:00:00:00:00:01,dl\_dst=f0:00:00:00:00:01 -generate Flow: metadata=0,in\_port=2,vlan\_tci=0x0000,dl\_src=90:00:00:00:00:01,dl\_dst=f0:00:00:00:00:01,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=2,vlan\_tci=0x0000 OpenFlow actions=mod\_vlan\_vid:20,resubmit(,2) Resubmitted flow: metadata=0,in\_port=2,dl\_vlan=20,dl\_vlan\_pcp=0,dl\_src=90:00:00:00:00:01,dl\_dst=f0:00:00:00:00:01,dl\_type=0x0000 Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=2 cookie=0 OpenFlow actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=3 cookie=0 priority=50 OpenFlow actions=resubmit(,10),resubmit(,4) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=10 cookie=0 vlan\_tci=0x0014/0x0fff,dl\_dst=f0:00:00:00:00:01 OpenFlow actions=load:0x1->NXM\_NX\_REG0\[0..15\] Resubmitted flow: reg0=0x1,metadata=0,in\_port=2,dl\_vlan=20,dl\_vlan\_pcp=0,dl\_src=90:00:00:00:00:01,dl\_dst=f0:00:00:00:00:01,dl\_type=0x0000 Resubmitted regs: reg0=0x1 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop No match Final flow: unchanged Relevant fields: skb\_priority=0,in\_port=2,vlan\_tci=0x0000,dl\_src=90:00:00:00:00:01,dl\_dst=f0:00:00:00:00:01,dl\_type=0x0000,nw\_frag=no Datapath actions: drop 因为刚才学习到了mac地址f0:00:00:00:00:01，所以这次在table 10中找到了这条记录，这次同时也学习到了mac地址90:00:00:00:00:01 下面我们再发送第一次的包 $ sudo ovs-appctl ofproto/trace helloworld in\_port=1,dl\_vlan=20,dl\_src=f0:00:00:00:00:01,dl\_dst=90:00:00:00:00:01 -generate Flow: metadata=0,in\_port=1,dl\_vlan=20,dl\_vlan\_pcp=0,dl\_src=f0:00:00:00:00:01,dl\_dst=90:00:00:00:00:01,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=1 OpenFlow actions=resubmit(,2) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=2 cookie=0 OpenFlow actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=3 cookie=0 priority=50 OpenFlow actions=resubmit(,10),resubmit(,4) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=10 cookie=0 vlan\_tci=0x0014/0x0fff,dl\_dst=90:00:00:00:00:01 OpenFlow actions=load:0x2->NXM\_NX\_REG0\[0..15\] Resubmitted flow: reg0=0x2,metadata=0,in\_port=1,dl\_vlan=20,dl\_vlan\_pcp=0,dl\_src=f0:00:00:00:00:01,dl\_dst=90:00:00:00:00:01,dl\_type=0x0000 Resubmitted regs: reg0=0x2 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop No match Final flow: unchanged Relevant fields: skb\_priority=0,in\_port=1,vlan\_tci=0x0014/0x0fff,dl\_src=f0:00:00:00:00:01,dl\_dst=90:00:00:00:00:01,dl\_type=0x0000,nw\_frag=no Datapath actions: drop 发现也在table 10中找到了记录 实现第五个table 4: Output Processing 这个时候，register 0中包含了output port，如果是0则说明是flood。 对于port 1来讲，是trunk port，所以携带的vlan tag就让他带着，从port 1出去。 sudo ovs-ofctl add-flow helloworld "table=4 reg0=1 actions=1" 对于port 2来讲，是vlan 20的，然而出去的时候，vlan tag会被抹掉，从port 2发出去 对于port 3， 4来讲，是vlan 30的，然而出去的时候，vlan tag会被抹掉，从port 3, 4出去 $ sudo ovs-ofctl add-flows helloworld - <<'EOF' table=4 reg0=2 actions=strip\_vlan,2 table=4 reg0=3 actions=strip\_vlan,3 table=4 reg0=4 actions=strip\_vlan,4 EOF 对于broadcast来讲，我们希望一个vlan的broadcast仅仅在这个vlan里面发送，不影响其他的vlan。 $ sudo ovs-ofctl add-flows helloworld - <<'EOF' table=4 reg0=0 priority=99 dl\_vlan=20 actions=1,strip\_vlan,2 table=4 reg0=0 priority=99 dl\_vlan=30 actions=1,strip\_vlan,3,4 table=4 reg0=0 priority=50            actions=1 EOF 所以对于register = 0的，也即是broadcast的，属于vlan 20的，则从port 1, 2出去，属于vlan 30的，则从port 1, 3, 4出去。 $ sudo ovs-ofctl dump-flows helloworld NXST\_FLOW reply (xid=0x4): cookie=0x0, duration=92909.119s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=0 actions=resubmit(,1) cookie=0x0, duration=93099.078s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, dl\_src=01:00:00:00:00:00/01:00:00:00:00:00 actions=drop cookie=0x0, duration=93034.593s, table=0, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, dl\_dst=01:80:c2:00:00:00/ff:ff:ff:ff:ff:f0 actions=drop cookie=0x0, duration=88519.81s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=2,vlan\_tci=0x0000 actions=mod\_vlan\_vid:20,resubmit(,2) cookie=0x0, duration=88519.81s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=4,vlan\_tci=0x0000 actions=mod\_vlan\_vid:30,resubmit(,2) cookie=0x0, duration=88519.81s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=3,vlan\_tci=0x0000 actions=mod\_vlan\_vid:30,resubmit(,2) cookie=0x0, duration=88650.855s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=99,in\_port=1 actions=resubmit(,2) cookie=0x0, duration=88729.276s, table=1, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, priority=0 actions=drop cookie=0x0, duration=85959.15s, table=2, n\_packets=0, n\_bytes=0, idle\_age=65534, hard\_age=65534, actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) cookie=0x0, duration=1673.508s, table=3, n\_packets=0, n\_bytes=0, idle\_age=1673, priority=50 actions=resubmit(,10),resubmit(,4) cookie=0x0, duration=1533.371s, table=3, n\_packets=0, n\_bytes=0, idle\_age=1533, priority=99,dl\_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,4) cookie=0x0, duration=332.478s, table=4, n\_packets=0, n\_bytes=0, idle\_age=332, reg0=0x3 actions=strip\_vlan,output:3 cookie=0x0, duration=228.839s, table=4, n\_packets=0, n\_bytes=0, idle\_age=228, priority=50,reg0=0x0 actions=output:1 cookie=0x0, duration=483.068s, table=4, n\_packets=0, n\_bytes=0, idle\_age=483, reg0=0x1 actions=output:1 cookie=0x0, duration=332.478s, table=4, n\_packets=0, n\_bytes=0, idle\_age=332, reg0=0x4 actions=strip\_vlan,output:4 cookie=0x0, duration=332.478s, table=4, n\_packets=0, n\_bytes=0, idle\_age=332, reg0=0x2 actions=strip\_vlan,output:2 cookie=0x0, duration=228.84s, table=4, n\_packets=0, n\_bytes=0, idle\_age=228, priority=99,reg0=0x0,dl\_vlan=30 actions=output:1,strip\_vlan,output:3,output:4 cookie=0x0, duration=228.84s, table=4, n\_packets=0, n\_bytes=0, idle\_age=228, priority=99,reg0=0x0,dl\_vlan=20 actions=output:1,strip\_vlan,output:2 cookie=0x0, duration=1438.031s, table=10, n\_packets=0, n\_bytes=0, idle\_age=1438, hard\_age=1109, vlan\_tci=0x0014/0x0fff,dl\_dst=f0:00:00:00:00:01 actions=load:0x1->NXM\_NX\_REG0\[0..15\] cookie=0x0, duration=2090.451s, table=10, n\_packets=0, n\_bytes=0, idle\_age=2090, vlan\_tci=0x0014/0x0fff,dl\_dst=50:00:00:00:00:02 actions=load:0x2->NXM\_NX\_REG0\[0..15\] cookie=0x0, duration=1258.881s, table=10, n\_packets=0, n\_bytes=0, idle\_age=1258, vlan\_tci=0x0014/0x0fff,dl\_dst=90:00:00:00:00:01 actions=load:0x2->NXM\_NX\_REG0\[0..15\] cookie=0x0, duration=2403.127s, table=10, n\_packets=0, n\_bytes=0, idle\_age=2403, vlan\_tci=0x0014/0x0fff,dl\_dst=50:00:00:00:00:01 actions=load:0x1->NXM\_NX\_REG0\[0..15\] 首先来测试一个multicast和broadcast 如果是一个port 1来的vlan 30的broadcast $ sudo ovs-appctl ofproto/trace helloworld in\_port=1,dl\_dst=ff:ff:ff:ff:ff:ff,dl\_vlan=30 Flow: metadata=0,in\_port=1,dl\_vlan=30,dl\_vlan\_pcp=0,dl\_src=00:00:00:00:00:00,dl\_dst=ff:ff:ff:ff:ff:ff,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=1 OpenFlow actions=resubmit(,2) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=2 cookie=0 OpenFlow actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=3 cookie=0 priority=99,dl\_dst=01:00:00:00:00:00/01:00:00:00:00:00 OpenFlow actions=resubmit(,4) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=4 cookie=0 priority=99,reg0=0x0,dl\_vlan=30 **OpenFlow actions=output:1,strip\_vlan,output:3,output:4 skipping output to input port** Final flow: metadata=0,in\_port=1,vlan\_tci=0x0000,dl\_src=00:00:00:00:00:00,dl\_dst=ff:ff:ff:ff:ff:ff,dl\_type=0x0000 Relevant fields: skb\_priority=0,in\_port=1,dl\_vlan=30,dl\_vlan\_pcp=0,dl\_src=00:00:00:00:00:00,dl\_dst=ff:ff:ff:ff:ff:f0/ff:ff:ff:ff:ff:f0,dl\_type=0x0000,nw\_frag=no Datapath actions: pop\_vlan,12,13 结果是port 1就不发送了，发送给了port 3, 4 $ sudo ovs-appctl ofproto/trace helloworld in\_port=3,dl\_dst=ff:ff:ff:ff:ff:ff Flow: metadata=0,in\_port=3,vlan\_tci=0x0000,dl\_src=00:00:00:00:00:00,dl\_dst=ff:ff:ff:ff:ff:ff,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=3,vlan\_tci=0x0000 OpenFlow actions=mod\_vlan\_vid:30,resubmit(,2) Resubmitted flow: metadata=0,in\_port=3,dl\_vlan=30,dl\_vlan\_pcp=0,dl\_src=00:00:00:00:00:00,dl\_dst=ff:ff:ff:ff:ff:ff,dl\_type=0x0000 Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=2 cookie=0 OpenFlow actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=3 cookie=0 priority=99,dl\_dst=01:00:00:00:00:00/01:00:00:00:00:00 OpenFlow actions=resubmit(,4) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=4 cookie=0 priority=99,reg0=0x0,dl\_vlan=30 **OpenFlow actions=output:1,strip\_vlan,output:3,output:4 skipping output to input port** Final flow: metadata=0,in\_port=3,vlan\_tci=0x0000,dl\_src=00:00:00:00:00:00,dl\_dst=ff:ff:ff:ff:ff:ff,dl\_type=0x0000 Relevant fields: skb\_priority=0,in\_port=3,vlan\_tci=0x0000,dl\_src=00:00:00:00:00:00,dl\_dst=ff:ff:ff:ff:ff:f0/ff:ff:ff:ff:ff:f0,dl\_type=0x0000,nw\_frag=no Datapath actions: push\_vlan(vid=30,pcp=0),10,pop\_vlan,13 接着我们测试mac learning $ sudo ovs-appctl ofproto/trace helloworld in\_port=1,dl\_vlan=30,dl\_src=10:00:00:00:00:01,dl\_dst=20:00:00:00:00:01 -generate Flow: metadata=0,in\_port=1,dl\_vlan=30,dl\_vlan\_pcp=0,dl\_src=10:00:00:00:00:01,dl\_dst=20:00:00:00:00:01,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=1 OpenFlow actions=resubmit(,2) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=2 cookie=0 OpenFlow actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=3 cookie=0 priority=50 OpenFlow actions=resubmit(,10),resubmit(,4) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop No match Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=4 cookie=0 priority=99,reg0=0x0,dl\_vlan=30 OpenFlow actions=output:1,strip\_vlan,output:3,output:4 skipping output to input port Final flow: metadata=0,in\_port=1,vlan\_tci=0x0000,dl\_src=10:00:00:00:00:01,dl\_dst=20:00:00:00:00:01,dl\_type=0x0000 Relevant fields: skb\_priority=0,in\_port=1,dl\_vlan=30,dl\_vlan\_pcp=0,dl\_src=10:00:00:00:00:01,dl\_dst=20:00:00:00:00:01,dl\_type=0x0000,nw\_frag=no Datapath actions: pop\_vlan,12,13 由于这两个地址没有出现过，则除了进行学习以外，广播发送给port 3，4 $ sudo ovs-appctl ofproto/trace helloworld in\_port=4,dl\_src=20:00:00:00:00:01,dl\_dst=10:00:00:00:00:01 -generate Flow: metadata=0,in\_port=4,vlan\_tci=0x0000,dl\_src=20:00:00:00:00:01,dl\_dst=10:00:00:00:00:01,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=4,vlan\_tci=0x0000 OpenFlow actions=mod\_vlan\_vid:30,resubmit(,2) Resubmitted flow: metadata=0,in\_port=4,dl\_vlan=30,dl\_vlan\_pcp=0,dl\_src=20:00:00:00:00:01,dl\_dst=10:00:00:00:00:01,dl\_type=0x0000 Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=2 cookie=0 OpenFlow actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=3 cookie=0 priority=50 OpenFlow actions=resubmit(,10),resubmit(,4) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=10 cookie=0 vlan\_tci=0x001e/0x0fff,dl\_dst=10:00:00:00:00:01 OpenFlow actions=load:0x1->NXM\_NX\_REG0\[0..15\] Resubmitted flow: reg0=0x1,metadata=0,in\_port=4,dl\_vlan=30,dl\_vlan\_pcp=0,dl\_src=20:00:00:00:00:01,dl\_dst=10:00:00:00:00:01,dl\_type=0x0000 Resubmitted regs: reg0=0x1 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=4 cookie=0 reg0=0x1 OpenFlow actions=output:1 Final flow: unchanged Relevant fields: skb\_priority=0,in\_port=4,vlan\_tci=0x0000,dl\_src=20:00:00:00:00:01,dl\_dst=10:00:00:00:00:01,dl\_type=0x0000,nw\_frag=no Datapath actions: push\_vlan(vid=30,pcp=0),10 回复的时候，由于学习过了，则仅仅从port 1发送出去。 $ sudo ovs-appctl ofproto/trace helloworld in\_port=1,dl\_vlan=30,dl\_src=10:00:00:00:00:01,dl\_dst=20:00:00:00:00:01 -generate Flow: metadata=0,in\_port=1,dl\_vlan=30,dl\_vlan\_pcp=0,dl\_src=10:00:00:00:00:01,dl\_dst=20:00:00:00:00:01,dl\_type=0x0000 Rule: table=0 cookie=0 priority=0 OpenFlow actions=resubmit(,1) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=1 cookie=0 priority=99,in\_port=1 OpenFlow actions=resubmit(,2) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=2 cookie=0 OpenFlow actions=learn(table=10,NXM\_OF\_VLAN\_TCI\[0..11\],NXM\_OF\_ETH\_DST\[\]=NXM\_OF\_ETH\_SRC\[\],load:NXM\_OF\_IN\_PORT\[\]->NXM\_NX\_REG0\[0..15\]),resubmit(,3) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=3 cookie=0 priority=50 OpenFlow actions=resubmit(,10),resubmit(,4) Resubmitted flow: unchanged Resubmitted regs: reg0=0x0 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=10 cookie=0 vlan\_tci=0x001e/0x0fff,dl\_dst=20:00:00:00:00:01 OpenFlow actions=load:0x4->NXM\_NX\_REG0\[0..15\] Resubmitted flow: reg0=0x4,metadata=0,in\_port=1,dl\_vlan=30,dl\_vlan\_pcp=0,dl\_src=10:00:00:00:00:01,dl\_dst=20:00:00:00:00:01,dl\_type=0x0000 Resubmitted regs: reg0=0x4 reg1=0x0 reg2=0x0 reg3=0x0 reg4=0x0 reg5=0x0 reg6=0x0 reg7=0x0 Resubmitted  odp: drop Rule: table=4 cookie=0 reg0=0x4 OpenFlow actions=strip\_vlan,output:4 Final flow: reg0=0x4,metadata=0,in\_port=1,vlan\_tci=0x0000,dl\_src=10:00:00:00:00:01,dl\_dst=20:00:00:00:00:01,dl\_type=0x0000 Relevant fields: skb\_priority=0,in\_port=1,dl\_vlan=30,dl\_vlan\_pcp=0,dl\_src=10:00:00:00:00:01,dl\_dst=20:00:00:00:00:01,dl\_type=0x0000,nw\_frag=no Datapath actions: pop\_vlan,13 由于在回复中进行了学习，因而发送的时候，仅仅发送port 4