起因

最近在看一个SDN相关的课程，链接：http://www.csg.ethz.ch/education/lectures/ATCN/hs2014
虽然只能看课件，但感觉讲的很不错。课程还附带了几个课后任务，任务说明很详细，可以跟着一步一步做。其中一个就是基于POX控制器写一个简单的load balancer的控制程序。本文主要记录整个实验过程以及一些想法。

POX基础

  • launch: launch函数用来完成加载模块的初始化工作，比如绑定各种事件响应函数，也可以在launch函数中注册其他类。
  • Connection Up事件：当交换机和控制器建立连接时，触发该事件。
  • connection.send()：connection指控制器与某交换机之间的连接，通过send()方法，控制器可以向该交换机发送OpenFlow信息。
  • ofp_flow_mod：OpenFlow信息，向交换机中安装流表。ofp_flow_mod包含一些值得关注的域：match对象、actions列表、priority、idle timeout、hard timeout。
  • ofp_match：ofp_flow_mod中的match对象，用来匹配数据包。比如要匹配MAC地址，就需要设置match对象的dl_src和dl_dst域。
  • ofp_action_output：首先这是一个action对象，可以添加到ofp_flow_mod的actions列表中，指示将数据包输出到某个端口或者一些特殊定义的端口，比如of.OFPP_FLOOD。
  • ofp_packet_out：控制器发给交换机的OpenFlow信息，指示交换机如何发送某个数据包。

问题

实验中的网络拓扑是一个星型拓扑，一个交换机连着8个主机。其中前4个主机（h1-h4）相当于Clients，可以向作为Servers的后4个主机（h5-h8）发起请求。交换机被当作一个load balancer来平衡Clients与Servers之间的请求。Clients通过一个公共的Service IP发起请求，交换机相当于一个透明的代理，将请求分发到不同的Server端。
分发策略通过简单的随机方式实现。

需求分析

  1. 首先，交换机作为服务的透明代理，它需要知道真正的4个server分别和自己的哪些端口相连，这才能将请求正确地分发到不同的Server端。所以交换机应该主动发送ARP请求，记录Server的MAC地址，以及所在的端口。为了尽量减少请求的等待时长，交换机与控制器建立连接后，控制器应该马上指示交换机发送ARP请求。

  2. 客户端通过一个公共的IP（Service IP）来访问服务，而实际可能访问的是某个server。为了建立客户端到server的连接，当客户端发送ARP请求Service IP的MAC地址时，交换机应该使用一个伪MAC回答（这里使用0A:00:00:00:00:01）。而且这时交换机应该记录下客户端IP对应的MAC地址以及输入端口，这样当服务器响应后，它才能正确地传输给客户端。

  3. Server响应客户端前，会向客户端发送ARP请求，交换机应该以自己的伪MAC地址回答。

  4. 客户端向Service IP发起请求，如果是新出现的客户端IP地址，交换机应该将请求随机定向到某个server上。这需要将数据包的目的IP地址和目的MAC地址重写为server的IP地址和MAC地址。同时源MAC地址为客户端的MAC地址，需要改写为交换机的伪MAC地址。

  5. 服务器向客户端响应时，交换机应该将源IP地址修改为Service IP地址，将源MAC地址改为交换机的伪MAC地址。同时将目的MAC地址改为正确的客户端MAC地址。

实现

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
from pox.core import core
from pox.openflow import *
import pox.openflow.libopenflow_01 as of
from pox.lib.packet.arp import arp
from pox.lib.packet.ethernet import ethernet, ETHER_BROADCAST
from pox.lib.packet.ipv4 import ipv4
from pox.lib.addresses import EthAddr, IPAddr
log = core.getLogger()
import time
import random
class SimpleLoadBalancer(object):
    def __init__(self, service_ip, server_ips = []): #initialize
        core.openflow.addListeners(self)
        self.service_ip = service_ip
        self.server_ips = server_ips
        self.server_ip2mac = {}
        self.mac2port = {}
        self.ip2mac = {}
        self.client2server = {}
    def _handle_ConnectionUp(self, event): #new switch connection
        self.lb_mac = EthAddr("0A:00:00:00:00:01") #fake mac of load balancer
        self.connection = event.connection
        for server_ip in self.server_ips:
            self.send_proxied_arp_request(self.connection, server_ip)
    def update_lb_mapping(self, client_ip): #update load balancing mapping
        rnd = random.randint(0, len(self.server_ip2mac)-1)
        server_ip = self.server_ip2mac.keys()[rnd]
        return server_ip
    def send_proxied_arp_reply(self, packet, connection, outport, requested_mac):
        r = arp()
        r.opcode = arp.REPLY
        r.hwsrc = requested_mac
        r.hwdst = packet.src
        r.protosrc = packet.next.protodst
        r.protodst = packet.next.protosrc
        e = ethernet(type=ethernet.ARP_TYPE, src=self.lb_mac, dst=packet.src)
        e.set_payload(r)
        log.debug("Load balancer reply to %s's ARP \"%s is at %s\"" %
                  (r.protodst, r.protosrc, r.hwsrc))
        msg = of.ofp_packet_out()
        msg.data = e.pack()
        msg.actions.append(of.ofp_action_output(port=outport))
        connection.send(msg)
    def send_proxied_arp_request(self, connection, ip):
        r = arp()
        r.opcode = arp.REQUEST
        r.hwsrc = self.lb_mac
        r.hwdst = ETHER_BROADCAST
        r.protosrc = self.service_ip
        r.protodst = ip
        e = ethernet(type=ethernet.ARP_TYPE, src=self.lb_mac, dst=ETHER_BROADCAST)
        e.set_payload(r)
        msg = of.ofp_packet_out()
        msg.data = e.pack()
        msg.actions.append(of.ofp_action_output(port=of.OFPP_FLOOD))
        log.info("Load balancer ARPing for %s's MAC" % ip)
        connection.send(msg)
   def install_flow_rule_client_to_server(self, connection, outport, client_ip,
                                           server_ip, buffer_id=of.NO_BUFFER):
        match = of.ofp_match()
        match.dl_type = ethernet.IP_TYPE
        match.nw_src = client_ip
        match.nw_dst = self.service_ip
        msg = of.ofp_flow_mod()
        msg.match = match
        msg.buffer_id = buffer_id
        msg.idle_timeout = 10
        server_mac = self.server_ip2mac[server_ip]
        msg.actions.append(of.ofp_action_dl_addr.set_src(self.lb_mac))
        msg.actions.append(of.ofp_action_dl_addr.set_dst(server_mac))
        msg.actions.append(of.ofp_action_nw_addr.set_dst(server_ip))
        msg.actions.append(of.ofp_action_output(port=outport))
        log.info("Installing flow rule for client %s to server %s" % (client_ip, server_ip))
        connection.send(msg)
    def install_flow_rule_server_to_client(self, connection, outport, server_ip,
                                           client_ip, buffer_id=of.NO_BUFFER):
        match = of.ofp_match()
        match.dl_type = ethernet.IP_TYPE
        match.nw_src = server_ip
        match.nw_dst = client_ip
        msg = of.ofp_flow_mod()
        msg.match = match
        msg.buffer_id = buffer_id
        msg.idle_timeout = 10
        client_mac = self.ip2mac[client_ip]
        msg.actions.append(of.ofp_action_nw_addr.set_src(self.service_ip))
        msg.actions.append(of.ofp_action_dl_addr.set_src(self.lb_mac))
        msg.actions.append(of.ofp_action_dl_addr.set_dst(client_mac))
        msg.actions.append(of.ofp_action_output(port=outport))
        log.info("Installing flow rule for server %s to client %s" % (server_ip, client_ip))
        connection.send(msg)
    def _handle_PacketIn(self, event):
        packet = event.parsed
        connection = event.connection
        inport = event.port
        self.mac2port[packet.src] = inport
        if packet.type == packet.ARP_TYPE:
            r = packet.next
            if r.opcode == r.REQUEST:
                if r.protodst == self.service_ip:
                    log.info("%s ARPing for %s's MAC" % (str(r.protosrc), str(self.service_ip)))
                    self.send_proxied_arp_reply(packet, connection, inport, self.lb_mac)
                # server请求client 地址
                elif r.protosrc in self.server_ips and r.protodst not in self.server_ips:
                    log.info("Server %s ARPing for client %s's MAC" % (r.protosrc, r.protodst))
                    self.send_proxied_arp_request(connection, r.protodst)
                    self.send_proxied_arp_reply(packet, connection, inport, self.lb_mac)
            elif r.opcode == r.REPLY:
                # load balancer请求server ip的arp响应
                if r.protosrc in self.server_ips and r.protodst == self.service_ip:
                    log.info("Server %s reply to load balancer's ARP \"%s is at %s\"" %
                             (r.protosrc, r.protosrc, r.hwsrc))
                    self.server_ip2mac[r.protosrc] = r.hwsrc
                # client回复load balancer的arp请求
                elif r.protosrc not in self.server_ips and r.protodst == self.service_ip:
                    log.info("Client %s reply to load balancer's ARP \"%s is at %s\"" %
                             (r.protosrc, r.protosrc, r.hwsrc))
                    self.ip2mac[r.protosrc] = packet.src
        elif packet.type == ethernet.IP_TYPE:
            ip = packet.next
            if ip.dstip == self.service_ip:
                if ip.srcip not in self.server_ips:
                    # 客户端向service发起的请求
                    server_ip = self.update_lb_mapping(ip.srcip)
                    server_mac = self.server_ip2mac[server_ip]
                    outport = self.mac2port[server_mac]
                    buffer_id = event.ofp.buffer_id
                    log.debug("client to server port: %s" % outport)
                    self.install_flow_rule_client_to_server(connection, outport,
                                                                ip.srcip, server_ip, buffer_id)
            elif ip.srcip in self.server_ips and ip.dstip not in self.server_ips:
                client_mac = self.ip2mac.get(ip.dstip)
                if not client_mac:
                    self.send_proxied_arp_request(connection, ip.dstip)
                    return
                outport = self.mac2port[client_mac]
                log.debug("client to server port: %s" % outport)
                self.install_flow_rule_server_to_client(connection, outport,
                                                            ip.srcip, ip.dstip)
        else:
            log.info("Unknown Packet type: %s" % packet.type)
            return
        return
#launch application with following arguments:
#ip: public service ip, servers: ip addresses of servers (in string format)
def launch(ip, servers):
    log.info("Loading Simple Load Balancer module")
    server_ips = servers.replace(","," ").split()
    server_ips = [IPAddr(x) for x in server_ips]
    service_ip = IPAddr(ip)
    core.registerNew(SimpleLoadBalancer, service_ip, server_ips)