Preface

Twisted存在Todo List很久了, 之前在有关于与Tornado做抉择的时候, 选择了文档更加让我舒服的Tornado, 虽然也没有舒服到哪里去. 这次重写Monkey组件, 我打算用一些其他新的,或者是完全不一样的东西来完成这件事情.

首先,先对项目进行一个分析:

  • 项目是一个需要对外服务的, 所以是需要某一种统一的通信协议, 当然, 尝试使用完全不一样的东西从这个时候就开始了, 选择Google Protocol Buffers
  • 项目有至少两个自循环任务, 一个需要循环跑验证, 另一个需要循坏跑过期.无论是多线程, 多进程, 协程, 分布式或Actor, 重得玩点不一样的, 包括RabbitMQ, ZeroMQ, beanstalk等
  • 项目是需要大量发送http请求的, 如何解决请求和响应, 也是需要有解决方案的.

Why

根据上述需求, 其实可选的东西, 已经很明确了, Node, Twisted, Tornado, GO, Erlang, Scala Akka, 那我们来选一个.
Node毕竟是JS系的, 鉴于自己每次都在JS自己坑自己,还是不要轻易尝试放纵的滋味.
Tornado挺熟的, 第一备选.
GO, 是最开始的决定, 毕竟技能多, 不压身, 结果, 还没有复习完package就不想使用了, 肯定好几次这样都是缘分不到.
Akka, 非常的选择, 一度曾经都开始着手谢了, 哎, SDKMan 卡得一批, 又难得解决

所以,最后根据明主的决定, 艰难的选择了Twisted, 才不是因为自己觉得这个框架要熟悉, 乘此机会熟悉呢.

What

Twisted is an event-driven networking engine written in Python and licensed under the open source MIT license. Twisted runs on Python 2 and an ever growing subset also works with Python 3.

How

Writing Server

Your protocol handling class will usually subclass twisted.internet.protocol.Protocol. Most protocol handlers inherit either from this class or from one of its convenience children. An instance of the protocol class is instantiated per-connection, on demand, and will go away when the connection is finished. This means that persistent configuration is not saved in the Protocol.

The persistent configuration is kept in a Factory class, which usually inherits from twisted.internet.protocol.Factory. The buildProtocol method of the Factory is used to create a Protocol for each new connection.

Protocols

A Twisted protocol handles data in an asynchronous manner. The protocol responds to events as they arrive from the network and the events arrive as calls to methods on the protocol.

Here is a simple example:

1
2
3
4
5
6
from twisted.internet.protocol import Protocol

class Echo(Protocol):

    def dataReceived(self, data):
        self.transport.write(data)

This is one of the simplest protocols. It simply writes back whatever is written to it, and does not respond to all events. Here is an example of a Protocol responding to another event:

1
2
3
4
5
6
7
from twisted.internet.protocol import Protocol

class QOTD(Protocol):

    def connectionMade(self):
        self.transport.write("An apple a day keeps the doctor away\r\n")
        self.transport.loseConnection()

This protocol responds to the initial connection with a well known quote, and then terminates the connection.

The connectionMade event is usually where setup of the connection object happens, as well as any initial greetings (as in the QOTD protocol above, which is actually based on [RFC 865][1]). The connectionLost event is where tearing down of any connection-specific objects is done. Here is an example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
from twisted.internet.protocol import Protocol

class Echo(Protocol):

    def __init__(self, factory):
        self.factory = factory

    def connectionMade(self):
        self.factory.numProtocols = self.factory.numProtocols + 1
        self.transport.write(
            "Welcome! There are currently %d open connections.\n" %
            (self.factory.numProtocols,))

    def connectionLost(self, reason):
        self.factory.numProtocols = self.factory.numProtocols - 1

    def dataReceived(self, data):
        self.transport.write(data)

loseConnection() and abortConnection()

loseConnection is gratefully close connection
abortConnection is immediately close connection due to network failures, or bugs or maliciousness in the other side of the connection, data written to the transport may not be deliverable

Run

1
2
3
4
5
6
7
8
9
10
11
12
from twisted.internet.protocol import Factory
from twisted.internet.endpoints import TCP4ServerEndpoint
from twisted.internet import reactor

class QOTDFactory(Factory):
    def buildProtocol(self, addr):
        return QOTD()

# 8007 is the port you want to run under. Choose something >1024
endpoint = TCP4ServerEndpoint(reactor, 8007)
endpoint.listen(QOTDFactory())
reactor.run()

In this example, I create a protocol Factory. I want to tell this factory that its job is to build QOTD protocol instances, so I set its buildProtocol method to return instances of the QOTD class. Then, I want to listen on a TCP port, so I make a [TCP4ServerEndpoint][1] to identify the port that I want to bind to, and then pass the factory I just created to its listen method.

endpoint.listen() tells the reactor to handle connections to the endpoint’s address using a particular protocol, but the reactor needs to be running in order for it to do anything. reactor.run()starts the reactor and then waits forever for connections to arrive on the port you’ve specified. You can stop the reactor by hitting Control-C in a terminal or calling reactor.stop().

For more information on different ways you can listen for incoming connections, see [the documentation for the endpoints API][2]. For more information on using the reactor, see [the reactor overview][3].

Helper Protocols

Many protocols build upon similar lower-level abstractions.

1
2
3
4
5
6
7
8
9
10
11
from twisted.protocols.basic import LineReceiver

class Answer(LineReceiver):

    answers = {'How are you?': 'Fine', None: "I don't know what you mean"}

    def lineReceived(self, line):
        if line in self.answers:
            self.sendLine(self.answers[line])
        else:
            self.sendLine(self.answers[None])

Factories

Simpler Protocol Creation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
from twisted.internet.protocol import Factory, Protocol
from twisted.internet.endpoints import TCP4ServerEndpoint
from twisted.internet import reactor

class QOTD(Protocol):

    def connectionMade(self):
        # self.factory was set by the factory's default buildProtocol:
        self.transport.write(self.factory.quote + '\r\n')
        self.transport.loseConnection()


class QOTDFactory(Factory):

    # This will be used by the default buildProtocol to create new protocols:
    protocol = QOTD

    def __init__(self, quote=None):
        self.quote = quote or 'An apple a day keeps the doctor away'

endpoint = TCP4ServerEndpoint(reactor, 8007)
endpoint.listen(QOTDFactory("configurable quote"))
reactor.run()

Factory Startup and Shutdown

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
from twisted.internet.protocol import Factory
from twisted.protocols.basic import LineReceiver


class LoggingProtocol(LineReceiver):

    def lineReceived(self, line):
        self.factory.fp.write(line + '\n')


class LogfileFactory(Factory):

    protocol = LoggingProtocol

    def __init__(self, fileName):
        self.file = fileName

    def startFactory(self):
        self.fp = open(self.file, 'a')

    def stopFactory(self):
        self.fp.close()