bottle是一个简单的python-web服务框架，可以和其它WSGI服务组合提供web服务。它最大的特色是所有代码都在单个文件中，这样限制了项目的代码量不会爆炸，同时又仅依赖python标准库，是个不错的学习项目，我们一起来阅读一下吧。整篇文章分下面几个部分:
- 项目结构
- api设计
- run-server
- Routing
- Bottle
- request && response
- plugins
- hook
- template
- 小技巧
- 小结
项目结构
本次阅读代码版本是 0.11.1, 代码获取方法请看之前的文章[requests 源码阅读], 就不再赘述。大概浏览bottle的3000行代码后，我们可以知道它分下面几个功能部分:
| 名称 |
描述 |
| Routing |
路由部分，包括每个路由规则Route和路由规则的字典Router |
| Application |
wsgi规范约定框架应用程序，由Bottle实现 |
| Request&&Response |
http和wsgi的请求，响应实现相关 |
| Plugins |
插件，bottle的json，模板，middleware都是使用插件机制实现；包括额外的plugins目录中sqlite3和werkzeug插件 |
| Util&&Helper |
工具和帮助类 |
| Server Adapter |
各种wsgi服务的适配器，默认使用的是WSGIRefServer |
| Template Adapter |
各种模板引擎的适配器，默认使用自己实现的SimpleTemplate |
| Control |
全局常量及api等 |
api设计
首先还是从示例开始观察bottle的api，从使用入手逐步往里跟踪探索。
1
2
3
4
5
6
7
|
from bottle import route, run, template
@route('/hello/<name>')
def index(name):
return template('<b>Hello {{name}}</b>!', name=name)
run(host='localhost', port=8080)
|
示例展示了下面几个内容:
- 使用route装饰器定义一个路由规则，包装一个响应函数
- 使用template生成一个模板
- 使用run函数启动服务
查看代码可以知道，除了route装饰器API，还有下面的各种API:
1
2
3
4
5
6
7
8
9
10
|
get = make_default_app_wrapper('get')
post = make_default_app_wrapper('post')
put = make_default_app_wrapper('put')
delete = make_default_app_wrapper('delete')
error = make_default_app_wrapper('error')
mount = make_default_app_wrapper('mount')
hook = make_default_app_wrapper('hook')
install = make_default_app_wrapper('install')
uninstall = make_default_app_wrapper('uninstall')
url = make_default_app_wrapper('get_url')
|
可以使用上面的API定义路由:
1
2
3
4
5
6
7
|
@get('/login') # or @route('/login')
def login():
pass
@post('/login') # or @route('/login', method='POST')
def do_login():
pass
|
装饰器的具体实现:
1
2
3
4
5
6
7
8
9
10
11
12
|
def make_default_app_wrapper(name):
''' Return a callable that relays calls to the current default app. '''
@functools.wraps(getattr(Bottle, name))
def wrapper(*a, **ka):
return getattr(Bottle(), name)(*a, **ka) # 动态获取Bottle对象的方法
return wrapper
...
class Bottle(object)
def route(self, path=None, method='GET', callback=None, name=None,
apply=None, skip=None, **config):
pass
|
现在我们大概学习了3种API的封装方法：
- redis-py中使用命令模式封装api
- requests中使用session.request的api
- bottle使用装饰器封装路由api
对于模版渲染，也提供view, simpletal_view 等api:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
def template(*args, **kwargs):
pass
def view(tpl_name, **defaults):
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
if isinstance(result, (dict, DictMixin)):
tplvars = defaults.copy()
tplvars.update(result)
return template(tpl_name, **tplvars) # 由tempate具体实现
return result
return wrapper
return decorator
mako_view = functools.partial(view, template_adapter=MakoTemplate)
cheetah_view = functools.partial(view, template_adapter=CheetahTemplate)
jinja2_view = functools.partial(view, template_adapter=Jinja2Template)
simpletal_view = functools.partial(view, template_adapter=SimpleTALTemplate)
|
下面2种view的实现方法是等价的:
1
2
3
4
5
6
|
def hello(name='World'):
return template('hello_template', name=name)
@view('hello_template')
def hello(name='World'):
return dict(name=name)
|
run-server
run 函数定义了服务启动的模式:
1
2
3
4
5
6
7
8
9
10
11
|
def run(app=None, server='wsgiref', host='127.0.0.1', port=8080,
interval=1, reloader=False, quiet=False, plugins=None,
debug=False, **kargs):
app = Bottle() # 创建一个APP对象
for plugin in plugins or []:
app.install(plugin) # 安装所有插件
...
if isinstance(server, basestring):
server = load(server) # 根据名称动态加载服务
server = server(host=host, port=port, **kargs) # 创建服务
server.run(app) # 运行服务
|
默认的 wsgirefserver 主要使用系统模块 wsgiref ，我们以后再介绍这个模块。
1
2
3
4
5
6
|
class WSGIRefServer(ServerAdapter):
def run(self, handler): # pragma: no cover
from wsgiref.simple_server import make_server, WSGIRequestHandler
....
srv = make_server(self.host, self.port, handler, **self.options)
srv.serve_forever()
|
run函数中可以动态的使用loadClassFromName方式加载服务：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
def load(target, **namespace):
""" Import a module or fetch an object from a module.
* ``package.module`` returns `module` as a module object.
* ``pack.mod:name`` returns the module variable `name` from `pack.mod`.
* ``pack.mod:func()`` calls `pack.mod.func()` and returns the result.
The last form accepts not only function calls, but any type of
expression. Keyword arguments passed to this function are available as
local variables. Example: ``import_string('re:compile(x)', x='[a-z]')``
"""
module, target = target.split(":", 1) if ':' in target else (target, None)
if module not in sys.modules: __import__(module)
if not target: return sys.modules[module]
if target.isalnum(): return getattr(sys.modules[module], target)
package_name = module.split('.')[0]
namespace[package_name] = sys.modules[package_name]
return eval('%s.%s' % (module, target), namespace)
|
更简洁的加载模块的方式，可以用下面代码:
1
2
|
from importlib import import_module
module = import_module("module_name_str")
|
Routing
在查看Routing之前，需要先了解一下Bottle中路由的实现:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
class Bottle(object):
def __init__(self, catchall=True, autojson=True):
...
self.routes = [] # List of installed :class:`Route` instances.
self.router = Router() # Maps requests to :class:`Route` instances.
....
def add_route(self, route):
''' Add a route object, but do not change the :data:`Route.app`
attribute.'''
self.routes.append(route)
self.router.add(route.rule, route.method, route, name=route.name)
...
def route(self, path=None, method='GET', callback=None, name=None,
apply=None, skip=None, **config):
def decorator(callback):
for rule in makelist(path) or yieldroutes(callback): # 可以多个path对应一个callback
for verb in makelist(method): # 一个callback可以支持多个method
verb = verb.upper()
route = Route(self, rule, verb, callback, name=name,
plugins=plugins, skiplist=skiplist, **config)
self.add_route(route)
return callback
return decorator(callback) if callback else decorator
|
bottle中每个路由规则都会创建一个Route对象，主要包括了path，method和callback三个部分。所有的路由又使用一个routes数组和一个Router字典集中管理。Router字典用于加速路由的查找，比如一个http请求，使用routes数组查找路由的算法复杂度是O(n)/O(log(n))，使用字典router查找可以是O(1)。
Route实现比较简单，主要就是对callback的包装，特别点的地方是使用plugin包装callback， 这和django的middleware很类似。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
class Route(object):
def __init__(self, app, rule, method, callback, name=None,
plugins=None, skiplist=None, **config):
#: The path-rule string (e.g. ``/wiki/:page``).
self.rule = rule
#: The HTTP method as a string (e.g. ``GET``).
self.method = method
#: The original callback with no plugins applied. Useful for introspection.
self.callback = callback
#: The name of the route (if specified) or ``None``.
...
self.plugins = plugins or []
#: A list of plugins to not apply to this route (see :meth:`Bottle.route`).
...
def __call__(self, *a, **ka):
return self._make_callback()
def _make_callback(self):
callback = self.callback
for plugin in self.all_plugins():
# 使用plugin包装
callback = plugin.apply(callback, context)
...
return callback
|
Router的实现会复杂一些，主要包括创建，添加路由规则和查找路由规则三个部分:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
class Router(object):
...
def __init__(self, strict=False):
self.rules = {} # A {rule: Rule} mapping
self.builder = {} # A rule/name->build_info mapping
self.static = {} # Cache for static routes: {path: {method: target}}
self.dynamic = [] # Cache for dynamic routes. See _compile()
#: If true, static routes are no longer checked first.
self.strict_order = strict
self.filters = {'re': self.re_filter, 'int': self.int_filter,
'float': self.float_filter, 'path': self.path_filter}
...
def add(self, rule, method, target, name=None):
...
def match(self, environ):
...
|
Router的查找实现是http服务的核心，关系到服务的效率，需要精读。本期文章我们暂不进行介绍，只需要了解Router的功能是接受路由添加，根据请求查找路由即可。
Bottle
Bottle 的核心并不复杂，主要代码如下:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
class Bottle(object):
def __init__(self, catchall=True, autojson=True):
...
# Core plugins
self.plugins = [] # List of installed plugins.
self.hooks = HooksPlugin()
# 安装默认插件
self.install(self.hooks)
if self.config.autojson:
self.install(JSONPlugin())
self.install(TemplatePlugin())
...
def wsgi(self, environ, start_response):
out = self._cast(self._handle(environ)) # 处理请求，返回响应
start_response(response._status_line, response.headerlist)
return out
def __call__(self, environ, start_response):
''' Each instance of :class:'Bottle' is a WSGI application. '''
return self.wsgi(environ, start_response)
|
Bottle是wsgi规范的application，接受wsgi-server的call。这块涉及wsgi的实现，以后我们在flask和django中也会看到，后续我们再行了解，现在只需要记住实现的语法:
- 请求的环境environ，也可以理解为context
- 创建http响应头的方法回调用方法
- 返回二进制流
URL的响应函数_handle处理:
1
2
3
4
5
6
7
8
9
10
11
|
def _handle(self, environ):
...
environ['bottle.app'] = self
request.bind(environ) # 创建绑定到线程的request对象
response.bind() # 创建绑定到线程的response对象
route, args = self.router.match(environ) # 查找route
environ['route.handle'] = route
environ['bottle.route'] = route
environ['route.url_args'] = args
return route.call(**args) # 执行route
...
|
业务返回的数据还需要使用_cast序列化为二进制数据:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
def _cast(self, out, peek=None):
...
# Join lists of byte or unicode strings. Mixed lists are NOT supported
# 转换为二进制数据
if isinstance(out, (tuple, list))\
and isinstance(out[0], (bytes, unicode)):
out = out[0][0:0].join(out) # b'abc'[0:0] -> b''
# Encode unicode strings
if isinstance(out, unicode):
out = out.encode(response.charset)
# Byte Strings are just returned
if isinstance(out, bytes):
if 'Content-Length' not in response:
response['Content-Length'] = len(out) # 处理http头的的Content-Length
return [out]
...
|
request && response
request和response的处理，是把一些数据绑定到了线程变量上，这样可以支持多线程并发。比如 request 代码主要有:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
#: Thread-local storage for :class:`LocalRequest` and :class:`LocalResponse`
#: attributes.
_lctx = threading.local()
def local_property(name):
def fget(self):
try:
return getattr(_lctx, name)
except AttributeError:
raise RuntimeError("Request context not initialized.")
def fset(self, value): setattr(_lctx, name, value)
def fdel(self): delattr(_lctx, name)
return property(fget, fset, fdel,
'Thread-local property stored in :data:`_lctx.%s`' % name) ##
class LocalRequest(BaseRequest):
''' A thread-local subclass of :class:`BaseRequest` with a different
set of attribues for each thread. There is usually only one global
instance of this class (:data:`request`). If accessed during a
request/response cycle, this instance always refers to the *current*
request (even on a multithreaded server). '''
bind = BaseRequest.__init__
environ = local_property('request_environ')
#: A thread-safe instance of :class:`LocalRequest`. If accessed from within a
#: request callback, this instance always refers to the *current* request
#: (even on a multithreaded server).
request = LocalRequest()
|
上面的代码，如果不了解wsgiref的代码，比较难理顺， 我们暂时跳过。可以看看对象属性和线程绑定的封装方法，先看下面的代码：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
|
class P:
def __init__(self,x):
self.__set_x(x)
def __get_x(self):
return self.__x
def __set_x(self, x): # 对写入值进行额外控制
if x < 0:
self.__x = 0
elif x > 1000:
self.__x = 1000
else:
self.__x = x
x = property(__get_x, __set_x) # 使用property装饰器封装getter/setter方法
p1 = P(1001)
p1.x # 1000
p1.x = -12
p1.x # 0
|
示例来自参考链接的 Properties vs. Getters and Setters
理解property后，再看environ的实现就比较容易了：
1
2
3
4
5
|
_lctx = threading.local()
def fset(self, value): setattr(_lctx, name, value) # 设置到线程上
def fget(self): return getattr(_lctx, name) # 从线程上获取
...
environ = local_property('request_environ')
|
plugin
bottle的插件并未定义接口，这里也体现了python的鸭子模型：当看到一只鸟走起来像鸭子，游泳起来也像鸭子，叫起来也像鸭子，那么这只鸟就可以被称为鸭子。插件只需要具有setup,apply和close等方法，就可以被bottle使用。
1
2
3
4
5
6
7
8
9
10
|
def install(self, plugin):
''' Add a plugin to the list of plugins and prepare it for being
applied to all routes of this application. A plugin may be a simple
decorator or an object that implements the :class:`Plugin` API.
'''
if hasattr(plugin, 'setup'): plugin.setup(self)
if hasattr(plugin, 'apply'): # 使用apply函数装饰路由的callback
def uninstall(self, plugin):
if hasattr(plugin, 'close'): plugin.close()
|
下面是一个将response返回值序列化为JSON的插件:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
|
class JSONPlugin(object):
name = 'json'
def __init__(self, json_dumps=json_dumps):
self.json_dumps = json_dumps
def apply(self, callback, route):
dumps = self.json_dumps
def wrapper(*a, **ka):
rv = callback(*a, **ka)
if isinstance(rv, dict):
#Attempt to serialize, raises exception on failure
json_response = dumps(rv)
#Set content type only if serialization succesful
response.content_type = 'application/json'
return json_response
return rv
return wrapper
|
可以看到apply实际上是一个装饰器，对callback结果进行判断，如果返回的是一个字典，则序列化为json，并设置对应的http头。
我们再看看插件模块中的SQLitePlugin:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
|
class SQLitePlugin(object):
name = 'sqlite'
api = 2
def __init__(self, dbfile=':memory:', autocommit=True, dictrows=True,
keyword='db'):
...
def apply(self, callback, route):
...
def wrapper(*args, **kwargs):
# Connect to the database
db = sqlite3.connect(dbfile)
# This enables column access by name: row['column_name']
if dictrows: db.row_factory = sqlite3.Row
# Add the connection handle as a keyword argument.
kwargs[keyword] = db
try:
rv = callback(*args, **kwargs)
if autocommit: db.commit()
except sqlite3.IntegrityError, e:
db.rollback()
raise HTTPError(500, "Database Error", e)
finally:
db.close()
return rv
# Replace the route callback with the wrapped one.
return wrapper
|
- 查找callback的参数中是否有db关键字参数，如果有db，则注入一个sqlite的连接
- 业务完成后，自动进行db的事务操作和连接关闭
下面是一个使用sqlite持久数据的示例:
1
2
3
4
5
6
7
8
9
10
|
from bottle import route, install, template
from bottle_sqlite import SQLitePlugin
install(SQLitePlugin(dbfile='/tmp/test.db'))
@route('/show/<post_id:int>')
def show(db, post_id):
c = db.execute('SELECT title, content FROM posts WHERE id = ?', (post_id,))
row = c.fetchone()
return template('show_post', title=row['title'], text=row['content'])
|
参考这个机制，我们可以给bottle添加mysql插件，redis插件等。
hook
HooksPlugin是bottle的默认插件, 可以对请求和响应进行一些额外的处理：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
class HooksPlugin(object):
_names = 'before_request', 'after_request', 'app_reset'
def __init__(self):
self.hooks = dict((name, []) for name in self._names)
self.app = None
def add(self, name, func):
''' Attach a callback to a hook. '''
was_empty = self._empty()
self.hooks.setdefault(name, []).append(func) # 注入钩子
...
def trigger(self, name, *a, **ka):
''' Trigger a hook and return a list of results. '''
hooks = self.hooks[name]
if ka.pop('reversed', False): hooks = hooks[::-1]
return [hook(*a, **ka) for hook in hooks]
def apply(self, callback, route):
if self._empty(): return callback
def wrapper(*a, **ka):
self.trigger('before_request') # 额外处理请求
rv = callback(*a, **ka)
self.trigger('after_request', reversed=True) # 额外处理响应
return rv
return wrapper
|
hook的使用示例:
1
2
3
4
5
6
7
8
9
10
11
12
13
|
from bottle import hook, response, route
@hook('after_request')
def enable_cors():
response.headers['Access-Control-Allow-Origin'] = '*' # 资源支持跨域访问
@route('/foo')
def say_foo():
return 'foo!'
@route('/bar')
def say_bar():
return {'type': 'friendly', 'content': 'Hi!'}
|
可以看到要对request&&response进行额外处理，即可以自已定义插件，也可以直接使用hook插件的before_request和after_request钩子。
Template
TemplatePlugin也是bottle默认插件, 用来实现模版功能。主要代码：
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
|
class TemplatePlugin(object):
...
def apply(self, callback, route):
conf = route.config.get('template')
if isinstance(conf, (tuple, list)) and len(conf) == 2:
return view(conf[0], **conf[1])(callback)
...
def view(tpl_name, **defaults):
def decorator(func):
@functools.wraps(func)
def wrapper(*args, **kwargs):
result = func(*args, **kwargs)
if isinstance(result, (dict, DictMixin)):
tplvars = defaults.copy()
tplvars.update(result)
return template(tpl_name, **tplvars) # 由tempate具体实现
return result
return wrapper
return decorator
def template(*args, **kwargs):
adapter = kwargs.pop('template_adapter', SimpleTemplate)
return TEMPLATES[tplid].render(kwargs)
|
模版的实现是传统web服务的重要点。传统web服务一般前后端一体，由类似jsp等语法渲染html；新型web服务多前后端分离，后端只需输出json数据。这里模版的具体实现，我们也先跳过，以后再研究。
小技巧
从bottle代码中，我们还可以看到一些有用的小技巧。
使用 __slots__ 优化对象属性:
1
2
3
4
5
6
|
class BaseRequest(object):
...
__slots__ = ('environ')
def __init__(self, environ=None):
self.environ = {} if environ is None else environ
|
关于 __slots__ 的使用，这里 stackoverflow 有非常详细的介绍。
使用 cached_property 装饰器缓存复杂属性:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
class cached_property(object):
''' A property that is only computed once per instance and then replaces
itself with an ordinary attribute. Deleting the attribute resets the
property. '''
def __init__(self, func):
self.func = func
def __get__(self, obj, cls):
if obj is None: return self
value = obj.__dict__[self.func.__name__] = self.func(obj)
return value
class Route(object):
@cached_property
def call(self):
''' The route callback with all plugins applied. This property is
created on demand and then cached to speed up subsequent requests.'''
return self._make_callback()
def _make_callback(self): # 给callback增加插件的的操作对路由的所有对象一致，只需要执行一次
callback = self.callback
for plugin in self.all_plugins():
callback = plugin.apply(callback, context)
return callback
# route.call(**args)
|
使用 lazy_attribute 装饰器延迟属性计算:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
class lazy_attribute(object):
''' A property that caches itself to the class object. '''
def __init__(self, func):
functools.update_wrapper(self, func, updated=[])
self.getter = func
def __get__(self, obj, cls):
value = self.getter(cls)
setattr(cls, self.__name__, value)
return value
class SimpleTemplate(BaseTemplate)
@lazy_attribute # 正则的编译到时候的时候才进行处理，不用模版就不需要编译
def re_pytokens(cls):
return re.compile(r'''
(''(?!')|""(?!")|'{6}|"{6} # Empty strings (all 4 types)
|'(?:[^\\']|\\.)+?' # Single quotes (')
|"(?:[^\\"]|\\.)+?" # Double quotes (")
|'{3}(?:[^\\]|\\.|\n)+?'{3} # Triple-quoted strings (')
|"{3}(?:[^\\]|\\.|\n)+?"{3} # Triple-quoted strings (")
|\#.* # Comments
)''', re.VERBOSE)
|
使用 DictProperty 装饰器控制属性只读:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
|
class DictProperty(object):
''' Property that maps to a key in a local dict-like attribute. '''
def __init__(self, attr, key=None, read_only=False):
self.attr, self.key, self.read_only = attr, key, read_only
def __call__(self, func):
functools.update_wrapper(self, func, updated=[])
self.getter, self.key = func, self.key or func.__name__
return self
...
def __set__(self, obj, value):
if self.read_only: raise AttributeError("Read-Only property.") # 只读控制
getattr(obj, self.attr)[self.key] = value
def __delete__(self, obj):
if self.read_only: raise AttributeError("Read-Only property.")
del getattr(obj, self.attr)[self.key]
class BaseRequest(object):
@DictProperty('environ', 'bottle.request.headers', read_only=True) # 不允许修改request的headers
def headers(self):
''' A :class:`WSGIHeaderDict` that provides case-insensitive access to
HTTP request headers. '''
return WSGIHeaderDict(self.environ)
|
以上三种装饰器的需求，在日常研发中都很常见，我们可以把它加入自己的工具类。
小结
最后我们来简单小结一下bottle的源码，也就是python的wsgi-application大概是如何实现的：
- 可以注入业务自定义的路由规则及路由管理
- 适配wsgi的规范启动Application，接受并响应http请求
- 每个http请求，根据路由规则查找对应的callback，进行业务处理
- 可以使用插件机制扩展需求
参考链接