Friday, December 19, 2008

Plugins - part 2: modularization

In the first post on the Plugins series, I introduced the small application used to demonstrate how one could modularize applications using a plugin architecture. The digital ink was barely dry on that post that already two people rose to the challenge and presented their solution, one using the standard method with the Zope Component Architecture, the other a modified method using grok. I will comment on these two solutions later in this series.

With apologies to the more advanced users, I have decided to proceed fairly slowly and cover many simple concepts with this series of plugins. Thus, this second post will not yet discuss plugins, but simply lay the groundwork for future posts. By the way, for those interested, and as pointed out by Lennart Regebro in his post, all the code samples that I will use can be browsed at, or retrieved from, my py-fun google code repository.

As a first step before comparing different approaches to dealing with plugins, I will take the sample application introduced in the first post and modularize it.

The core application (calculator.py) is as follows:

import re

from plugins.base import OPERATORS, init_plugins

class literal_token(object):
def __init__(self, value):
self.value = value
def nud(self):
return self.value

class end_token(object):
lbp = 0

def tokenize(program):
for number, operator in re.findall("\s*(?:(\d+)|(\*\*|.))", program):
if number:
yield literal_token(int(number))
elif operator in OPERATORS:
yield OPERATORS[operator]()
else:
raise SyntaxError("unknown operator: %r" % operator)
yield end_token()

def expression(rbp=0):
global token
t = token
token = next()
left = t.nud()
while rbp < token.lbp:
t = token
token = next()
left = t.led(left)
return left

def calculate(program):
global token, next
next = tokenize(program).next
token = next()
return expression()

if __name__ == "__main__":
init_plugins(expression)
assert calculate("+1") == 1
assert calculate("-1") == -1
assert calculate("10") == 10
assert calculate("1+2") == 3
assert calculate("1+2+3") == 6
assert calculate("1+2-3") == 0
assert calculate("1+2*3") == 7
assert calculate("1*2+3") == 5
assert calculate("6*2/3") == 4
assert calculate("2**3") == 8
assert calculate("2*2**3") == 16
print "Done!"


For the next few posts, when I demonstrate some very simple plugin approaches, this core application will remain untouched. This is one important characteristic of plugin-based application: in a well-designed application, plugin writers should not have to modify a single line of the core modules to ensure that their plugins can be used.

Communication between plugins and the core application is ensured via an Application Programming Interface (API) unique to that application. In our example, the API is a simple Python dict (OPERATORS) written in capital letters only to make it stand out.

In a sub-directory (plugins), in addition to an empty __init__.py file, we include the following three files:

1. base.py

OPERATORS = {}

def init_plugins(expression):
'''simulated plugin initializer'''
from plugins import op_1, op_2

op_1.expression = expression
op_2.expression = expression

OPERATORS['+'] = op_1.operator_add_token
OPERATORS['-'] = op_1.operator_sub_token
OPERATORS['*'] = op_1.operator_mul_token
OPERATORS['/'] = op_1.operator_div_token
OPERATORS['**'] = op_2.operator_pow_token

2. op_1.py

class operator_add_token(object):
lbp = 10
def nud(self):
return expression(100)
def led(self, left):
return left + expression(10)

class operator_sub_token(object):
lbp = 10
def nud(self):
return -expression(100)
def led(self, left):
return left - expression(10)

class operator_mul_token(object):
lbp = 20
def led(self, left):
return left * expression(20)

class operator_div_token(object):
lbp = 20
def led(self, left):
return left / expression(20)


and 3. op_2.py

class operator_pow_token(object):
lbp = 30
def led(self, left):
return left ** expression(30-1)


The last two files have been simply extracted with no modification from the original application. Instead of having 2 such files containing classes of the form operator_xxx_token, I could have included them all in one file, or split into 5 different files. The number of files is irrelevant here: they are only introduced to play the role of plugins in this application.

The file base.py plays the role here of a plugin initialization module: it ensures that plugins are properly registered and made available to the core program.

Since I wanted to change the original code as little as possible, a "wart" is present in the code as written since it was never intended to be a plugin-based application: the function expression() was accessible to all objects in the initial single-file application. It is now needed in a number of modules. The file base.py takes care of ensuring that "plugin" modules have access to that function in a transparent way. This will need to be changed when using some standard plugin frameworks, as was done in the zca example or the grok one.

In the next post, I will show how to take this now modularized application and transform it into a proper plugin-based one.

No comments: