What are multi-methods used for ? Simply put, they allow for function overloading, ie. they allow for different implementations of the same function to be provided for different contexts, and the appropriate context and therefore the implementing function to be automatically selected and performed at runtime. The typical method of function overloading is based on differing type based signatures (a carryover from the C++ / Java days). There is a multimethod module in python. That can be used for type based switching. There is another way function overloading gets used, an approach used by clojure which it terms as multi methods based on an intermediate function used to compute a value, and then switches based upon that value. Note: Switching = Coming to a fork on the road and then deciding which path to take.
For an easier reading on clojure multimethods, you may want to first read Clojure multi-methods, a nice post by Alex Miller. One more reason to read that post, is that the specific situation I use to demonstrate the usage of multi methods is also identical to the one described in that post.
Here’s one way to implement clojure style multimethods in python. However I will put the cart before the horse, and demonstrate how multi methods could be used in python, before explaining the underlying support functions I use to make that feasible.
Situation
We have a person structure with a name and an age as member attributes. We would like to print a person’s name but prefix it with Young, Adult or Senior based on the person’s age.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | # Declare the class class Person(object): """ A person. Has a name and an age""" def __init__(self,name,age): self.name = name self.age = age # Instantiate the objects jimmy = Person("Jimmy",5) alex = Person("Alex",36) edna = Person("Edna",99) # Run the multi method print name(jimmy) print name(alex) print name(edna) #Expected output is # Young Jimmy # Adult Alex # Senior Edna |
To do the same, we will actually declare a switcher function which computes a value to switch upon.
1 2 3 4 5 | def ticket(person): """ The switcher function to decide the result to be used for multi method switching """ if person.age < 16 : return "young" elif person.age >= 66 : return "senior" else : return "adult" |
We shall need a switcher which will switch based on this value.
1 2 | # Declare the existence of a multi method switcher name = multi(ticket) |
Ahh, so we now need three methods to be declared, one each for each value. These we shall do so using decorator @multi_method. Note all the three functions have the same name. Both the decorator and the support that allows the functions to have the same name comes up later.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | # Add the first switched function for result value "young" @multi_method(name, "young") def name(person): return "Young " + person.name # Add the next switched function for the result value "adult" @multi_method(name, "adult") def name(person): return "Adult " + person.name # Add yet another switched function for the result value "senior" @multi_method(name, "senior") def name(person): return "Senior " + person.name |
Now this can be made to work. But it needs a couple of constructs behind the scenes. First we need the constructor for a multi function, which accepts the switcher function (in this case ticket).
1 2 3 4 5 6 7 8 9 10 11 12 13 | def multi(switcher_func): """ Declares a multi map based method which will switch to the appropriate function based on the results of the switcher func""" def dispatcher(*args, **kwargs): key = switcher_func(*args, **kwargs) func = dispatcher.dispatch_map[key] if func : return func(*args,**kwargs) else : raise Exception("No function defined for dispatch key: %s" % key) dispatcher.dispatch_map = {} return dispatcher |
This declares an inner function dispatcher, which has the logic to perform the switch based on the supplied switcher function and the run time arguments to the function. What this method however does is that it sets up a dictionary called dispatch_map as an attribute of the dispatcher function which will be used to store the possible result values and the resultant function to switch to.
Having done that, we still need to define each of the individual functions for each possible value of the switcher function. That is done using the following decorator where the dispatch_map that got defined earlier is populated with the appropriate result and the function to dispatch to.. Note that the function name gets changed in order to avoid a conflict.
1 2 3 4 5 6 7 8 9 10 | def multi_method(dispatcher, result): """ The multi method decorator which allows one method at a time to be added to the broader switch for the given result value""" def inner(wrapped): dispatcher.dispatch_map[result] = wrapped # Change the method name from clashing with the multi and allowing # all multi methods to be written using the same name wrapped.func_name = "_" + wrapped.func_name + "_" + str(result) return dispatcher return inner |
There’s a way to use these capabilities for class members. So if I wanted to dispatch the __str__ method based on the values, the class would now be
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | class Person(object): """ A person. Has a name and an age""" def __init__(self,name,age): self.name = name self.age = age __str__ = multi(ticket) @multi_method(__str__,"young") def str_young(self): return "Young " + self.name @multi_method(__str__,"adult") def str_adult(self): return "Adult " + self.name @multi_method(__str__,"senior") def str_senior(self): return "Senior " + self.name |
Not bad for an hours work.
Update 1:
Even as I had just tweeted this codeblog post, another tweet caught my attention (possibly unrelated) which expected the switching condition for each function to be defined at each function level and not as a composite function returning some value. Since this was fresh in my mind, I thought that would be an interesting option too, and in a few minutes had another implementation based on the conditions being specified at a per function level. Here’s the separate version for the same. Pay attention to the fact that there is no ticket function, and that condition for each function is declared in the decorator itself.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | def multi(): """ Declares a multi map based method which will switch to the appropriate function based on the results of the switcher func""" def dispatcher(*args, **kwargs): for condition, func in dispatcher.functions : if condition(*args, **kwargs) : return func(*args,**kwargs) raise "No function identified for dispatch key" dispatcher.functions = [] return dispatcher def multi_method(dispatcher, condition): """ The multi method decorator which allows one method at a time to be added to the broader switch for the given result value""" def inner(wrapped): dispatcher.functions.append((condition, wrapped)) # Change the method name from clashing with the multi and allowing # all multi methods to be written using the same name wrapped.func_name = "_" + wrapped.func_name + "_" + str(condition) return dispatcher return inner class Person(object): """ A person. Has a name and an age""" def __init__(self,name,age): self.name = name self.age = age # Declare the existence of a multi method switcher name = multi() # Add the first switched function for result value "young" @multi_method(name, lambda p : p.age < 16) def name(person): return "Young " + person.name # Add the next switched function for the result value "adult" @multi_method(name, lambda p : 16 <= p.age < 66) def name(person): return "Adult " + person.name # Add yet another switched function for the result value "senior" @multi_method(name, lambda p : 66 <= p.age) def name(person): return "Senior " + person.name # Instantiate the objects jimmy = Person("Jimmy",5) alex = Person("Alex",36) edna = Person("Edna",99) # Run the multi method print name(jimmy) print name(alex) print name(edna) |
Update 2
I was requested to demonstrate this in terms of the clojure polymorphism as described in Clojure – runtime_polymorphism (code snippet from that page listed below) :
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | (defmulti encounter (fn [x y] [(:Species x) (:Species y)])) (defmethod encounter [:Bunny :Lion] [b l] :run-away) (defmethod encounter [:Lion :Bunny] [l b] :eat) (defmethod encounter [:Lion :Lion] [l1 l2] :fight) (defmethod encounter [:Bunny :Bunny] [b1 b2] :mate) (def b1 {:Species :Bunny :other :stuff}) (def b2 {:Species :Bunny :other :stuff}) (def l1 {:Species :Lion :other :stuff}) (def l2 {:Species :Lion :other :stuff}) (encounter b1 b2) -> :mate (encounter b1 l1) -> :run-away (encounter l1 b1) -> :eat (encounter l1 l2) -> :fight |
Here’s how the same effect can be had in python using the first version of multi and multi_method I described above.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | # Declare the existence of a multi method switcher encounter = multi(lambda x,y : (x["Species"], y["Species"])) @multi_method(encounter, ("Bunny","Lion")) def encounter(a1, a2): return "run-away" @multi_method(encounter, ("Lion","Bunny")) def encounter(a1, a2): return "eat" @multi_method(encounter, ("Bunny","Bunny")) def encounter(a1, a2): return "mate" @multi_method(encounter, ("Lion","Lion")) def encounter(a1, a2): return "fight" b1 = {"Species" : "Bunny", "Other" : "Stuff"} b2 = {"Species" : "Bunny", "Other" : "Stuff"} l1 = {"Species" : "Lion", "Other" : "Stuff"} l2 = {"Species" : "Lion", "Other" : "Stuff"} print encounter(b1, b2) print encounter(b1, l1) print encounter(l1, b1) print encounter(l1, l2) |
The resultant output is
1 2 3 4 | mate run-away eat fight |
I posted this on Hacker News, but thought it might be worthwhile duplicating it here since this post will likely make the rounds of the aggregators.
This is very nice, but one additional aspect of Clojure’s multimethods that is not mentioned is the role of ad-hoc hierarchies. That is, if we define a multimethod `say` as follows:
2
3
4
5
6
(defmethod say ::feline [_] (println "MEOW"))
(say {:whatiz ::feline})
; MEOW
We would expect that nothing would happen if we then call it with the following:
2
; No method in multimethod 'say' for dispatch value: :lion
But we can define a hierarchy on the fly with the following:
And now we have a whimpy lion:
2
; MEOW
This is good for simulating derived behaviors without the baggage of explicitly grouping those behaviors with the type. The multimethod dispatch **and** the hierarchical dispatch are completely open for extension. This is really cool.
Fogus,
Hmm, I would imagine that would seem very unidiomatic in python, but isn’t too hard to implement.
The following is a simplistic implementation (no n-level hierarchies or multiple children for one parent supported).
I had to modify the dispatcher just a little bit.
Base infrastructure code
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
children = {}
def derive(child,parent):
parents[child] = parent
children[parent] = child
def multi(switcher_func):
""" Declares a multi map based method which will switch to the
appropriate function based on the results of the switcher func"""
def dispatcher(*args, **kwargs):
key = switcher_func(*args, **kwargs)
func = dispatcher.dispatch_map.get(key,None)
if not func :
func = dispatcher.dispatch_map.get(parents[key], None)
if func :
return func(*args,**kwargs)
else :
raise Exception("No function defined for dispatch key: %s" % key)
dispatcher.dispatch_map = {}
return dispatcher
Usage code :
2
3
4
5
6
7
8
9
@multi_method(say,"feline")
def say(p):
return "MEOW"
derive("lion", "feline")
print say({"whatiz" : "lion"})
Output :
My sample code isn’t meant to suggest that python can match clojure for each of its capabilities in functional programming constructs. However I feel quite happy implementing even a subset of many constructs from other languages in python with relative ease as with the multi-method example in this post.
For a full generic / predicate dispatch module in Python, including inheritance management, multi-methods and much more, see PEAK Rules http://pypi.python.org/pypi/PEAK-Rules