Monday, March 14, 2022

Implementing precursor in Python

 
In Part 1 of this series I explained why the Python interpretation of super() is flawed and how it limits the user of super() to carefully controlled situations.  In Part 2, I discussed the theoretical principles behind super(), and how it can be implemented in Python based on these principles.

The sample implementation uses inspection to find out the required information about the caller of the precursor(), precursor_of(), and static_super() functions.  This is inefficient, but is easily implemented using available Python features.  A better implementation, like that of the existing super(), would be done in the Python compiler.

 This post explains the sample implementation; if you are familiar with the Python compiler, I would like to encourage you to  create an implementation that is as fast as super().

We start with the best version (the one that best fits the theoretical principles), which is the precursor, in https://github.com/yishaif/parseltongue/blob/main/extensions/super/precursor.py.

Assuming that the precursor call was made according to the rules, from a method of some class, where the method overrides an implementation in one or more superclasses, we need to find which is the class in which the method that called precursor was implemented.  This is complicated by the fact that the method implementation may be overridden in one or more subclasses, so we can't access the method implementation from the target object (that would give us the method implementation that is first on the MRO list).  The only way to get this information (without delving into the compiler) is to use runtime inspection of the call stack.

This is done by the function find_static_caller.  It first inspects the calling frame, which it gets as an argument.  (This is computed in the function precursor, using the expression inspect.getouterframes(inspect.currentframe())[1].)  The calling frame is the one corresponding to the calling method.  From that frame we extract the compiled code, the method name, and the actual arguments passed to the method.  As a sanity check, we assert that there is at least one argument, which would be the target of the call (usually, but not necessarily, called self).  The most tricky part is finding the particular method implementation that called precursor.  This is done by looping over the MRO to find an implementation of the same method (by name) whose compiled code is identical to the code object we extracted from the calling frame.  If we find it, we return the method target (as a convenience), and, more importantly, the calling class.  If we don't find the calling class, we fail with an exception.

The precursor function uses this information to find the superclass implementation that needs to be called.  It does this by getting the list of base classes of the calling class, verifying that there is only one base class, and calling the method implementation in that superclass.  The method may or may not have been defined in that class; if it hasn't, the method implementation called with be the same one that would be called on an object of that base class.

If there are multiple base classes, precursor_of must be called instead of precursor, to provide an explicit superclass whose implementation is to be called.  The details are similar to that of precursor, except that precursor_of checks that its argument is indeed a class that is a superclass of the calling class, and returns a function that calls the method implementation in the specified class.

As I said in Part 2 of this series, it is wrong to call a different method from a middle class in the hierarchy; only the implementation belonging to the actual class of the target object can be responsible for maintaining the method contract.  However, if you really want to do that (at your own risk!), you can use the implementation of static_super in https://github.com/yishaif/parseltongue/blob/main/extensions/super/static_super.py.

This implementation is similar to that of precursor However, instead of directly calling the precursor method, it needs to return an object on which you can call the method of your choice.  This object will belong to the SuperCaller class.  This class is initialized by with the target object and the superclass containing the implementation of the method to be called.  When a method is invoked on that object, the __getattr__ method will be called to find the implementation, since this class has no regular methods.  This method attempts to find such an implementation in the given superclass (using the expression getattr(self.cls, method)), and, if successful, returns a function that will invoke this implementation with the given target object and any other arguments it is given (this uses the convenience function functools.partial).