Callbacks and Scoping with Python and Javascript by Julian Ceipek
Why Should I Care?
GUIs (Graphical User Interfaces) are pretty cool. They make applications look shiny, they let ordinary mortals play with our code, and (depending on who you ask) they're a lot more fun than command line programs.
Here's where the trouble comes in. When you're writing a GUI application, you need to provide the user with choices:
How might we go about creating this interactivity? Well, we might imagine having a function
button_clicked(the_button) that does something based on which button is clicked:
# Make Some Buttons
...
def button_clicked(the_button):
if the_button == "Open Map":
# Do Lots Of Stuff
elif the_button == "Refresh My Location":
# Do Some Other Stuff
elif the_button == "Correct My Location":
# And Even More Stuff
...
def event_loop():
# When a button is clicked, call button_clicked with the appropriate string
...
This might work well enough for simple programs with a few buttons, but as you can probably tell, this can get long and ugly very quickly, especially when you are working on a project collaboratively. Most GUI frameworks provide a neater, more event-based way of dealing with user actions with callbacks.
Callbacks Responding to Events
The premise of a callback is pretty easy to understand. When some event happens, perform some action by calling a function. To get this to work, we can register functions with events.
In PySide, a pythonic wrapper for the modern GUI framework QT, the code we wrote above can be rewritten in the following way:
class DemoWindow(QtGui.QDialog):
def create_buttons(self):
self.open_button = QtGui.QAction("Open Map", self, triggered = self.open_webapp)
self.refresh_button = QtGui.QAction("Refresh My Location", self, triggered=self.initiate_location_refresh)
self.correct_location_button = QtGui.QAction("Correct My Location", self, triggered=self.correct_location)
...
def open_webapp(self):
# Do Lots Of Stuff
def initiate_location_refresh(self):
# Do Some Other Stuff
def correct_location(self):
# And Even More Stuff
...
Without going too far into the details of how PySide works,
the triggered keyword argument for the QtGui.QAction constructor specifies the function
that will be executed when the button is clicked. For example, line 3 in the code example above sets the behavior of
the "Open Map" button to call the open_webapp function.
a_function and
a_function() in languages like Python and JavaScript. The former is a function object while the latter
executes the function. The ability to pass around function objects like this makes creating callbacks really easy in Python
and JavaScript.
Exercise 1: Try it out!
Try running each of the following lines in a python shell. Make sure you understand what is going on before skipping to the next section.
def get_greeting(name): return "Hello, %s." % name
print get_greeting
print get_greeting("YOUR NAME HERE")
greeting_fn = get_greeting
print greeting_fn
print greeting_fn("YOUR NAME HERE")
Dynamic Callbacks Down the Rabbit Hole with Variable Scope
Callbacks are great for static buttons you define manually, but how do you use them with dynamic buttons with different behaviors? A simple example might be a "Correct My Location" feature that lets the user choose his or her location from a dropdown menu:
To avoid clouding the problem with buttons and server communication, let's imagine a list of dynamically created functions that each print out a different number:
def create_dynamic_callbacks():
button_callbacks = list() # Create a list for our callbacks
for num in range(10): # Loop through the numbers 0 to 9
# Create a new function for each iteration of the loop
def new_callback():
return num
# Append the new function to the list of callbacks
button_callbacks.append(new_callback)
# Try out all the callbacks to see if they work
# (we wouldn't do this in an actual program):
for callback in button_callbacks:
print callback()
create_dynamic_callbacks()
Exercise 2: Try it out!
Copy the above code into a file and run it. Is the result what you expect?While we might imagine that the program above should print out the numbers from 0 through 9, that assumption is clearly false. WHAT? You might ask. Is Python broken? Has my whole life been a lie?
Before you throw down your keyboard in disgust, I can assure you that Python is behaving exactly as it should. Let's get a better handle on what's going on by testing out some theories.
Theory One Many Copies of the Same Function?
Is the new_callback function only getting created once?
def create_dynamic_callbacks():
button_callbacks = list() # Create a list for our callbacks
for num in range(10): # Loop through the numbers 0 to 9
# Create a new function for each iteration of the loop
def new_callback():
return num
# Append the new function to the list of callbacks
button_callbacks.append(new_callback)
# TESTING THEORY ONE:
for callback in button_callbacks:
print callback
# Try out all the callbacks to see if they work
# (we wouldn't do this in an actual program):
for callback in button_callbacks:
print callback()
create_dynamic_callbacks()
... <function new_callback at 0x10046f7d0> <function new_callback at 0x10046f848> <function new_callback at 0x10046f8c0> <function new_callback at 0x10046f938> <function new_callback at 0x10046f9b0> <function new_callback at 0x10046fa28> <function new_callback at 0x10046faa0> <function new_callback at 0x10046fb18> <function new_callback at 0x10046fb90> <function new_callback at 0x10046fc08>
Nope. If you understood Exercise 1, you'll remember that different variables can both reference the same function object. In this case, you can see that each function object has a different address; each function is different.
Theory Two The value of num?
Is num actually assigned to each instance of the function?
def create_dynamic_callbacks():
button_callbacks = list() # Create a list for our callbacks
for num in range(10): # Loop through the numbers 0 to 9
# Create a new function for each iteration of the loop
def new_callback():
return num
# Append the new function to the list of callbacks
button_callbacks.append(new_callback)
# TESTING THEORY TWO:
num = 1337
# Try out all the callbacks to see if they work
# (we wouldn't do this in an actual program):
for callback in button_callbacks:
print callback()
create_dynamic_callbacks()
1337 1337 1337 1337 1337 1337 1337 1337 1337 1337
I think we're on to something here. Let's make sure:
def create_dynamic_callbacks():
button_callbacks = list() # Create a list for our callbacks
for num in range(10): # Loop through the numbers 0 to 9
# Create a new function for each iteration of the loop
def new_callback():
return num
# Append the new function to the list of callbacks
button_callbacks.append(new_callback)
# Try out all the callbacks to see if they work
# (we wouldn't do this in an actual program):
for callback in button_callbacks:
print callback()
print num
num = 1
print button_callbacks[0]()
num = 3
print button_callbacks[1]()
print button_callbacks[2]()
num = 7
print button_callbacks[3]()
create_dynamic_callbacks()
... 9 1 3 3 7
There we have it. In Python, variables in function objects don't get evaluated when the function is created; they get evaluated when the function is called. This is a powerful aspect of Python as a dynamic language, but it really isn't what we want in this case.
Binding Variables to Functions Taming Dynamic Callbacks
The problem with the dynamic function creation code we have been dealing with is one of variable scope.
The num variable is local to the create_dynamic_callbacks function but one level above all of the
new_callback functions.
We want a new local number variable to be created every time we create a new function. Since variables are evaluated the moment functions are called, we can solve our problem by calling a function to make our function:
def create_dynamic_callbacks():
button_callbacks = list() # Create a list for our callbacks
for num in range(10): # Loop through the numbers 0 to 9
# Create a function to return a function with a local num variable
def new_callback_creator(local_num):
def new_callback():
return local_num
return new_callback
# Create a new function with a local copy of num
callback = new_callback_creator(num)
# Append the new function to the list of callbacks
button_callbacks.append(callback)
# Try out all the callbacks to see if they work
# (we wouldn't do this in an actual program):
for callback in button_callbacks:
print callback()
create_dynamic_callbacks()
0 1 2 3 4 5 6 7 8 9
Yo dawg, I heard you like functions, so I put a function in your function to create a new function every time you loop. It ain't pretty, but it gets the job done. Now we can create buttons with dynamic callbacks. Hooray!
Asynchronous Callbacks in Javascript Concurrency with Style
While creating functions for buttons is one possible usecase, callbacks are also really useful for something called asynchronous programming. At the beginning of this tutorial, I talked about the importance of GUIs for your applications. The most important aspect of GUI development is interactivity. If you happen to perform a long operation while your program is running, you might block the user from performing any actions while that operation is happening.
This is A Very Bad Thing™, and you should never do it. Blocking operations are anathema to interactivity and will make users yell at you with righteous anger, as they should.
"But I need to do Cool Things™ in my program," you say. "Cool Things™ take a long time. Whatever shall I do?"
Never fear: with the magic of asynchronous programming, you can perform operations in the background, while the user is interacting with the interface. Let's switch languages for a moment and examine how asynchronous programming is used to perform background operations in the Marauder's Map@Olin Web Application.
JavaScript behaves similarly to Python when it comes to callbacks, but it also allows you to create multi-line anonymous functions, while Python restricts us to single-line lambda expressions when we want to create nameless functions.
When you launch the Marauder's Map Web application, it refreshes every user's position and then starts a function to refresh the position of every user again every second:
$(function () { // Once the webpage has loaded
...
var visibleUsers = {}; // Will keep track of users and their positions
...
var imgWidth;
var imgHeight;
$('#map-img').on('load', function () { // Once the image has loaded
// Get the dimensions of the image
imgWidth = $('#map-img').width();
imgHeight = $('#map-img').height();
updateUsers(visibleUsers, imgWidth, imgHeight, function () { // Once the update function has finished
setInterval(function() { // Every second (1000 milliseconds)
// Update user positions
updateUsers(visibleUsers, imgWidth, imgHeight, function () {} // Do nothing on completion
)}, 1000);
});
});
...
});
This excerpt from the Marauder's Map@Olin illustrates multiple uses for callbacks. The first line defines an anonymous function
to execute when the webpage loads, and line 7 begins the definition for an anonymous function that will be executed when a
page element with the id map-img has loaded. Both functions are callbacks that are called when an operation
is complete. Similarly, line 11 launches a function that runs a callback when it is finished. However, the setInterval
function on the next line is different. While it is still connected to a callback, it runs it every second.
updateUsers function is an asynchronous function, which means that the rest of the
program can continue executing before the function is finished. Additionally, if it takes longer than a second to
execute, it may be running at the same time as another instance of itself!
Let's take a closer look at the updateUsers function to see why it is asynchronous:
function updateUsers(usersObject, boundsWidth, boundsHeight, cb) {
var newUsers = {};
// Request user positions from the server asynchronously
Api.getPositions(true, function (err, json) {
var extendedPositions = json.positions;
// Loop through all user positions and update the users
for (var i=0; i < extendedPositions.length; i++) {
associatePositionMarkerWithBind(usersObject, extendedPositions[i], boundsWidth, boundsHeight);
newUsers[extendedPositions[i].username] = true;
}
for (var uname in usersObject) {
...
/* Delete the markers of users who should no longer be on the map because
the server didn't tell us their positions.*/
if (newUsers[uname] != true) {
delete usersObject[uname];
...
}
...
}
cb(); // Execute the callback once users have been updated
});
}
The specifics of how the updateUsers function works aren't important for our purposes; let's focus on line
4, which defines an asynchronous server request. Since the remainder of the function is encapsulated in an another anonymous
callback, the function won't do anything until the server responds. Meanwhile, anything after a call to updateUsers
can execute before the function has finished executing. In the example below, "Hello," will probably be logged before "Olin":
updateUsers({}, 100, 100, function () {
console.log("Olin");
});
console.log("Hello,");
Why is this important? It means that any interactive code placed after the function can continue to run while long server operations are happening, and the user won't get frustrated by the application appearing to freeze. It also means that we have to be very careful when we are designing algorithms that use data generated by the function.
To demonstrate the importance of this, let's pretend that the braces in updateUsers were moved around slightly:
function updateUsers(usersObject, boundsWidth, boundsHeight, cb) {
var newUsers = {};
// Request user positions from the server asynchronously
Api.getPositions(true, function (err, json) {
var extendedPositions = json.positions;
// Loop through all user positions and update the users
for (var i=0; i < extendedPositions.length; i++) {
associatePositionMarkerWithBind(usersObject, extendedPositions[i], boundsWidth, boundsHeight);
newUsers[extendedPositions[i].username] = true;
}
});
for (var uname in usersObject) {
...
/* Delete the markers of users who should no longer be on the map because
the server didn't tell us their positions.*/
if (newUsers[uname] != true) {
delete usersObject[uname];
...
}
...
}
cb(); // Execute the callback once users have been updated
});
}
Since Api.getPositions will not execute its callback until the server sends a response, usersObject and
newUsers will not have been modified by lines 8 and 9 before the for loop deletes every user on the map.
Oh noes!
Use asynchronous programming wisely.