Links: Web Programming Without Tiers?
Ezra Cooper, Sam Lindley, Philip Wadler, and Jeremy Yallop
University of Edinburgh
Abstract. Links is a programming language for web applications that generates
code for all three tiers of a web application from a single source, compiling into
JavaScript to run on the client and into SQL to run on the database. Links supports rich clients running in what has been dubbed ‘Ajax’ style, and supports
concurrent processes with statically-typed message passing. Links is scalable in
the sense that session state is preserved in the client rather than the server, in
contrast to other approaches such as Java Servlets or PLT Scheme. Client-side
concurrency in JavaScript and transfer of computation between client and server
are both supported by translation into continuation-passing style.
1
Introduction
A typical web system is organized in three tiers, each running on a separate computer
(see Figure 1). Logic on the middle-tier server generates pages to send to a front-end
browser and queries to send to a back-end database. The programmer must master a
myriad of languages: the logic is written in a mixture of Java, Perl, PHP, and Python;
the pages described in HTML, XML, and JavaScript; and the queries are written in SQL
or XQuery. There is no easy way to link these — to be sure that a form in HTML or a
query in SQL produces data of a type that the logic in Java expects. This is called the
impedance mismatch problem.
Links eliminates impedance mismatch by providing a single language for all three
tiers. In the current version, Links translates into JavaScript to run on the browser
and SQL to run on the database. The server component is written in O’Caml; it consists of a static analysis phase (including Hindley-Milner typechecking), a translator to
JavaScript and SQL, and an interpreter for the Links code that remains on the server.
Increasingly, web applications designers are migrating work into the browser. “Rich
client” systems, such as Google Mail and Google Maps, use a new style of interaction
dubbed “Ajax” [11]. Client-side Links compiles into JavaScript, a functional language
widely available on most browsers. JavaScript is notoriously variable across platforms,
so we have designed the compiler to target a common subset that is widely available. It
would be easy to extend Links to support additional target languages, such as Flash or
Java. However, we expect the popularity of Ajax will mean that standard and reliable
versions of JavaScript will become available over the next few years.
Links is a strict, typed, functional language. It incorporates ideas proven in other
functional languages, including:
– database query optimization, as found in Kleisli and elsewhere,
?
Submitted to FMCO ‘06
query
request
Browser
(HTML, XML,
JavaScript)
Server
(Java, Perl, PHP,
Python, Ruby)
response
Database
(SQL, XQuery)
result
Fig. 1. Three-tier model
– continuations for web interaction, as found in PLT Scheme and elsewhere, and
– concurrency with message passing, as found in Erlang and elsewhere,
All three of these features work better with immutable values rather than mutable objects. In Links, side effects play a limited (though important) role, being used for updates to the database and display, and communication between concurrent processes.
Types ensure consistency between forms in the browser, logic in the server, and queries
on the database; and between the sender and receiver of a message in a concurrent
program.
Links programs are scalable in the sense that session state is preserved in the client
rather than the server. Many commercial web tools (like J2EE) and most research web
tools (including current releases of PLT Scheme [15] and Mozart QHTML [9]) are not
scalable in this sense.
In Links, all server state is serialized and passed to the client, then restored to the
server when required. This resumption passing style extends the continuation passing
style commonly used in PLT Scheme and elsewhere. Links functions are labelled as to
whether they are intended to execute on the client or the server; functions running on
the client may invoke those on the server, and vice-versa.
Database programming. Queries are written in the Links notation and compiled into
SQL, a technique pioneered by Kleisli [7, 35] and now used in LINQ [18].
Web interaction. The notion that a programming language could provide support for
web interaction first appears in the programming language MAWL [1]. The notion of
continuation from functional programming has been particularly fruitful, being applied
by a number of researchers to improve interaction with a web client, including Quiennec
[25], Graham [13] (in a commercial system sold to Yahoo and widely used for building
web stores), Felleisen and others [14, 15], and Thiemann [31].
Concurrency. Links supports concurrent programming in the client, using “share nothing” concurrency where the only way processes can exchange data is by message passing, as pioneered in Erlang [2] and Mozart [32].
XML programming. Links provides convenient syntax for constructing XML data, similar to that provided by XQuery [36]. The current version does not support regular expression types, as found in XDuce and other languages, but we may add them in a future
version. Regular expression types were given a low priority, because they are already
well understood from previous research [19].
Other languages. Other languages for web programming include Xtatic [16], Scala
[22], Mozart [9], SML.NET [5], F] [30], Cω (based on Polyphonic C] [4] and Xen [6]),
HOP [29], Ocsigen [3]. These languages have many overlaps with Links, as they are
also inspired by the functional programming community.
However, none of these languages shares Links’ objective of generating code for
all three tiers of a web application from a single source — scripts for the front-end
client, logic for the middle-tier server, and queries for the back-end database. We expect
that providing a single, unified language as an alternative to the current multiplicity of
languages will be a principal attraction of Links.
This paper. We introduce Links by describing three examples in Section 2. Section 3
sketches our implementation of concurrency and client-server interaction. Section 4
gives an SQL-compilable subset of Links and details how it is compiled into SQL,
while Section 5 describes typing for processes. Section 6 discusses some shortcomings
of the current implementation and Section 7 concludes.
2
Links by example
This section introduces Links by a series of examples. The reader is encouraged to try
these examples online at
http://groups.inf.ed.ac.uk/links/examples/
We begin with a dictionary-suggest example to introduce the basic functionality of
Links, and then present two further examples that demonstrate additional capabilities:
a draggable list, and a progress bar.
2.1
Dictionary suggest
The dictionary suggest application presents a text box, into which the user may type a
word. As the user types, the application displays a list of words that could complete the
one being typed. A dictionary database is searched, and the first ten words beginning
with the same prefix as the typed word are presented (see Figure 2). In addition, the
user can add, update and delete definitions. To add a new definition the user fills in
and submits the form at the bottom of the page by clicking ‘Add’. To update an existing
definition the user clicks on one of the suggestions, which replaces the suggestion with a
form containing the existing definition, and then edits and submits the form by clicking
‘Update’. This form also includes buttons for cancelling the update (which restores the
original suggestion) and deleting the definition.
Fig. 2. Dictionary suggest
This application is of interest because it must perform a database lookup and update
the display at every keystroke. Applications such as Google Suggest [17] have a similar
structure.
The Links version is based on an ASP.NET version of the same application, available online [21], using the same dictionary and formatting. It extends the ASP.NET
version by allowing the definitions to be added, updated and deleted. The dictionary
contains 99,320 entries. Links is acceptably fast for this application, and response times
for the Links and ASP.NET versions appear identical. (However, we have only tested
with a lightly loaded server.)
The code for the application is shown in Figures 3–5; following is a short walkthrough of the code. On each keystroke, a Suggest message containing the current
contents of the text field is sent to the handler process. The handler process passes
the text content to the function suggest. This function calls completions on the
server to find the first ten words with the given prefix, and format on the client to
format the list returned. Doing the server interaction in a separate handler process
allows the user interaction to remain responsive, even while looking up suggestions.
var defsTable =
table "definitions" with
(id:String, word:String, meaning:String)
where id readonly from database "dictionary";
fun newDef(def) server { insert defsTable values [def] }
fun updateDef(def) server {
update (var d <-- defsTable) where (d.id == def.id)
set (word=def.word, meaning=def.meaning)
}
fun deleteDef(id) server {
delete (var def <-- defsTable) where (def.id == id)
}
fun completions(s) server {
if (s == "") [] else {
take(10, for (var def <-- defsTable)
where (def.word ˜ /s.*/) orderby (def.word)
[def])
}
}
fun suggest(s) client {
replaceChildren(format(completions(s)),
getNodeById("suggestions"))
}
fun editDef(def) client {
redraw(
,
def.id)
}
Fig. 3. Dictionary suggest in Links (1)
fun redraw(xml, defId) client {
replaceChildren(xml, getNodeById("def:"++defId))
}
fun formatDef(def) client {
{stringToXml(def.word)}
{stringToXml(def.meaning)}
}
fun format(defs) client {
<#>
Click a definition to edit it
for (var def <- defs)
{formatDef(def)}
#>
}
fun addForm(handler) client {
}
var handler = spawn {
fun receiver(s) {
receive {
case Suggest(s) -> suggest(s); receiver(s)
case NewDef(def) ->
newDef(def);
replaceChildren(addForm(self()), getNodeById("add"));
suggest(s); receiver(s)
}
}
receiver("")
};
Fig. 4. Dictionary suggest in Links (2)
Dictionary suggest
Dictionary suggest
Search for definitions
New definition
{addForm(handler)}
Fig. 5. Dictionary suggest in Links (3)
The rest of the code is concerned with modifying the database. A form for adding
definitions is created by the function addForm. Clicking ‘Add’ sends a NewDef message to the handler process containing a new definition. The handler process calls
newDef to add the definition, resets the form and updates the list of suggestions (in
case the new definition appears in the current list of suggestions).
Clicking on a definition invokes the function editDef, which calls the function
redraw in order to replace the definition with a form for editing it. Clicking ‘Cancel’
reverses this operation. Clicking ‘Update’ or ‘Delete’ performs the corresponding modification to the definition by calling updateDef or deleteDef on the server, and
then updates the list of suggestions by calling the function redraw on the client.
Syntax. The syntax of Links resembles that of JavaScript. This decision was made not
because we are particularly fond of this syntax, but because we believe it will be familiar
to our target audience. Low-level syntactic details become a matter of habit that can be
hard to change: we conjecture that using the familiar f(x,y) in place of the unfamiliar
f x y will significantly lower the barrier to entry of functional programming.
One difference from JavaScript syntax is that we do not use the keyword return,
which is too heavy to support a functional style. Instead, we indicate return values subtly
(perhaps too subtly), by omitting the trailing semicolon.
Types. Links uses Hindley-Milner type inference with row variables [24]. As basic
types Links supports integers, floats, characters, booleans, lists, functions, records, and
variants.
– A list is written [e1 ,...,ek ], and a list type is written [A].
– A lambda abstraction is written fun (x1 ,...,xk ) {e}, and a function type is
written (A1 ,...,Ak ) -> B.
– A record is written (f1 =e1 ,...,fk =ek ), and a record type is written
(f1 :A1 ,...,fk :Ak |r), where r is an optional row variable. Field names for
records begin with a lower-case letter.
– A variant is written Fi (ei ) and a variant type is written
[|F1 :A1 ,...,Fk :Ak |r|]. Field names for variants begin with an uppercase letter.
Strings are lists of characters (as in Haskell), and tuples are records with natural number
labels (as in SML). Apart from the table declaration in Figure 3, none of the examples
in the paper explicitly mention types. This is partly because type inference renders type
annotations unnecessary and partly to save space. Nevertheless, it should be stressed
that all of the examples are type-checked statically, and static typing is an essential part
of Links.
Links currently does not support any form of overloading; we expect to support
overloading in future using a simplified form of type classes. As with regular expression
types, this was left to the future because it is well understood from other research efforts.
In particular, the WASH and iData systems make effective use of type classes to support
generic libraries [31, 23].
XML. Links includes special syntax for constructing and manipulating XML data.
XML data is written in ordinary XML notation, using curly braces to indicate embedded code. Embedded code may occur either in attributes or in the body of an element.
The Links notation is similar to that used in XQuery, and has similar advantages. In
particular, it is easy to paste XML boilerplate into Links code. The parser begins parsing XML when a < is immediately followed by a legal tag name; a space must always
follow < when it is used as a comparison operator; legal XML does not permit a space
after the < that opens a tag. Links also supports syntactic sugar <#> ... #> for
specifying an XML forest literal as in the function format in Figure 4.
The client maintains an XML tree representing the current document to display;
usually this will be in XHTML, the dialect of XML corresponding to HTML. This distinguished tree is referred to as the Document Object Model, or DOM for short. Links
provides library functions to access and modify the DOM, based on the similar operations specified by the W3C DOM standard. Links supports two types for manipulating
XML: DomNode is a mutable reference (similar to a ref type in ML), while Xml is
an immutable list of trees. There is an operation that converts the former to the latter
by making a deep copy of the tree rooted at the node, returning it in a singleton list.
We expect eventually to support regular expression types for XML that refine each of
these two types, and to support a notation like XPath for manipulating trees, but as
these points are well understood (but a lot of work to implement) they are not a current
priority.
Regular expressions. Matching a string against a regular expression is written e ˜ /r/
where r is a regular expression. Curly braces may be used to interpolate a string into a
regular expression, so for example /{s}.*/ matches any string that begins with the
value bound to the variable s.
Interaction. The Links code specifies display of an XML document, the crucial part of
which is the following:
The l:name attribute specifies that the string typed into the field should be bound to a
string variable s.
The attributes l:name and l:onkeyup are special. The attribue l:onkeyup
is followed by Links code in curly braces that is not immediately evaluated, but is
evaluated whenever a key is released while typing in the form. (Normally, including
curly braces in XML attributes, as elsewhere in XML, causes the Links code inside the
braces to be evaluated and its value to be spliced into the XML.) The l:name attribute
on an input field must contain a Links variable name, and that variable is bound to the
contents of the input field.
The attributes that Links treats specially are l:name and all the attributes connected with events: l:onchange, l:onsubmit, l:onkeyup, l:onmousemove,
and so on. These attributes are prefixed with l:, using the usual XML notation for
namespaces; in effect, l denotes a special Links namespace.
The scope of variables bound by l:name is the Links code that appears in the
attributes connected with events. Links static checking ensures that a static error is
raised if a name is used outside of its scope; this guarantees that the names mentioned
on the form connect properly to the names referred to by the application logic. In this,
Links resembles MAWL and Jwig, and differs from PLT Scheme or PHP.
Our experience has shown that this style of interaction does not necessarily scale
well, and it may be preferable to use a library of higher-order forms as developed in
WASH or iData. We return to this point in the Section 6.
List comprehensions. The Links code calls the function suggest each time a key
is released. This in turn calls completions to find the first ten completions of the
prefix, and format to format the results as HTML for display. Both completions
and format use for loops, in the former case also involving where and orderby
clauses. These constructs correspond to what is written as a list comprehension in languages such as Haskell or Python. Each comprehension is equivalent to an ordinary
expression using standard library functions, we give three examples, where the comprehension is on the left and its translation is on the right.
for (var x <- e1)
e2
concatMap(fun(x){e2},e1)
for (var x <- e1)
where (e2)
e3
concatMap(
fun(x){if (e2) e3 else []},
e1)
for (var x <- e1)
orderby (e2)
e3
concatMap(
fun(x){e3},
orderBy(fun(x){e2},e1))
Here concatMap(f,xs) applies function f to each element in list xs and concatenates the results, and orderBy(f,xs) sorts the list xs so that it is in ascending order
after f is applied to each element. The orderby notation is conceptually easier for the
user (since there is no need to repeat the bound variables) and technically easier for the
compiler (because it is closer to the SQL notation that we compile into, as discussed
in the next section; indeed, currently orderby clauses only work reliably in code that
compiles into SQL, because the absence of overloading means we have not yet implemented comparison on arbitrary types in the client or server).
Database. Links provides facilities to query and update SQL databases, where database
tables are viewed as lists of records. We hope to provide similar facilities for XQuery
databases in the future.
Links provides a database expression that denotes a connection to a database,
and a table expression that denotes a table within a database. A database is specified
by name (and optional configuration data, which is usually read from a configuration
file), and a table is specified by the table name, the type signature of a row in the table,
and the database containing the table.
The type of a table is distinct from the type of its list of records, since tables
(unlike lists) may be updated. The coercion operation asList takes a table into the
corresponding list, and for (var x <-- e1) e2 (with a long arrow) is equivalent to
for (var x <- asList(e1)) e2 (with an ordinary arrow).
In the following example, there is a table of words, where each row contains three
string fields, the word itself, its type (noun, verb, and so on), and its meaning (definition). Typically, one expresses queries using the Links constructs such as for, where,
and orderby, and functions on lists such as take and drop. The Links compiler is
designed to convert these into appropriate SQL queries over the relevant tables whenever possible. For example, the expression
take(10, for (var def <-- defsTable)
where (def.word ˜ /s.*/) orderby (def.word)
[def])
compiles into the equivalent SQL statement
SELECT def.meaning AS meaning, def.word AS word
FROM definitions AS def
WHERE def.word LIKE ’{s}%’
ORDER BY def.word ASC
LIMIT 10 OFFSET 0
This form of compilation was pioneered in systems such as Kleisli [7, 35], and is also
central to Microsoft’s new Language Integrated Query (LINQ) for .NET [18]. A related approach, based on abstract interpretation rather than program transformation, is
described by Wiederman and Cook [34].
Note that the match operator ˜ on regular expressions in Links is in this case compiled into the LIKE operator of SQL, and that the call to take in Links is compiled
into LIMIT and OFFSET clauses in SQL. At run time, the phrase {s} in the SQL is
replaced by the string contained in the variable s, including escaping of special characters where necessary. Links also contains statements to update, delete from, and insert
into a database table, closely modeled on the same statements in SQL.
The LIMIT and OFFSET clauses are not part of the SQL standard, but are available
in PostgreSQL, MySQL, and SQLite, the three targets currently supported by Links.
For non-standard features such as this, Links can generate different code for different
targets; for instance, it generates different variants of INSERT for PostgreSQL and
MySQL.
Section 4 presents a subset of Links that is guaranteed to compile into SQL queries.
Concurrency. Links allows one to spawn new processes. The expression spawn {e}
creates a new process and returns a fresh process identifier, the primitive self() returns the identifier of the current process, the command e1 ! e2 sends message e2 to
the process identified by e1, and the expression
receive {
case p1 -> e1
...
case pn -> en
}
extracts the next message sent to the current process (or waits if there is no message),
and executes the first case where the pattern matches the message. (Unlike Erlang, it is
an error if no case matches, which fits better with our discipline of static typing.)
By convention, the messages sent to a process belong to a variant type. Event handlers are expected to execute quickly, so we follow a convention that each handler either sends a message or spawns a fresh process. For instance, in the dictionary suggest
application, each event handler sends a message to a separate handler process that is
responsible for finding and displaying the completions of the current prefix. This puts
any delay resulting from the database lookup into a separate process, so any keystroke
will be echoed immediately. Furthermore, the messages received by the handler process
are processed sequentially, ensuring that the updates to the DOM happen in the correct
order.
As well as any new processes spawned by the user, there is one distinguished process: the main process. The top-level program and all event handlers are run in the main
process. Having all event handlers run in a single process allows us to guarantee that
events are processed in the order in which they are received.
Client-server. The keyword server in the definition of completions causes invocations of that function to be evaluated on the server rather than the client, which is
required because the database cannot be accessed directly from the client.
When the dictionary suggest application is invoked, the server does nothing except
to transmit the Links code (compiled into JavaScript) to the client. If the user presses
and releases a key then the suggest function will continue to run on the client as
its definition is annotated with the keyword client. The client runs autonomously
until completions is called, at which point the XMLHttpRequest primitive of
JavaScript is used to transmit the arguments to the server and creates a new process on
the server to evaluate the call. No server resources are required until completions is
called. This contrasts with Java Servlets, which keep a process running on the server at
all times, or with PLT Scheme, which keeps a continuation cached on the server.
Location annotations server and client are only allowed on top-level function
definitions. If a top-level function definition does not have a location annotation then it
Fig. 6. Draggable list: before and after dragging
will be compiled for both the client and the server. A location annotation should be read
as “this function must be run in the specified location”. The implementation strategy for
client-server communication is discussed further in Section 3.
2.2
Draggable list
The draggable list application displays an itemized list on the screen. The user may
click on any item in the list and drag it to another location in the list (see Figure 6).
The code for draggable list is shown in Figure 7. The example exploits Links’ ability
to run concurrent processes in the client. Each draggable list is monitored by a separate
process.
Each significant event on an item in the list (mouse up, mouse down, mouse out)
has attached to it code that sends a message to the process indicating the event. The
process itself is represented by two mutually recursive functions, corresponding to two
states. The process starts in the waiting state; when the mouse is depressed it changes to
the dragging state; and when the mouse is released it reverts to the waiting state. When
the mouse is moved over an adjacent item while in the dragging state, the dragged item
and the adjacent item are swapped.
The function corresponding to the waiting state takes as a parameter the id of the
element containing the draggable list; the function corresponding to the dragging state
takes the same parameter, and a second parameter indicating the item currently being
dragged. Both functions are written in tail recursive style, each either calling itself (to
remain in that state) or the other (to change state). This style of coding state for a process
was taken from Erlang.
In this simple application, the state of the list can be recovered just by examining
the text items in the list. In a more sophisticated application, one might wish to add
another parameter representing the abstract contents of the list, which might be distinct
from the items that appear in the display.
Observe that the code has been written so that there can be multiple draggable lists,
each monitored by its own process.
fun waiting(id) {
receive {
case MouseDown(elem)
->
if (isElementNode(elem)
&& (parentNode(elem) == getNodeById(id)))
dragging(id,elem)
else waiting(id)
case MouseUp
-> waiting(id)
case MouseOver(newElem) -> waiting(id)
}
}
fun dragging(id,elem) {
receive {
case MouseUp
-> waiting(id)
case MouseDown(elem) ->
if (isElementNode(elem)
&& (parentNode(elem) == getNodeById(id)))
dragging(id,elem)
case MouseOut(toElem) ->
if (isElementNode(toElem)
&& (parentNode(elem) == getNodeById(id)))
{swapNodes(elem,toElem); dragging(id,elem)}
else dragging(id,elem)
}
}
fun draggableList (id,items) client {
var dragger = spawn { waiting(id) };
{ for (var item <- items)
{ item }
}
}
Draggable lists
Great Bears
{
draggableList("bears",
["Pooh", "Paddington", "Rupert", "Edward"])
}
Fig. 7. Draggable lists in Links
Fig. 8. Progress bar
Fig. 9. Progress bar displaying the final answer
2.3
Progress bar
The progress bar demonstrates an application where computation bounces back and
forth between client and server. A computation is performed on the server, and the
progress of this computation is demonstrated with a progress bar (see Figure 8). When
the computation is completed, the answer is displayed (see Figure 9).
The code for the progress bar application is shown in Figure 10. In this case, the
computation performed on the server is uninteresting, simply counting up to the number
typed by the user. In a real application the actual computation chosen might be more
interesting.
Periodically, the function computation (running on the server) invokes the function showProgress (running on the client) to update the display to indicate progress.
When this happens, the state of the computation on the server is pickled and transmitted
to the client. This is done by applying continuation passing style and defunctionalization to the code on the server; the client is passed the name of a function corresponding
to the continuation of the computation, and the data needed to continue the computation. Note that no state whatsoever is maintained on the server when computation is
moved to the client. The implementation is discussed in more detail in Section 3.
One advantage of writing the computation in this way is that if the client quits at
some point during the computation (either by surfing to another page, terminating the
browser, or taking a hammer to the hardware) then no further work will be required of
the server.
On the other hand, the middle of an intensive computation may not be the best time
to pickle the computation and ship it elsewhere. A more sophisticated design would
asynchronously notify the client while continuing to run on the server, terminating the
computation if the client does not respond to progress updates after a reasonable interval. This is not possible currently, because the design of Links deliberately limits the
ways in which the server can communicate with the client, in order to guarantee that
long-term session state is maintained on the client rather than the server. This is not
fun compute(count, total) server {
if (count < total) {
showProgress(count, total);
compute(count+1, total)
} else "done counting to " ++ intToString(total)
}
fun showProgress(count, total) client {
var percent =
100.0 *. intToFloat(count) /. intToFloat(total);
replaceNode(
|
,
getNodeById("bar")
)
}
fun showAnswer(answer) client {
domReplaceNode(
{stringToXml(answer)}
getNodeById("bar")
);
}
Fig. 10. Progress bar in Links
appropriate in all circumstances, and future work on Links will need to consider a more
general design.
3
Client-server computing
A Links program can be seen as a distributed program that executes across two locations: a client (browser) and a server. The programmer can optionally annotate a
function definition to indicate where it should run. Functions on the client can invoke
functions on the server, and vice-versa.
f
main
g
Call to f (server)
Client
Server
{Call f}
Call to g (client)
{Call g, k}
{Continue r, k}
Return r from g
Return s from f
Source language:
call/return style
{Return s}
Implementation:
request/response style
Fig. 11. Semantic behaviour of client/server annotations
This symmetry is implemented on top of the asymmetric mechanisms made available by web browsers and servers. Current standards only permit the browser to invoke
a function on the server. The server can return a value to the browser when done, but
there is no provision for the server to invoke a function on the browser directly.
Our implementation is also scalable, in the sense that session state is preserved in
the client rather than the server. Since server resources are at a premium in the web environment, our implementation requires no server resources unless the server is actively
doing work.
We achieve this by using a variation of continuation-passing style, which we call
resumption-passing style. It is now common on the web to use continuations to “invert
back the inversion of control” [26], permitting a process server to retain control after
sending a form to the client. Here, we permit a process on the server to retain control
after invoking an arbitrary function on the client.
Figure 11 shows how the call/return style of programming offered by Links differs
from the standard request/response style, and how request/response style can emulate
the call/return style. The left-hand diagram shows a sequence of calls between functions
annotated with “client” and “server.” The solid line indicates the active thread of control
as it descends into these calls, while the dashed line indicates a stack frame which is
waiting for a function to return. In this example, main is a client function which calls a
server function f which in turn calls a client function g. The semantics of this call and
return pattern are familiar to every programmer.
The right-hand diagram shows the same series of calls as they are implemented.
The dashed line here indicates that some local storage is being used as part of this
computation. During the time when g has been invoked but has not yet returned a value,
the server stores nothing locally, even though the language provides an illusion that f
is “still waiting” on the server. All of the server’s state is encapsulated in the value k,
which it passed to the client with its call.
To accomplish this, the program is compiled to two targets, one for each location,
and begins running at the client. In this compilation step, server-annotated code is re-
placed on the client by a remote procedure call to the server. This RPC call makes an
HTTP request indicating the server function and its arguments. The client-side thread
is effectively suspended while the server function executes.
Likewise, on the server side, a client function is replaced by a stub. This time, however, the stub will not make a direct call to the client, since in general web browsers are
not addressable by outside agents. However, since the server is always working on behalf of a client request, we have a channel on which to communicate. So the stub simply
gives an HTTP response indicating the server-to-client call, along with a representation
of the server’s continuation, to be resumed when the server-to-client call is complete.
Upon returning in this way, all of the state of the Links program is present at the client,
so the server need not store anything more.
When the client has completed the server-to-client call, it initiates a new request
to the server, passing the result and the server continuation. The server resumes its
computation by applying the continuation.
3.1
Client-side Concurrency
A Links program is concurrent. We implement concurrency on the client, even though
JavaScript does not contain primitives to support concurrency. Concurrency, client-toserver communication and server-to-client communication are implemented using the
following techniques:
–
–
–
–
compiling to continuation-passing style,
inserting explicit context-switch instructions,
registering asynchronous remote calls using XMLHttpRequest,
eliminating the stack between context switches using either setTimeout or a
trampoline.
Producing JavaScript code in CPS allows us to reify each process’s control state.
The compiler wraps every function application and every continuation application in
special “yield” functions. After a certain number of calls to “yield” the current continuation is added to the JavaScript VM’s collection of timeout callbacks allowing another
continuation to run. This is implemented by calling the setTimeout primitive of
JavaScript, which registers a callback to be invoked after a specified period of time.
The “yield” function for function application is shown in Figure 12.1 The variable
_yieldGranularity determines how often to yield to other processes. After every
_yieldGranularity calls to _yield and _yieldCont control is relinquished
using one of two methods.
– If execution is outside an event handler, then the built-in setTimeout function is invoked with a callback that sets the process id and then calls f(a, k).
Calling setTimeout allows other processes to run. The _sched_pause parameter specifies a lower bound on how long to wait before allowing this callback to run. For older browsers _sched_pause must be non-zero, otherwise
1
The “yield” function for continuation application is the same except it takes as arguments
continuation k and value a, and the function applications f(a, k) are replaced with continuation applications k(a).
the callback may be run immediately and other callbacks may not get a chance
to run. For newer browsers _sched_pause may be set to zero, as the callback
is placed at the back of a queue of setTimeout callbacks. Typical JavaScript
implementations do not use tail-call elimination, and have a severely limited call
stack. Our use of setTimeout also sidesteps this limitation, since setTimeout
discards the current stack. In our case the stack contains only tail calls that we need
never return to, so this reclaims memory while retaining the desired behaviour. The
timeout also provides an opportunity for the event handler or any callbacks from
XMLHttpRequest to run.
– Inside an event handler execution is synchronous, allowing events to be handled
atomically. In order to prevent event handlers from blocking concurrency, we recommend that they be short-lived (ideally just send a message or spawn a new process). Although event-handling is synchronous, it is still important to make sure
that the browser does not run out of stack. Although running out of stack inside
event handlers is unlikely to be a problem if programmers follow our guidelines, a
trampoline is provided just in case. In order to yield, an exception is thrown containing the current continuation. Throwing the exception unwinds the stack. The
trampoline catches the exception and then invokes the continuation. The trampoline is shown in Figure 13.
Unsurprisingly, the output of the CPS-translating compiler is slower than comparable direct style code. The performance penalty is significant but not disastrous: the total
run-time of programs we measured was within an order of magnitude of equivalent
hand-written JavaScript versions.
function _yield(f, a, k) {
++_yieldCount;
if ((_yieldCount % _yieldGranularity) == 0) {
if (!_handlingEvent) {
var current_pid = _current_pid;
setTimeout((function() {
_current_pid = current_pid;
f(a, k)}),
_sched_pause);
}
else {
throw new _Continuation(function () { f(a,k) });
}
}
else {
return f(a,k);
}
}
Fig. 12. The yield function
function _Continuation(v) { this.v = v }
function _wrapEventHandler(handler) {
return function(event) {
var active_pid = _current_pid;
_current_pid = _mainPid;
_handlingEvent = true;
var cont = function () { handler(event) }
for (;;) {
try {
cont();
_handlingEvent = false;
_current_pid = active_pid;
return;
}
catch (e) {
if (e instanceof _Continuation) {
cont = e.v;
continue;
}
else {
_handlingEvent = false;
_current_pid = active_pid;
throw e;
}
}
}
}
}
Fig. 13. The event handler trampoline
Client-to-server calls are implemented using the asynchronous form of the function
XMLHttpRequest, to which we pass a callback function that invokes the continuation of the calling process.
3.2
Related work: continuations for web programming
The idea of using continuations as a language-level technique to structure web programs
has been discussed in the literature [25, 26] and used in several web frameworks (such
as Seaside, Borges, PLT Scheme, RIFE and Jetty) and applications, such as ViaWeb’s
e-commerce application [12], which became Yahoo! Stores. Most these take advantage
of a call/cc primitive in the source language, although some implement a continuationlike object in other ways. Each of these systems creates URLs which correspond to
continuation objects. The most common technique for establishing this correspondence
is to store the continuation in memory, associated with a unique identifier which is
included as part of the URL.
Relying on in-memory tables makes such a system vulnerable to system failures
and difficult to distribute across multiple machines. Whether stored in memory or in a
database, the continuations can require a lot of storage. Since URLs can live long in
bookmarks, emails, and other media, it is impossible to predict for how long a continuation should be retained. Most of the above frameworks destroy a continuation after
a given period of time. Even with a modest lifetime, server storage needs can be quite
high, as each request may generate many continuations and there may be many simultaneous users.
Our approach differs from these implementations by following the approach described by the PLT Scheme team [14]. Rather than storing a continuation object at the
server and referring to it by ID, we serialize continuations and embed them completely
in URLs and hidden form variables. To make this efficient, we assume that the program
itself is fixed over time. Then we can defunctionalize the continuation object ([27]),
replacing functions with unique identifiers. The resulting representation only identifies
the closures which need to execute in the future of the computation, and the dynamic
data needed by those closures.
4
Query compilation
A subset of Links expressions compiles precisely to a subset of SQL expressions. Figure 14 gives the grammar of the SQL-compilable subset of Links expressions and Figure 15 gives the grammar for the subset of SQL to which we compile.
All terms are identified up to α-conversion (that is up to renaming of bound variables, field names and table aliases). We allow free variables in queries, though they
are not strictly allowed in SQL. Values for these variables will be supplied either at
compile time, during query rewriting, or else at runtime. We use vector notation v̄ to
mean v1 ,...,vk , where 1 ≤ k and abuse vector notation in the obvious way in
order to denote tuples, record patterns and lists of tables. For uniformity, we write an
empty ‘from’ clause as from •. In standard SQL the clause would be omitted altogether. We assume a single database db, and write table t with (f1 ,...,fk )
for asList(table t where (f1 :A1 ,...,fk :Ak ) from db).
(expressions)
e ::= take(n, e) | drop(n, e) | s
(simple expressions)
s ::=
|
|
|
|
for (pat <- s) s
let x = b in s
where (b) s
table t with f̄
[(b̄)]
(basic expressions)
b ::=
|
|
|
|
b1 op b2
not b
x
lit
z.f
pat ::= z | (f̄=x̄)
(patterns)
op ::= like | > | = | < | <> | and | or
(operators)
lit ::= true | false | string-literal | n
(literal values)
i, m, n
(finite integers)
(field names)
f, g
(variables)
x, y
(record variables)
z
(table names)
t
Fig. 14. Grammar of an SQL-compilable subset of Links.
(queries)
q ::= select Q limit ninf offset n
(query bodies)
Q ::= cols from tables where c
cols ::= c̄
(column lists)
tables ::= t̄ as ā | •
(table lists)
(SQL expressions)
c, d ::=
|
|
|
|
(integers)
ninf ::= n | ∞
(table aliases)
c op d
not c
x
lit
a.f
a
Fig. 15. The SQL grammar that results from query compilation.
TABLE
table t with f̄
−→ (a is fresh)
select a.f̄ from t as a where true limit ∞ offset 0
L ET
let x = b in s −→ s[b/x]
T UPLE
[(b̄)] −→ select b̄ from • where true limit ∞ offset 0
J OIN
for ((f̄=x̄)