An essay expanding on the theme of types and the web. I wrote this for Guile's manual, but it applies generally, I think. The point about SXML can apply fruitfully to python as well. -- Andy
It is a truth universally acknowledged, that a program with good use of data types, will be free from many common bugs. Unfortunately, the common practice in web programming seems to ignore this maxim. This subsection makes the case for expressive data types in web programming.
By "expressive data types", I mean that the data types say something about how a program solves a problem. For example, if we choose to represent dates using SRFI 19 date records (see SRFI-19), this indicates that there is a part of the program that will always have valid dates. Error handling for a number of basic cases, like invalid dates, occurs on the boundary in which we produce a SRFI 19 date record from other types, like strings.
With regards to the web, data types are help in the two broad phases of HTTP messages: parsing and generation.
Consider a server, which has to parse a request, and produce a response. Guile will parse the request into an HTTP request object (see Requests), with each header parsed into an appropriate Scheme data type. This transition from an incoming stream of characters to typed data is a state change in a program---the strings might parse, or they might not, and something has to happen if they do not. (Guile throws an error in this case.) But after you have the parsed request, "client" code (code built on top of the Guile HTTP stack) will not have to check for syntactic validity. The types already make this information manifest.
This state change on the parsing boundary makes programs more robust, as they themselves are freed from the need to do a number of common error checks, and they can use normal Scheme procedures to handle a request instead of ad-hoc string parsers.
The need for types on the response generation side (in a server) is more subtle, though not less important. Consider the example of a POST handler, which prints out the text that a user submits from a form. Such a handler might include a procedure like this:
;; First, a helper procedure (define (para . contents) (string-append "<p>" (string-concatenate contents) "</p>")) ;; Now the meat of our simple web application (define (you-said text) (para "You said: " text)) (display (you-said "Hi!")) -| <p>You said: Hi!</p>
This is a perfectly valid implementation, provided that the incoming text does not contain the special HTML characters <, >, or &. But this provision is not reflected anywhere in the program itself: we must assume that the programmer understands this, and performs the check elsewhere.
(display (you-said "<script src=\"http://bad.com/nasty.js\" />")) -| <p>You said: <script src="http://bad.com/nasty.js" /></p>
The fundamental problem here is that both user data and the program template are represented using strings. This identity means that types can't help the programmer to make a distinction between these two, so they get confused.
There are a number of possible solutions, but perhaps the best (in the Guile context) is to treat HTML not as strings, but as native s-expressions: as SXML. The basic idea is that HTML is either text, represented by a string, or an element, represented as a tagged list. So foo becomes "foo", and <b>foo</b>; becomes (b "foo"). Attributes, if present, go in a tagged list headed by @, like (img (@ (src "http://example.com/foo.png"))). See the SXML Wikipedia page, for more info.
The good thing about SXML is that HTML elements cannot be confused with text. Let's make a new definition of para:
(define (para . contents) `(p ,@contents)) (use-modules (sxml simple)) (sxml->xml (you-said "Hi!")) -| <p>You said: Hi!</p> (sxml->xml (you-said "<i>Rats, foiled again!</i>")) -| <p>You said: <i>Rats, foiled again!</i></p>
So we see in the second example that HTML elements cannot be unwittingly introduced into the output. However it is now perfectly acceptable to pass SXML to you-said; in fact, that is the big advantage of SXML over everything-as-a-string.
(sxml->xml (you-said (you-said "<Hi!>"))) -| <p>You said: <p>You said: <Hi!></p></p>
The SXML types allow procedures to compose. The types make manifest which parts are HTML elements, and which are text. (Using types to disallow nested paragraphs is an exercise for the reader.)
So you needn't worry about escaping user input; the type transition back to a string handles that for you. XSS vulnerabilities are a thing of the past.