wingologA mostly dorky weblog by Andy Wingo2024-01-11T14:10:38Ztekutihttps://wingolog.org/feed/atomAndy Wingohttps://wingolog.org/micro macro story timehttps://wingolog.org/2024/01/11/micro-macro-story-time2024-01-11T14:10:38Z2024-01-11T14:10:38Z

Today, a tiny tale: about 15 years ago I was working on Guile’s macro expander. Guile inherited this code from an early version of Kent Dybvig’s portable syntax expander. It was... not easy to work with.

Some difficulties were essential. Scope is tricky, after all.

Some difficulties were incidental, but deep. The expander is ultimately a function that translates Scheme-with-macros to Scheme-without-macros. However, it is itself written in Scheme-with-macros, so to load it on a substrate without macros requires a pre-expanded copy of itself, whose data representations need to be compatible with any incremental change, so that you will be able to use the new expander to produce a fresh pre-expansion. This difficulty could have been avoided by incrementally bootstrapping the library. It works once you are used to it, but it’s gnarly.

But then, some difficulties were just superflously egregious. Dybvig is a totemic developer and researcher, but a generation or two removed from me, and when I was younger, it never occurred to me to just email him to ask why things were this way. (A tip to the reader: if someone is doing work you are interested in, you can just email them. Probably they write you back! If they don’t respond, it’s not you, they’re probably just busy and their inbox leaks.) Anyway in my totally speculatory reconstruction of events, when Dybvig goes to submit his algorithm for publication, he gets annoyed that “expand” doesn’t sound fancy enough. In a way it’s similar to the original SSA developers thinking that “phony functions” wouldn’t get published.

So Dybvig calls the expansion function “χ”, because the Greek chi looks like the X in “expand”. Fine for the paper, whatever paper that might be, but then in psyntax, there are all these functions named chi and chi-lambda and all sorts of nonsense.

In early years I was often confused by these names; I wasn’t in on the pun, and I didn’t feel like I had enough responsibility for this code to think what the name should be. I finally broke down and changed all instances of “chi” to “expand” back in 2011, and never looked back.

Anyway, this is a story with a very specific moral: don’t name your functions chi.

Andy Wingohttps://wingolog.org/goto 1965https://wingolog.org/2009/08/10/goto-2009-08-10T07:45:43Z2009-08-10T07:45:43Z

travels through space

Time passes, and space with it. I don't like airplanes overmuch, so I caught a train up to Paris last weekend, and from there Kate & I took a plane to Dublin to visit Jan Jaime Jingle Hemmett Schmidt.

Dublin was lovely, lovely -- performing for us in weather and in song, in light and dark -- the dark being the fine stout, of course. I couldn't pick a favorite part, though the one that keeps popping into mind is Newgrange, a 5000-year-old neolithic passage tomb, forty-some meters in diameter, aligned so that a shaft of light would pass into the chamber on the winter solstice.

That image aligns nicely with my Mumford readings of late. Before the construction of such artifacts, either as sticks planted in the ground or as massive monuments, time did not exist -- at least, not as we know it now, as an outside thing ticking on discretely without us -- as opposed to the spurting yet continuous flow experienced within.

I spent the rest of last week hanging out in deserted August Paris, the city-time of open-air cinema and ambles in unpeopled streets. But I won't lie: I spent a fair piece of it indoors, hack on the mind.

travels through time

At my last dispatch, I was in 1987, implementing flat closures, described in Dybvig's dissertation.

The recent spatial travel to Dublin, unaccompanied by the laptop, left my mind unusually clear. So spatially back in Paris on Wednesday I left again for 2002, implementing Fixing Letrec. The important contribution of that paper was a systematic, direct way to translate mutually recursive lambda expressions to a generalized form of the fixed-point operator, also known as the Y combinator. Quoth the prophets Waddell, Sarkar & Dybvig:

The transformation converts letrec expressions into an equivalent mix of let, set!, and fix expressions. A fix expression is a variant of letrec that binds only unassigned variables to lambda expressions. It represents a subset of letrec expressions that can be handled easily by later passes of a compiler.

In particular, no assignments through external variables are necessary to implement mutually recursive procedures bound by fix. Instead, the closures produced by a fix expression can be block allocated and “wired” directly together. This leaves the fix-bound variables unassigned, thus simplifying optimizations such as inlining and loop recognition.

fix is identical to the labels construct handled by Steele’s Rabbit compiler and the Y operator of Kranz’s Orbit compiler and Rozas’ Liar compiler.

For me, this passage was tantalizing, but I didn't quite understand the optimizations that a transformation to fix allowed. Yes, I had read Steele's dissertation a number of times, but Guile's compiler doesn't do a CPS rewrite, so it was unclear how some of the optimizations applied.

But my recent "flat closure" work had pointed out one optimization, the "block allocation" mentioned in the above paragraph. Consider two mutually recursive functions:

  (define (analyze n)
    (define (even? n)
      (or (= n 0)
          (odd? (- n 1))))
    (define (odd? n)
      (or (= n 1)
          (even? (- n 1))))

    (cond ((< n 0)   'negative-number)
          ((even? n) 'even-number)
          (else 'odd-number)))

Ignore for the moment the dubious "analysis" that this function performs. (Can you spot the bug?) What I want to focus on are the internal definitions, even? and odd?. Using define in a nested context like this is simply sugar around letrec, so this is equivalent to:

  (define (analyze n)
    (letrec ((even? (lambda (n)
                      (or (= n 0)
                          (odd? (- n 1)))))
             (odd?  (lambda (n)
                      (or (= n 1)
                          (even? (- n 1))))))
      (cond ((< n 0)   'negative-number)
            ((even? n) 'even-number)
            ((odd? n)  'odd-number)))

Here we see the mutual recursion expressed more clearly. Looking more closely at the even? and odd? bindings, we see that each has one free variable:

  (lambda (n)
    (or (= n 0)
        (odd? (- n 1))))
          ^ odd? is free in even?

  (lambda (n)
    (or (= n 1)
        (even? (- n 1))))
          ^ even? is free in odd?

If even? and odd? are compiled as mutually recursive closures (a point to which I will return shortly), they need to capture each other's bindings -- kindof a chicken-and-egg problem, no?

In the lambda calculus, this problem is solved via the Y combinator, which basically involves passing the functions of interest to another function, which in turn invokes the functions of interest, passing themselves as their arguments. It's pretty neat. Thomas Holubar and Jos Koot have been discussing this very topic at FLIBUG, incidentally (see Thomas's slides on the need for Y for a brief introduction).

origins of letrec

To continue the digression, I understand that the letrec construct was originally defined by Peter Landin in a 1965 paper, A Correspondence between ALGOL 60 and Church's Lambda-Notation. He uses a very schemely intermediate language to model ALGOL 60's semantics with sugar on top of a slightly extended lambda calculus.

In the following discussion, from the paper, Landin describes how mutually recursive definitions and labels (goto targets) can be expressed using letrec. Take φn to mean an arbitrary expression; italics are mine.

Definitions can also arise from the block-body -- their definees being the labels that are local to the block. These are defined in terms of locals, including each other, and they may be referred to by procedure and switch declarations. Hence labels must be grouped with switches and procedures as a single simultaneously recursive definition. The overall treatment of a block is therefore as follows.

[On the left we have ALGOL 60, on the right Landin's "Applicative Expression" intermediate language.]

begin                     let a=φ1
    real a;                   and A=φ2
    array A φ2;             let rec P=φ3
    procedure P φ3;                 and S=φ4
    switch S φ4;                    and L=φ6
    φ5;                             and M=φ7
 L: φ6;                       φ5            
 M: φ7;
end

[Here is the translation into the lambda calculus. Note the Y!]

{λ(a,A).{λ(P,S,L,M).φ5}
        [Yλ(P,S,L,M).(φ3,φ4,φ6,φ7)]}
[φ1,φ2]

Applying Y to mutually recursive procedures/labels binds them together, allowing them to see themselves. So you see, Y was in letrec from the very beginning.

Note that labels, denoting basic blocks, are modelled as procedures of 0 arguments. In the light of Steele's 1977 contribution, which I will discuss shortly, Landin's 1965 paper was particularly prescient. Landin says:

The treatment of jumps springs from the observation that the symbol 'go to' in ALGOL 60 is redundant, and could be ignored by a preprocessor. That is to say, there is a considerable similarity between labels and the identifiers of parameterless nontype procedures. It is possible to use the same "calling mechanism" for both, leaving any difference to be made by the thing that is "called"...

It might therefore be supposed that labels can be eliminated formally by considering each labelled segment of program as a parameterless procedure declaration (and hence as a definition whose defiens is a λ()-expression).

Steele turns this isomorphism around, saying instead:

In general, procedure calls may be usefully thought of as GOTO statements which also pass parameters, and can be uniformly encoded as JUMP instructions. This is a simple, universal technique... Our approach results in all tail-recursive procedure calls being compiled as iterative code, almost without trying, for it is more a matter of the code generation strategy than of a specific attempt to remove recursions.

Landin passed away in June of this year.

mischief managed

Anyway, back to 2002. Scheme's letrec construct is more general than Landin's let rec; it allows arbitrary expressions on the right-hand side, not just procedures. So the first trick Waddell shows is how to transform letrec into "an equivalent mix of let, set!, and fix". Once he has the fix, which is the same as Y, we need to compile it -- and that's where we were before longjmping back to 1965.

In the lambda calculus, a very minimal language, often there are more efficient ways to implement higher-order constructs, such as fix.

So fix-bound procedures need to capture an environment including themselves, fine. The standard way (in Scheme now, not the lambda calculus) is to bind e.g. even? and odd? (as in our example above) to empty "boxes", then set! the contents of the boxes to the procedures. The fix-bound lambda-expressions have bound the right variables, and after the boxes have been filled in via set!, those expressions hold each other's value: mischief managed. Like this:

(let ((even? #f) (odd? #f))
  (set! even? (lambda (n) (or (= n 0) (odd? (- n 1)))))
  (set! odd? (lambda (n) (or (= n 1) (even? (- n 1)))))
  ...)

But this is not ideal, because it forces the fix-bound lambda expressions to be allocated on the heap, in boxes, whereas in fact they are never set! in the source text.

We can do better. Since the lambda expressions don't actually run until after the bindings have been made, and thus don't reference their free bindings until then, we can allocate them on the stack, then fix up their free variables, mutating the closures in place.

Thus we don't introduce extraneous set! constructs into the code. This simplifies inlining and loop detection, as Waddell notes, and also allows the fix-bound variables to be allocated on the stack instead of in boxes, removing an indirection at runtime.

even better

But we can do even better than that. In this example, we still allocate a closure for even? and for odd? -- but we can avoid that too. Notice that even? and odd? are always called in tail position with respect to the letrec construct. Landin tells us that labels, the targets of go to, may be viewed as procedures of 0 arguments, with respect to the lambda calculus; Steele goes the other way, in his 1977 paper, Debunking the 'Expensive Procedure Call' Myth, or, Procedure Call Implementations Considered Harmful, or, Lambda: The Ultimate GOTO. (If you program, and want just one article to read out of all of these, read this paper.)

So, for a tail-call to a fix-bound procedure, we can simply render the bindings of the procedures inline to the parent procedure, jump over them, then enter the body of the fix. Any tail-call to a fix-bound procedure compiles a goto -- after setting its arguments. Since the procedure is lexically bound, and never set!, we know exactly where those arguments are going to be, so we set them directly -- no need to shuffle them around.

So an empty loop:

(letrec ((loop (lambda () (loop))))
  (loop))

Looks like this:

   ;; jump over definition of `loop'
   0    (br :L124)                      ;; -> 16
   ;; definition of `loop'
   8    (br :L125)                      ;; -> 8
   ;; letrec body
  16    (br :L125)                      ;; -> 8

(You get this by typing ,x (lambda () (let loop () (loop))) into the Guile REPL.)

A loop counting down from 100 looks like this:

scheme@(guile-user)> ,x (lambda ()
                          (let lp ((n 100))
                            (or (zero? n)
                                (lp (1- n)))))
Disassembly of #<program b742e6d0 at <unknown port>:51:3 ()>:

   0    (br :L141)                      ;; -> 32
   8    (local-ref 0)                   ;; `n'
  10    (make-int8:0)                   ;; 0
  11    (ee?)                           
  12    (local-set 1)                   ;; `t'
  14    (local-ref 1)                   ;; `t'
  16    (br-if-not :L142)               ;; -> 24
  19    (local-ref 1)                   ;; `t'
  21    (return)                        
  24    (local-ref 0)                   ;; `n'
  26    (sub1)
  27    (local-set 0)                   ;; `n'
  29    (br :L143)                      ;; -> 8
  32    (make-int8 100)                 ;; 100
  34    (local-set 0)                   
  36    (br :L143)                      ;; -> 8

I mean, that's pretty damn good. We could do better if we reordered the blocks a bit, and compiled the conditional branch br-if-not into something more specific, but still -- pretty tight for compiling scheme to bytecode.

(The `t', if you are curious, is a result of the expansion of or.)

conclusions

Guile has applied Waddell's "Fixing Letrec" strategy to transform Scheme's general letrec into more primitive constructs, including fix. fix-bound lambda expressions that need to be allocated as closures are now faster and cons less, given that they aren't allocated in boxes, and an important subset of fix expressions is now rendered as inline code and wired together using goto, a pleasant illustration that indeed, lambda is the ultimate goto.

I didn't think I would get to this kind of bytecode this fast, but once the simplification to fix was made, the rest of the optimizations were obvious. But the hack continues, there's always more to do. Expect to see this out in Guile's 1.9.2 release that's coming on Saturday. Happy hacking!

Andy Wingohttps://wingolog.org/the good hackhttps://wingolog.org/2009/07/26/the-good-hack2009-07-26T21:48:06Z2009-07-26T21:48:06Z

Good evening, internet!

I haven't written in a little while, and I suppose a hack-update is due. So first an overview, then a digression into nargery, then a discussion of what's really on my mind these days: Mumford and Eisenstein.

Hup hup, then!

guile

Guile is now on a monthly release schedule, in the runup to 2.0. We've already made two releases, 1.9.0 and 1.9.1.

The feedback has generally been that it works, that it is faster, but that in a few cases, some old macros have to change. The NEWS describe all the gory details, so I'll spare you them here.

Still though, I'm really proud for what we've been able to do: retrofit a compiler on top of a large base of informally defined interpreter semantics, expressed in code afloat on the internet. It's a pretty momentous refactoring effort.

I spoke at GUADEC briefly about Guile, in the impromptu lightning talk session. The story I really wanted to get out was not the details of Guile's compiler, but that Scheme is just one element of Guile. Elisp can exist there too, and Daniel Kraft's elisp branch seems to really be shaping up.

Of course, that's not to mention Daniel's brainfuck compiler, amusingly already merged as an example:

scheme@(guile-user)> ,L brainfuck
Guile Brainfuck interpreter 1.0 on Guile 1.9.0
Copyright (C) 2001-2008 Free Software Foundation, Inc.

Enter `,help' for help.
brainfuck@(guile-user)> +++ +++ +++ +           initialize counter (cell #0) to 10
... [                       use loop to set the next four cells to 70/100/30/10
...     > +++ +++ +             add  7 to cell #1
...     > +++ +++ +++ +         add 10 to cell #2 
...     > +++                   add  3 to cell #3
...     > +                     add  1 to cell #4
...     <<< < -                 decrement counter (cell #0)
... ]                   
... >++ .                   print 'H'
... >+.                     print 'e'
... +++ +++ +.              print 'l'
... .                       print 'l'
... +++ .                   print 'o'
... >++ .                   print ' '
... <<+ +++ +++ +++ +++ ++. print 'W'
... >.                      print 'o'
... +++ .                   print 'r'
... --- --- .               print 'l'
... --- --- --.             print 'd'
... >+.                     print '!'
... >.                      print '\n'
... 
... ]
Hello World!
brainfuck@(guile-user)>

The example is from wikipedia. You can see that the brainfuck reader actually integrates with readline. Since brainfuck programs are normally terminated by end-of-file, we provide an alternate terminal: if ] (the loop terminator) is seen while not inside a loop, we stop reading the brainfuck expression.

Actually, I have to use a similar trick with the ECMAScript reader -- while the grammar does not require it in all situations, you have to terminate expressions entered at the REPL with a semicolon.

Anyway! Guile progresses apace. My favorite addition from the last release was a set of vector and bytevector instructions to the VM. Probably the biggest effect for existing users is that vector-ref and vector-set have their own VM operations now, and are thus much faster.

But the bit that I like is that there are now VM ops for bytevector refs and sets for packed values: everything from unsigned 8-bit values to 64-bit floats. It should make large-scale data processing feasible within Guile's VM, and it will make native compilation for those operations fairly direct and efficient.

Finally, hackwise, I implemented what Kent Dybvig calls "display closures" for Guile.

Before, all heap-allocated values (values that were captured by other procedures and values that were set!) were allocated in a linear list, and closure creation just mean consing together that list of heap variable state with your program code. You can think of this as a degenerate case of the ribcage strategy.

I guess I should be a little clearer. Consider this function, which should add a common prefix to all strings in a list:

(define (munge-strings list prefix)
  (map (lambda (x)
         (string-append prefix x))
       list))

Semantically, you could identify the variables with numbers and it would be the same:

(define (munge-strings <0,0> <0,1>)
  (map (lambda (<1,0>)
         (string-append <0,1> <1,0>))
       <0,0>))

Here the first index refers to the procedure in which the variable is bound, and the second is the nth variable in that procedure. Well this leads itself to a natural implementation of closures: in the function that is mapped, just capture the environment at that point -- then you can get to prefix just by looking back to the first variable in the 0th frame.

But you can do better than that. A variable that's not referenced in an enclosed lambda can just be allocated on the stack -- so list goes on the stack. prefix stays on the heap, though. And that's where Guile was...

...until I realized, via reading Dybvig's 20-year-old dissertation, that I didn't actually have to have environment structures on the heap at all. Closures could just capture the variables that they need into a vector for their own use.

Shared variables can be copied, there's no problem -- except if those variables are ever set!. So in that case, a condition you can determine lexically, you need to allocate those variables in "boxes". Getting and setting values is indirected through the boxes, but those boxes can live whereever -- on the stack in the function that binds the variables, or in the free variables vectors of closures.

This scheme has the advantage that more values can be allocated on the stack, free variable lookup is O(1), and you don't tie the garbage collector's anthropomorphic hands by holding onto parts of the environment that you don't need.

So that's the novelty for the upcoming 1.9.2 release, slated for 15 August. Then we'll have a 1.9.3 in the following month, and 2.0 in October. Like clockwork :)

like clockwork

Lyn Gerry continues her reading of The Ascent of Humanity on her radio show, Unwelcome Guests. It's fascinating, and challenging.

Charles Eisenstein, the author of The Ascent of Humanity, owes a large debt to Lewis Mumford, a historian of technology -- or "technics", as seems to be his preferred word. Eisenstein quotes Mumford liberally. It is by a happy coincidence that a friend of mine pushed Mumford's Technics and Civilization on me a couple months ago -- in exchange for Expect Resistance -- and it is shaping up to be a great, great work.

Regarding "clockwork", as a phenomenon and as a paradigm, Mumford has this to say:

The clock, moreover, is a piece of power-machinery whose "product" is seconds and minutes: by its essential nature it dissociated time from human events and helped create the belief in an independent world of mathematically measurable sequences: the special world of science. There is relatively little foundation for this belief in common human expreience: throughout the year the days are of uneven duration, and not merely does the relation between day and night steadily change but a slight journey from East to West alters astronomical time by a certain number of minutes. In terms of the human organism itself, mechanical time is even more foreign: while human life has regularities of its own, the beat of the pulse, the breathing of the lungs, these change from hour to hour with mood and action, and in the longer span of days, time is measured not by the calendar but by the events that occupy it. The shepherd measures from the time the ewes lambed; the farmer measures back to the day of sowing or forward to the harvest: if growth has its own duration and regularities, behind it are not simply matter and motion but the facts of development: in short, history.

In Reuleaux's definition, quoted by Mumford, a machine is "a combination of resistant bodies so arranged that by their means the mechanical forces of nature can be compelled to do work accompanied by certain derminant motions." But those bodies need not be mineral in nature; indeed, Mumford identifies the human industrialism necessary for the Great Works of the Pyramids as one of the first steps into the machine age: the reduction of man to the "resistant body" of the machine.

So it's interesting to think about the relationship of timekeeping to the mechanization of the human. The ringing of the bells, whether via water-clock or the dawn's early light on the crier's face, introduced discrete time to humanity, and so human activity began to be fit to those times -- cutting off the legs to to fit the bed, so to speak. Mumford continues:

Abstract time became the new medium of existence. Organic functions themselves were regulated by it: one ate, not upon feeling hungry but when prompted by the clock: one slept, not when one was tired, but when the clock sanctioned it. A generalized time-consciouesness accompanied the wider use of clocks: dissociating time from organic sequences, it became easier for the men of the Renascence to indulge the fantasy of reviving the classic past or of reliving the splendors of antique Roman civilization: the cult of history, appearing first in daily ritual, finally abstracted iself as a special discipline.

I've been having lots of dreams about Namibia recently. A common time reference where I lived there is etango peni: "sun where", literally. It's usually indicated with an outstretched arm pointing to where the sun should be, as in, we'll meet this afternoon when the sun is there.

I guess my concern is, to what extent is our obedience to the clock and to the calendar a life-affirming concern, and on the other hand, to what extent does it reduce us to "resistant bodies"? Do we exist in time, or do we exist in a use/used relationship -- to time, and to those that mark the time?

To tie it back to the more nargy concerns above, I think that time-based releases do have a positive effect on the Guile machine. And the Guile machine is a liberatory one -- in whatever trade, programming or otherwise, we should not grind our grist at the lord's mill.

But the greater questions that Mumford and Eisenstein raise make me wonder further about Guile work, not just relative to other systems, or relative to my desire to hack, but relative to the story of humanity -- Ascent or otherwise -- what does furthering this technology really do for life?

Unfortunately, I have no answers for that one. I'll let the tubes know if I find out. Until then, or likely sooner, happy hacking, for Life!

Andy Wingohttps://wingolog.org/poverty, richeshttps://wingolog.org/2009/05/14/poverty-riches2009-05-14T19:31:43Z2009-05-14T19:31:43Z

riches, poverty

How many times have you heard the phrase, "the poorest of the poor, those that live on less than a dollar a day"?

I am tired of it.

You can grow your own food, and they call you "poor". Or you can buy transgenic, pesticide-laced, flavorless tomatoes at the store, and they call you "rich".

Poverty and riches do exist, but they have nothing to do with dollars.

hack reading

Fast and Effective Procedure Inlining, by Oscar Waddell and Kent Dybvig.

Great, great paper. It could be my ignorance, but I never realized that copy propagation, dead code elimination, constant folding, and procedure inlining were all the same operation.

I think I'm going to implement their algorithm soon. It should be straightforward, given that the new high-level intermediate language that I'm working on for Guile is basically the same as their language.