guile 2.2 omg!!!

15 March 2017 10:56 PM (guile | scheme | gnu | igalia | compilers | academia | relief)

Oh, good evening my hackfriends! I am just chuffed to share a thing with yall: tomorrow we release Guile 2.2.0. Yaaaay!

I know in these days of version number inflation that this seems like a very incremental, point-release kind of a thing, but it's a big deal to me. This is a project I have been working on since soon after the release of Guile 2.0 some 6 years ago. It wasn't always clear that this project would work, but now it's here, going into production.

In that time I have worked on JavaScriptCore and V8 and SpiderMonkey and so I got a feel for what a state-of-the-art programming language implementation looks like. Also in that time I ate and breathed optimizing compilers, and really hit the wall until finally paging in what Fluet and Weeks were saying so many years ago about continuation-passing style and scope, and eventually came through with a solution that was still CPS: CPS soup. At this point Guile's "middle-end" is, I think, totally respectable. The backend targets a quite good virtual machine.

The virtual machine is still a bytecode interpreter for now; native code is a next step. Oddly my journey here has been precisely opposite, in a way, to An incremental approach to compiler construction; incremental, yes, but starting from the other end. But I am very happy with where things are. Guile remains very portable, bootstrappable from C, and the compiler is in a good shape to take us the rest of the way to register allocation and native code generation, and performance is pretty ok, even better than some natively-compiled Schemes.

For a "scripting" language (what does that mean?), I also think that Guile is breaking nice ground by using ELF as its object file format. Very cute. As this seems to be a "Andy mentions things he's proud of" segment, I was also pleased with how we were able to completely remove the stack size restriction.

high fives all around

As is often the case with these things, I got the idea for removing the stack limit after talking with Sam Tobin-Hochstadt from Racket and the PLT group. I admire Racket and its makers very much and look forward to stealing fromworking with them in the future.

Of course the ideas for the contification and closure optimization passes are in debt to Matthew Fluet and Stephen Weeks for the former, and Andy Keep and Kent Dybvig for the the latter. The intmap/intset representation of CPS soup itself is highly endebted to the late Phil Bagwell, to Rich Hickey, and to Clojure folk; persistent data structures were an amazing revelation to me.

Guile's virtual machine itself was initially heavily inspired by JavaScriptCore's VM. Thanks to WebKit folks for writing so much about the early days of Squirrelfish! As far as the actual optimizations in the compiler itself, I was inspired a lot by V8's Crankshaft in a weird way -- it was my first touch with fixed-point flow analysis. As most of yall know, I didn't study CS, for better and for worse; for worse, because I didn't know a lot of this stuff, and for better, as I had the joy of learning it as I needed it. Since starting with flow analysis, Carl Offner's Notes on graph algorithms used in optimizing compilers was invaluable. I still open it up from time to time.

While I'm high-fiving, large ups to two amazing support teams: firstly to my colleagues at Igalia for supporting me on this. Almost the whole time I've been at Igalia, I've been working on this, for about a day or two a week. Sometimes at work we get to take advantage of a Guile thing, but Igalia's Guile investment mainly pays out in the sense of keeping me happy, keeping me up to date with language implementation techniques, and attracting talent. At work we have a lot of language implementation people, in JS engines obviously but also in other niches like the networking group, and it helps to be able to transfer hackers from Scheme to these domains.

I put in my own time too, of course; but my time isn't really my own either. My wife Kate has been really supportive and understanding of my not-infrequent impulses to just nerd out and hack a thing. She probably won't read this (though maybe?), but it's important to acknowledge that many of us hackers are only able to do our work because of the support that we get from our families.

a digression on the nature of seeking and knowledge

I am jealous of my colleagues in academia sometimes; of course it must be this way, that we are jealous of each other. Greener grass and all that. But when you go through a doctoral program, you know that you push the boundaries of human knowledge. You know because you are acutely aware of the state of recorded knowledge in your field, and you know that your work expands that record. If you stay in academia, you use your honed skills to continue chipping away at the unknown. The papers that this process reifies have a huge impact on the flow of knowledge in the world. As just one example, I've read all of Dybvig's papers, with delight and pleasure and avarice and jealousy, and learned loads from them. (Incidentally, I am given to understand that all of these are proper academic reactions :)

But in my work on Guile I don't actually know that I've expanded knowledge in any way. I don't actually know that anything I did is new and suspect that nothing is. Maybe CPS soup? There have been some similar publications in the last couple years but you never know. Maybe some of the multicore Concurrent ML stuff I haven't written about yet. Really not sure. I am starting to see papers these days that are similar to what I do and I have the feeling that they have a bit more impact than my work because of their medium, and I wonder if I could be putting my work in a more useful form, or orienting it in a more newness-oriented way.

I also don't know how important new knowledge is. Simply being able to practice language implementation at a state-of-the-art level is a valuable skill in itself, and releasing a quality, stable free-software language implementation is valuable to the world. So it's not like I'm negative on where I'm at, but I do feel wonderful talking with folks at academic conferences and wonder how to pull some more of that into my life.

In the meantime, I feel like (my part of) Guile 2.2 is my master work in a way -- a savepoint in my hack career. It's fine work; see A Virtual Machine for Guile and Continuation-Passing Style for some high level documentation, or many of these bloggies for the nitties and the gritties. OKitties!

getting the goods

It's been a joy over the last two or three years to see the growth of Guix, a packaging system written in Guile and inspired by GNU stow and Nix. The laptop I'm writing this on runs GuixSD, and Guix is up to some 5000 packages at this point.

I've always wondered what the right solution for packaging Guile and Guile modules was. At one point I thought that we would have a Guile-specific packaging system, but one with stow-like characteristics. We had problems with C extensions though: how do you build one? Where do you get the compilers? Where do you get the libraries?

Guix solves this in a comprehensive way. From the four or five bootstrap binaries, Guix can download and build the world from source, for any of its supported architectures. The result is a farm of weirdly-named files in /gnu/store, but the transitive closure of a store item works on any distribution of that architecture.

This state of affairs was clear from the Guix binary installation instructions that just have you extract a tarball over your current distro, regardless of what's there. The process of building this weird tarball was always a bit ad-hoc though, geared to Guix's installation needs.

It turns out that we can use the same strategy to distribute reproducible binaries for any package that Guix includes. So if you download this tarball, and extract it as root in /, then it will extract some paths in /gnu/store and also add a /opt/guile-2.2.0. Run Guile as /opt/guile-2.2.0/bin/guile and you have Guile 2.2, before any of your friends! That pack was made using guix pack -C lzip -S /opt/guile-2.2.0=/ guile-next glibc-utf8-locales, at Guix git revision 80a725726d3b3a62c69c9f80d35a898dcea8ad90.

(If you run that Guile, it will complain about not being able to install the locale. Guix, like Scheme, is generally a statically scoped system; but locales are dynamically scoped. That is to say, you have to set GUIX_LOCPATH=/opt/guile-2.2.0/lib/locale in the environment, for locales to work. See the GUIX_LOCPATH docs for the gnarlies.)

Alternately of course you can install Guix and just guix package -i guile-next. Guix itself will migrate to 2.2 over the next week or so.

Welp, that's all for this evening. I'll be relieved to push the release tag and announcements tomorrow. In the meantime, happy hacking, and yes: this blog is served by Guile 2.2! :)

70 responses

  1. rain1 says:

    Congratulations! Thank you for keeping the wonderful scheme language alive and well! The guile compiler is a fantastic tool both for the practical getting work done and the theoretical - learning scheme/programming/compiler implementation details.

  2. John Cowan says:

    This is good news, because I am about to reinstall my large suite of Schemes on my home system using Ubuntu on Windows, so I'll hold off on installing Guile. I do hope I don't have to wait hours, like I did the last time I installed Guile from source.

  3. Julian Graham says:

    Andy, you rock! Thanks so much for all your hard work on this. Can't wait to start using the official release.

  4. Javier Sancho says:

    Thank you so much, Andy! Your work and your posts are undoubtedly taking us to the top.

  5. gasche says:

    I think you should seriously consider writing an academic article about Guile, as a way indeed to present and transfer your knowledge and design choices for the system. For example, the deadline for article submission for the next "Trends of Functional Programming" symposium is on May 5 (the conference is June 19-22 in Canterbury, UK), and looking at last year's programme ( ) could give you an idea of what the conference feels like -- it is more open to implementation-oriented works than some other conferences, which is why I thought of it as a reasonable first step. Surely you could enlist the help of some colleagues that were recently in academia or still there for the "how do I structure/write a damn paper?" and "how do I get feedback on my draft?" parts.

  6. gasche says:

    Sigh, frames;

  7. Charles says:

    Congratulations on 2.2! It's been a pleasure to watch Guile improve under your watch and see the new applications it has found as a result. I look forward to trying out the latest. Also, thank you for all the informative and edifying blog posts on language design and implementation.

  8. amz3 says:

    Thanks for this awesome release. I am not not a compiler/VM dev just a user but AFAIK Guile offers what's required to build software. It's seems to me Guile 2.2 is similar in terms of features compared to CPython 3.6 except that Guile doesn't have a GIL.

    Guile only lakes more users :)

  9. RobertX says:

    How does Windows fit into this? Is it even a target?

  10. Johnny says:

    I just looked through ecraven's benchmarks. Sure they are benchmarks, and they onlt show a part of the truth, but consistently being AT LEAST 2x faster must mean something :)

    Well done!

  11. Adam says:

    When you talk about "the multicore Concurrent ML stuff" that you have yet to write about, is that stuff that is publicly available in your fibers repoo, or something else? In the ML world, this has been the subject of ongoing research but I'm curious what you've done.

  12. wingo says:

    Heya Adam,

    I understand that Manticore is a working multicore CML system. The work I have is in Fibers and is mostly in operations.scm, and pretty much follows Manticore. See also channels.scm in that same directory, and internal.scm for the scheduler implementation. In Fibers we coalesce the "poll" and "do" phases into a single "try" phase; otherwise it's pretty much the same as Manticore, though the channel protocol is atomic (no spinlocks). I need to do some more benchmarking and will blog about that sometime soon :)


  13. Adam says:

    Andy - Thanks! Really interesting stuff, hope you blog about it soon. I'm hoping more people learn about CML since it solves a lot of the problems I have as a UI programmer.

  14. Happy Room Play says:

    Andy, you rock! Thanks so much for all your hard work on this. Can't wait to start using the official release...

  15. mats says:

    I looked through the benchmarks by ecraven, and the speedup is incredible! Ported a bunch of python things (project euler solutions) to guile2.2 and had an instant 3-10x speedup.

  16. Bola says:

    Great post, This is a good news,I am about to reinstall my large suite of Schemes too on my home system using Ubuntu on Windows,

  17. Cartoon HD for BlackBerry says:

    Cartoon HD is the best app I have come across in recent years. It is heartening to note that these apps are free of cost and also give us the best offers without any cost.
    In today's Internet, it is essential to note that nothing is gratis. For more details check the process of download

  18. new futura condo says:

    This content is written very well. Your use of formatting when making your points makes your observations very clear and easy to understand. Thank you.

  19. increase font size windows 10 says:

    I may see this happening about guile.

  20. LadyFem says:

    Good info of 2.2, can't wait for more useful writing of you

  21. snow cone says:

    Thanks for your post. I’ve been thinking about writing a very comparable post over the last couple of weeks. I’ll probably keep it short and sweet and link to this instead if thats cool. Thanks.

  22. baby shopping list says:

    Thanks for every other informative site. The place else may just I get that kind of information written in such an ideal means? I have a venture that I’m just now operating on. and I have been on the look out for such information.

  23. Mactan Newtown says:

    Good post but I was wondering if you could write a litte more on this subject? I’d be very thankful if you could elaborate a little bit further. Appreciate it!

  24. best cat dewormer says:

    Cool stuff you have got and you keep update all of us.

  25. online Payday Loans Chula vista says:

    Since they represent considerable authority in these credits, it is typically a simple, quick procedure to get the cash that you require. Obviously, there is typically an expense that is charged for the credit.

  26. calcolare la percentuale says:

    That is really nice to hear. thank you for the update and good luck.

  27. Basement Renovation Contractors Near Yardley, PA says:

    Thanks for a wonderful share. Your article has proved your hard work and experience you have got in this field. Brilliant .i love it reading.

  28. Albuquerque Divorce Attorney says:

    That is really nice to hear. thank you for the update and good luck.

  29. Hairfluence All Natural Hair Growth says:

    Cool stuff you have got and you keep update all of us.

  30. car alarm installation Miami says:

    Cool stuff you have got and you keep update all of us.

  31. Best Shoes Coupons says:

    Great knowledge. do anyone mind merely reference back to it

  32. best anti snoring chinstrap says:

    Your content is nothing short of brilliant in many ways. I think this is engaging and eye-opening material. Thank you so much for caring about your content and your readers.

  33. Assassins Creed Unity Outfits says:

    Webkit is an HTML rendering motor utilized by Chrome and Safari. Webkit is the HTML/CSS rendering motor utilized as a part of Apple's Safari program and in Google's Chrome. CSS esteems prefixes with - WebKit-will is WebKit-particular, they're typically CSS3 or other non-institutionalized highlights.

  34. Biker Leather Jacket says:

    Thanks for sharing this piece of information. I really enjoyed it. keep up the good work and all the very best of luck!

  35. google bot says:

    Hey there thanks for sharing

  36. Website says:

    Check out the website and you will learn how to get more Instagram followers

  37. wd says:

    thank you so much.

  38. Steven says:

    Certification Provides the Pdf Resource, Practicing Software or another resources for CCNP Security Certification.

  39. neurologists near me says:

    It sounds like a hack there is a program called pokesav that allows you to edit pokemon with the use of an action replay

  40. SpotLoan says:

    The borrower is given the option of paying low monthly installments for a longer duration or can take up the option of paying higher than stipulated to finish off the loan much faster thereby saving more by avoiding paying the interest. As the name goes, this loan can be derived with the help of your signature. The loaned amount also depends on some other factors too, such as your repayment capability and your financial status as well.

  41. carpet cleaning St Petersburg says:

    very impressive post, have a good day!

  42. slither io says:

    Thanks for the things you share, we always like this to build a wonderful community that is a place to share knowledge, communicate with everyone.

  43. kitchen remodel says:

    I do not even know how I ended up here, but I thought this post was great.

  44. Marketing Project Help says:

    I Personally Like Your Post, You Have Shared Good Article. It Will Help Me In Great Deal.

  45. adamschule85 says:

    The changes are not so drastic to warrant such a publicity. forebet

  46. Matthew Carter says:

    Great read - I appreciate your work on Guile.

  47. website builder says:

    hackfriends expression is kinda funny

  48. IDN Poker says:

    Thanks for the information on this. I really enjoy the write-up.

  49. online essay writer says:

    Your writing ability is amazing. A good article should include all the relevant data for the topic in a gentle way so that the readers can get your thought easily. In this article, you have arranged all your ideas in a pleasant way. I am really impressed with that. I like to refer your site to get good topics to read. Almost I have all your posts and this one is the best. This article has clarified the ideas in an interesting way. Keep the same quality in the rest of the posts which is the best thing I have got from your articles. Thank you.

  50. Booksrush says:

    Great post loved it

  51. vex 3 online says:

    Great article. It helps a lot for my work. Thanks for sharing these effective and helpful ways.

  52. onmovies apk says:

    Thank you for sharing article and information. this is very helpful for beginner.

  53. topstudytips says:

    Thank you for sharing article and information. this is very helpful for beginner.

  54. independent animation studios says:

    Thanks for the information on this.

  55. Hydro Excavator Truck says:

    Thank you for sharing article and information.

  56. temple run 3 says:

    Your article is very useful, the content is great, I have read a lot of articles, but for your article, it left me a deep impression, thank you for sharing.

  57. London Escort says:

    I was truly searching for such kind of article and I have established it here. Incredible work sir and prop up ahead. Love your work!

  58. Sell My House Fast Lexington KY says:

    Really helpful article. Thanks a lot for sharing.

  59. Seamless Siding says:

    Thank you very much for sharing

  60. Pittsburgh DJ Services says:

    Very helpful, thank you for sharing

  61. educational therapy certification says:

    Our expert consultants can help promote smooth and effective adjustments by partnering with parents and schools to bring accurate, responsive and meaningful support strategies in place.

  62. Savita BHabhi says:

    This site certainly has the majority of the data and facts.Thank you for sharing this article. I would like to get more posts this way.

  63. Dehradun Escorts says:

    I can see that you are an expert at your field. Your information will be very useful for me. Thanks for all your help and wishing you all the success in your business.

  64. Dehradun Escorts says:

    I can see that you are an expert at your field. Your information will be very useful for me. Thanks for all your help and wishing you all the success in your business.

  65. Dehradun Escort says:

    This site certainly has the majority of the data and facts.Thank you for sharing this article. I would like to get more posts this way.

  66. Dehradun Call Girls says:

    This is such an incredible stuff, that you are giving and you give it away for nothing. I cherish seeing web journal that comprehend the benefit of giving a quality asset to free.

  67. word counter says:

    Share your fullest content I ever read, I will become a fan of you

  68. visakhapatnam escorts says:

    thanks for sharing this post very nice article.

  69. escorts in visakhapatnam says:

    very useful artile thanks for posting

  70. visakhapatnam call girls says:

    this is such a incredible stuff, that you are givving and you give it.

Leave a Reply