generators in v8

8 May 2013 7:31 PM (v8 | ecmascript | javascript | igalia | compilers | delimited continuations | continuations | promises)

Hey y'all, ES6 generators have landed in V8! Excellent!

Many of you know what that means already, but for those of you that don't, a little story.

A few months ago I was talking with Andrew Paprocki over at Bloomberg. They use JavaScript in all kinds of ways over there, and as is usually the case in JS, it often involves communicating with remote machines. This happens on the server side and on the client side. JavaScript has some great implementations, but as a language it doesn't make asynchronous communication particularly easy. Sure, you can do anything you want with node.js, but it's pretty easy to get stuck in callback hell waiting for data from the other side.

Of course if you do a lot of JS you learn ways to deal with it. The eponymous Callback Hell site lists some, and lately many people think of Promises as the answer. But it would be nice if sometimes you could write a function and have it just suspend in the middle, wait for your request to the remote computer to come through, then continue on.

So Andrew asked if me if we could somehow make asynchronous programming easier to write and read, maybe by implementing something like C#'s await operator in the V8 JavaScript engine. I told him that the way into V8 was through standards, but that fortunately the upcoming ECMAScript 6 standard was likely to include an equivalent to await: generators.

ES6 generators

Instead of returning a value, when you first call a generator, it returns an iterator:

// notice: function* instead of function
function* values() {
  for (var i = 0; i < arguments.length; i++) {
    yield arguments[i];

var o = values(1, 2, 3);  // => [object Generator]

Calling next on the iterator resumes the generator, lets it run until the next yield or return, and then suspends it again, resulting in a value:; // => { value: 1, done: false }; // => { value: 2, done: false }; // => { value: 3, done: false }; // => { value: undefined, done: true }

Maybe you're the kind of person that likes imprecise, incomplete, overly abstract analogies. Yes? Well generators are like functions, with their bodies taken to the first derivative. Calling next integrates between two yield points. Chew on that truthy nugget!


Anyway! Supending execution, waiting for something to happen, then picking up where you left off: put these together and you have a nice facility for asynchronous programming. And happily, it works really well with promises, a tool for asynchrony that lots of JS programmers are using these days.

Q is a popular promises library for JS. There are some 250+ packages that depend on it in NPM, node's package manager. So cool, let's take their example of the "Pyramid of Doom" from the github page:

step1(function (value1) {
    step2(value1, function(value2) {
        step3(value2, function(value3) {
            step4(value3, function(value4) {
                // Do something with value4

The promises solution does at least fix the pyramid problem:

.then(function (value4) {
    // Do something with value4
}, function (error) {
    // Handle any error from step1 through step4

But to my ignorant eye, some kind of solution involving a straight-line function would be even better. Remember what generators do: they suspend computation, wait for someone to pass back in a result, and then continue on. So whenever you would register a callback, whenever you would chain a then onto your promise, instead you suspend computation by yielding.

Q.async(function* () {
  try {
    var value1 = yield step1();
    var value2 = yield step2(value1);
    var value3 = yield step3(value2);
    var value4 = yield step4(value3);
    // Do something with value4
  } catch (e) {
    // Handle any error from step1 through step4

And for a super-mega-bonus, we actually get to use try and catch to handle exceptions, just as Gods and Brendan intended.

Now I know you're a keen reader and there are two things that you probably noticed here. One is, where are the promises, and where are the callbacks? Who's making these promises anyway? Well you probably saw already in the second example that using promises well means that you have functions that return promises instead of functions that take callbacks. The form of functions like step1 and such are different when you use promises. So in a way the comparison between the pyramid and the promise isn't quite fair because the functions aren't quite the same. But we're all reasonable people so we'll deal with it.

Note that it's not actually necessary that any of the stepN functions return promises. The promises library will lift a value to a promise if needed.

The second and bigger question would be, how does the generator resume? Of course you've already seen the answer: the whole generator function is decorated by Q.async, which takes care of resuming the generator when the yielded promises are fulfilled.

You don't have to use generators of course, and using them with Q does mean you have to understand more things: not only standard JavaScript, but newfangled features, and promises to boot. Well, if it's not your thing, that's cool. But it seems to me that the widespread appreciation of await in C# bodes well for generators in JS.

ES6, unevenly distributed

Q.async has been part of Q for a while now, because Firefox has been shipping generators for some time.

Note however that the current ES6 draft specification for generators is slightly different from what Firefox ships: calling next on the iterator returns an object with value and done properties, whereas the SpiderMonkey JS engine used by Firefox uses an exception to indicate that the iterator is finished.

This change was the result of some discussions at the TC39 meeting in March, and hasn't made it into a draft specification or to the harmony:generators page yet. All could change, but there seems to be a consensus.

I made a patch to Q to allow it to work both with the old SpiderMonkey-style generators as well as with the new ES6 style, and something like it should go in soon.

give it to me already!

So yeah, generators in V8! I've been working closely with V8 hackers Michael Starzinger and Andreas Rossberg on the design and implementation of generators in V8, and I'm happy to say that it's all upstream now, modulo yield* which should go in soon. Along with other future ES6 features, it's all behind a runtime flag. Pass --harmony or --harmony-generators to your V8 to enable it.

Barring unforeseen issues, this will probably see the light in Chromium 29, corresponding to V8 3.19. For now though this hack is so hot out of the fire that it hasn't even had time to cool down and make it to the Chrome Canary channel yet. Perhaps within a few weeks; whenever the V8 dependency in Chrome gets updated to the 3.19 tree.

As far as Node goes, they usually track the latest stable V8 release, and so for them it will probably also be a few weeks before it goes in. You'll still have to run Node with the --harmony flag. However if you want a sneak preview of the future, I uploaded a branch of Node with V8 3.19 that you can build from source. It might mulch your cat, but life is not without risk on the bleeding_edge.

Finally for Q, as I said ES6 compatibility should come soon; track the progress or check out your own copy here.

final notes

Thanks to the V8 team for their support in this endeavor, especially to Michael Starzinger for enduring my constant questions. There's lots of interesting things in the V8 pipeline coming out of Munich. Thanks also to Andrew Paprocki and the Bloomberg crew for giving me the opportunity to fill out this missing piece of real-world ES6. The Bloomberg folks really get the web.

This has been a hack from Igalia, your friendly neighborhood browser experts. Have fun with generators, and happy hacking!

74 responses

  1. C. Scott Ananian says:

    We've been using yield in spidermonkey to do this for a while:

    node-fibers also has some decent node-integration ideas; it might be worth stealing stuff from there.

  2. Dave Herman says:

    Congrats, and thank you for helping to spread ES6 functionality to more JavaScript engines! You made the web better today. <3

    You might want to take a look at, which I created a few years ago for the same purposes you talk about here. I do have a few changes I need to make, since it was built for Firefox's generators implementation, which need to be updated to the latest version of the ES6 design. And I want to eliminate the (non-Promises/A+-compliant!) promises implementation from it and reuse an existing promise library like RSVP.js. Once I make these changes it should work nicely with V8's generators implementation.


  3. James Long says:

    This is amazing, great work! I can't imagine this *not* rocking the node world. If we can get the cooperative threading libraries to stabilize, it seems like we won't have to talk about promises so much anymore.

  4. Matt Mastracci says:

    Should Function.toString() be returning function* instead of function?

    > function* foo() { }
    > foo.toString()
    'function foo() { }'

  5. Matt Mastracci says:

    One more bug - calling the constructor of a generator function crashes node:

    > function* foo() { }
    > foo.constructor()
    Illegal instruction: 4

  6. Andy WIngo says:

    Hi Matt, are you sure you are on the v8-3.19 branch in my repository? Generators should definitely print with function* there. It's a little confusing as 3.18 also has the first half of the generators work, so it's easy to think you're on the right branch when in reality not everything is working.

    The constructor() bug is a real one though. Thanks for the report. If you have any more, send them to me at -- and after a couple weeks or so, the v8 bug tracker is probably the right place.

  7. Brandon says:

    Very cool. I can see this being quite nice for JavaScript UI, which is also asynchronous, and which also tends to degenerate into a maze of callbacks invoking callbacks (which queue timeouts that invoke callbacks that queue timeouts.. ). Pyramid of doom indeed. For example, say I am drawing to a canvas. It might be convenient to implement animations as generators that yield once per frame. You step an animation simply by resuming the generator, and the animation is done when the generator exits. You can write your animations in whatever style feels natural, so long as they yield once per frame.

    Are these generators like Lua co-routines, or like Python generators? What happens when foo calls bar, and bar yields?

  8. Michael Hart says:

    v8 3.19 has been included in node.js v0.11.2:

    Just tested out your basic generator example (with node --harmony) and it works a treat!

  9. Arpad Borsos says:

    So TC39 agreed to drop `StopIteration`? Thank god! Exceptions should not be used for regular control flow, otherwise we would land in java land, like this:

  10. Staffan Eketorp says:

    Hey. Thanks for a nice blog post. I know I'm a little late, but it inspired me to look out for some libraries and experiment with new builds of canary and node. Since I believed there were a few things missing in Q and Galaxy I wrote my own library for generator support, Y-yield. I'd be super happy if you were able to give it some feedback or mention it in some blog/tweet.


  11. Staffan Eketorp says:

    By the way - I forgot the link:

  12. Vedran Rodic says:

    In nodeup podcast episode 46 you mentioned a certain bug in v8 when yield waits for too long. I guess this has been fixed by now?

    I'm sorry, I know you mentioned in the podcast that we should not ask you about this one (presuming because you know about it already), but I just couldn't find the bug number to confirm it has been solved.

  13. Gold: A Distinct Asset Class Case Solution says:

    All things considered, nothing I can say except for incredible! I trust you'll make more helpful articles in the up and coming days.

  14. C Assignment Help says:

    Awesome enlightening site. I'm truly awed subsequent to perusing this blog entry. I truly welcome the time and exertion you go through to impart this to us!

  15. iphone 8 release date says:

    Totally right article.

  16. Jack Fernado says:

    Thanks for sharing this useful information.

    If you are facing problems with your Email, contact us anytime for getting immediate resolution for their Ymail issues at our Yahoo contact support phone number UK. On getting trapped with any unexpected error that creates issue in smooth login into gmail account, a customer can get help on just a quick call at Gmail customer support contact helpline number UK. All sort of mailing issues are fixed in the minimum possible time.

  17. Mark Steve says:

    Very nice post. Thanks for sharing it.

  18. birthday wishes says:

    Slowly but surely I am growing to understand all of this JS. Long time coming...

  19. tubemate app download for ipad says:

    Download Tubemate For iPhone (6s,6,5s,5), iPad and iOS downloading on iphone or ipad it becomes very difficult as the app.

  20. merry christmas 2016 cards says:
  21. Sam Smith says:

    Any technical issue related to your password, or emailing as well as your account could take your time and energy in getting resolved. But by contacting yahoo customer service 365x24x7 you can get in touch with us. We are third party experts who resolve each issue through on call remote access and live chat. So call us at 1-800- 6314-296.

  22. Prayer Times says:

    I ve heard about it. Thanks for sharing. prayer times

  23. psoriasis says:

    I was taking a gander at some of your posts on this site and I consider this site is truly informational!

  24. Jackie Dada says:
  25. conforama says:

    Fantastic article great job to be continued please.

  26. YO YO says:
  27. Gmail Help Number UK says:

    #GmailHelplineNumberUK, #GmailPasswordRecoveryPhoneNumber, #GmailTechnicalSupportNumberUK, #GmailHelpDeskPhoneNumber, #GmailPasswordRecoverySupport, #GmailContactNumberUK, #GmailCustomerCareNumberUK

  28. Gmail Help Number UK says:

    #GmailHelplineNumberUK, #GmailPasswordRecoveryPhoneNumber, #GmailTechnicalSupportNumberUK, #GmailHelpDeskPhoneNumber, #GmailPasswordRecoverySupport, #GmailContactNumberUK, #GmailCustomerCareNumberUK

  29. Hotmail Helpline Number UK says:

    #HotmailHelplineNumberUK, #HotmailPasswordRecoveryPhoneNumber, #HotmailTechnicalSupportNumberUK, #HotmailHelpDeskPhoneNumber, #HotmailPasswordRecoverySupport, #HotmailContactNumberUK, #HotmailCustomerCareNumberUK

  30. McAfee Helpline Number UK says:

    Fantastic article McAfee Helpline Number UK 0800-090-3242

  31. Microsoft Helpline Number UK says:

    Nice Post, Microsoft Helpline Number UK 0800-098-8352

  32. Adobe Support Number UK says:

    Nice post thanks for shearing.... want technical support for adobe call adobe helpline number UK and get quick support.

  33. Dissertation Writing Help says:

    Science Channel’s are giving a complete knowledge to its viewers about every thing students write done dissertation on this subjects and show its importance.

  34. Homework Writing Service United Kingdom says:

    Those who come to read your article will find lots of helpful and informative tips

  35. Marketing Writing Service says:

    Good way of telling, good post to take facts regarding my presentation subject matter, which i am going to deliver in my college

  36. Aerospace Engineering Assignment Help says:

    only professional writers can make this kind of material, cheers

  37. Online Programming Project Help says:

    Things are very open and intensely clear explanation of issues. was truly information. Your website is very beneficial.

  38. العاب says:

    Things are very open and intensely clear explanation of issues. was truly information. Your website is very beneficial............

  39. buddy says:

    If you live in the Twin Cities and have a property which has suffered storm, fire mold or water damage, then get in touch with the professionals at water damage Minneapolis. Find the perfect message for the New year with these huge collections of New Year wishes

  40. ADIL says:
  41. Happy Valentines Day Wishes says:
  42. itube download says:
  43. download aptoide says:
  44. iRoot Apk says:
  45. holi pics says:
  46. happy lohri says:
  47. Manchun says:

    Nice Article Thank you. JanBask Training as a Trainer, we offering Salesforce, Hadoop, Dot Net, SQL, PMP, Business Analyst and QA Testing.

  48. Assignment Help Perth says:

    Thanks for sharing this blog its very informative and useful for us .

  49. Methodology Dissertation Help says:

    That was a very needy help for me.Thanks and keep sharing.

  50. Accounting Thesis Help says:

    Thanks for Nice and Informative Post. This article is really contains lot more information about This Topic.

  51. Valentine Day Greeting Cards says:
  52. MovieBox says:
  53. Valentines day jokes says:
  54. Australia day quotes says:

    Hey. Thanks for a nice blog post. I know I'm a little late, but it inspired me to look out for some libraries and experiment with new builds of canary and node. Australia Day is celebrated every year on 26th January in Australia with great joy and happiness. Australia day wishes images and Happy Australia Day quotes Australia Day is the official National Day of Australia.

  55. ipadapps says:

    thanks for this helpful post for me and all. please tell me how to generators in 8 .flekstore 3.0

    regard, thanks.ipad appsios appsshillong teer gametutuappBB Recscreen recorder apptop 10 upcoming apps ios

  56. Avast Customer Care Number UK says:

    When you use Avast you may get stuck in issue of antivirus not updating on regular basis and to get it corrected call at Avast Customer Care Number UK. You can call these technicians anytime you want as they serve you with 24x7 service.

  57. ana maqsood says:
  58. Norton Support Number UK 0800-098-8371 Norton Helpline Number UK says:

    Great Post.. Thank you for sharing.. I am waiting your next post.Norton Help Number UKNorton Contact Number UK

  59. shillong teer formula says:

    thanks for sharing this helpful and unique post. this is a right platform for all.

  60. nox app player for Mac says:

    nice !
    thanks for this unique post for me and all. i am waiting you next helpful post.

  61. Tutuapp says:

    thanks for this helpful post for me and all. i am also find this news from along time. thanks for sharing. i am also share to my friend circle.
    thanks i am waiting your next helpful post.iphone 7s featurestutuappiphone 7 VR headsetipad appsipad app

  62. shillong teer result says:

    thanks for this helpful post for me and all who find The Binding of Isaac: Rebirth game. can you tell me how to install in apple ipad mini. i have face some problem in two different games to install in ipad. Second game = Star Wars:Force Arena game. please tell me how to install easy and simple way.

  63. Ninja Spinki Challenges Free Download says:

    thanks for this helpful post for me and all game user's. please can you tell me how to play and install this game in iphone.

    flekstore 3.0ipad appsios appsshillong teer gametutuappBB Recscreen recorder apptop 10 upcoming apps iosfitness apps for ios

  64. Nomao Camera App For iPhone says:

    thanks for this helpful ppst for me and all flekstore 3.0 apps user. please anyone help me how to use and install in ipad mini.

    regard, thanks.ipad appsios appsshillong teer gametutuappBB Recscreen recorder apptop 10 upcoming apps iosfitness apps for ios

  66. Dell Printer Support Number UK 0800-098-8371 Dell Printer Contact Number UK says:

    Facing technical issues in installation of the Dell printer, Call us at Dell Printer Support 0800-098-8371 UK. Our expert will help you to solve these issues.

  67. veydy says:

    [url=]dog grooming[/url] dog grooming

  68. Buy YouTube Views says:

    nice and very informative stuff...

  69. Residential Property In Mumbai says:

    awesome article..

  70. dell Printer contact number UK says:

    When your Dell printer puts you in trouble due to its tech issues of Dell Printer then you are free to contact our technical executive by calling at 0800-098-8371 Dell Printer contact number UK.

  71. titi says:

    Things are very open and intensely clear explanation of issues. was truly information. Your website is very beneficial.jim

  72. lyrics says:

    i love this post thanks for sharing this

  73. dj punjabi says:

    kaimzzzz ppost

  74. Mp3 Songs download says:

    Nice Postzzz

Leave a Reply