Good evening tubes,
The JS benchmarks are terrible, but everyone is using them to optimize their implementations. There's a serious danger of garbage-in, garbage-out here.
To my eye the V8 ones are the best general-purpose benchmarks: the ones that represent not just the inner loops, but bigger programs. But hey, maybe that's just the Schemer in me amused at seeing cross-compiled Lisp benchmarks in their suite.
Tool support is terrible.
All of these projects have to implement their own profilers, disassemblers, debuggers, etc, and for all platforms, in theory. In practice, yuck. GDB just got support for JIT-generated code, and it's terrible: you have to generate whole object files (ELF on ELF architectures, presumably mach-O or whatever on Darwin, and who knows on windows!), and if you want debugging information, you have to generate DWARF.
Of course oprofile couldn't use the same format, it has its own thing, and it's not the same as the lovely but ungooglable perf, which appears to only support map files generated by some JVM no one has heard of, but has no support for garbage-collecting executable code or for line numbers. Aaargh!!!
Incidentally, my colleague Xan appears to have run into these issues independently.
I've been joking recently that I've been coming up like Neo, in the training sequence, eyes open, drawn breath: "whoa: I know
I try not to make promises about the future, so consider it simply a prediction, that the next entry will feature more parentheses than this one. Until then, happy hacking!