After some time's work, now I finally got a working irb for mruby. I'm such a lazy guy so you may already seen the demo from this, this or this. Anyway, for those of you who didn't see it, the demo is at here.
With all the work in webruby, actually it is not so hard to implement this. However, there are still two things I want to write down here as notes.
For simplicity, the web irb uses mrb_load_string to parse and execute ruby source together. Now here comes the problem, the function signature is like following:
Luckily, we do not need the return value here. Hence we can simply create a wrapper:
If at later times we decide to add logic to check the return value of mrb_load_string, we can simply added it here. For this driver function, the generated js function would only requires two arguments: the mrb state and the string to load.
Emscripten has a built-in optimizer, but it wouldn't work with mruby:
Oops, maybe I need to turn to Alon for help here-_-
Anyway, I've already created the repository for this, let's see if I can make this work:)
After a pretty long discussion and fix(Thanks to Alon for the awesome job in fixing this issue!), now setjmp/longjmp in emscripten works well. The C++ dependency in mruby-browser can be finally dropped, and all the mruby tests can pass without any particular hack. We can now say that the basic building block is there, and I can turn to focus on more interesting stuffs. It is also at this time that I think a new meaningful name is needed for this project. The original mruby-browser sounds too much like an experiment instead of a project for everyone to use.
The first thing came to my mind is RubyScript. With something already built called CoffeeScript or ClojureScript, it is not so hard for this particular name to come to my mind. However, Google tells us that someone has already uses this name. It looks quite like a hackathon project(20 commits in two days, and no commits since then for over a year). But I still do not want to take the risk that this guy(or lady, it is so hard for me to judge this by the name, can anyone give me a hint?) may want to bring this project back to life. Let's try something else.
What's worth mentioning is that someone is building MobiRuby, which brings mruby to iOS. Well, I'm bringing mruby to the Web, so what about WebRuby? Google tells us that there aren't so many people using this name, one is using this as a repository for a web course, while the other is just a sinatra-based web backend. Well, I will just pick this one as the name:)
Besides changing the name, I also change the build script a little bit. Now we can simply put mruby source code in src folder, the source code will be parsed and compiled when building the project. Only the final generated bytecode will be included in the js file or the webpage. This saves us the time for parsing the source code online. If at some point we need to parse the mruby source code online, we can easily bring this back since the parse code is still included in the generated js code.
Personally, one good thing about owning an open source project is that you can decide the priority of each feature:) As a developer who always dream about creating games, my next thing to work on will be a wrapper for OpenGL ES 2.0 API. I still haven't thought of some beautiful ideas on a calling interface between mruby and c(or js), so I think I will first focus on some particular library wrappers. Since mrbgems is already in HEAD, it becomes a natural idea to pack this library as a mrbgem. The reason for choosing OpenGL ES 2.0 over WebGL is that OpenGL ES 2.0 is more general, and maybe it can also be used with mruby in iOS or Android development. What's more, emscripten provides a translation between OpenGL ES 2.0 and WebGL, so we can simply take advantage of that instead of write yet another wrapper of WebGL.
Just as I said before, it is really fun working on this:)
With my Hadoop paper submitted last Friday, I can spend more time playing with mruby. Now after several days' hacking, I finally manage to make all mruby tests pass in a browser or in node.js.
Now it's time to keep a note on how to make these tests passed.
As this Issue is resolved(Thanks to Alon Zakai for his super fast commit to fix this!), the mruby source code can be compiled using emcc successfully, the sample main.c file also works. But there are still 5 tests left that are not passed: 2 of them failed, while the other 3 caused node.js to crash. These 5 tests are:
Tests for erf and erfc functions in math.rb
Float#round [220.127.116.11.12] in float.rb
String#to_f [18.104.22.168.39] in string.rb
Exception 14 in exception.rb
Proc.new [22.214.171.124.1] in proc.rb
To be honest, the result is quite good, since only 5 of the 489 tests got problems. I guess emscripten really has reached a pretty mature status thanks to Alon. Most of the fixes here are resolved from commits to mruby or emscripten directly. However, there are also annoying ones. Anyway, I will explain how to make each of them pass.
erf and erfc functions
Honestly, this is the first time that I heard about these two functions. They reside in the math.h header file of standard C library. The erf function is used to calculate the error function of a value x. While the 'erfc' function calculates the complementary error function of x. emscripten does not come with an implementation for this function. However, there is an implementation in math.c of mruby for MSVC, which does not provide erf/erfc functions. It was originally take from here:
This is an interesting and easy one. The test code resides at here. Actually all the round tests give the correct result, what went the wrong is that == is used to test equality for two floating point values. A small commit fixes this, easy one.
This is also related floating point value. The code is at here. b should be assigned to 123456789.0, when using check_float to compare b with 123456789.0, they should be treated as equality. Funny thing is that node.js would give the result of 1.4901161193848e-08 as the difference between the two values, while check_float would only consider two values to be the same if they are within 1E-12.
Simply changing Line #328 to 123456789 instead of 123456789.0 would give the correct result, but this is a very bad fix for this problem and does not really solve it. Basically there may be two reasons:
v8 does not provide that many precisions for floating point value.
It is still unknown which is the cause for this problem. What I choose to do now is to let mruby use float instead of double. When using float, check_float would accept two values within 1E-5, for which the current result of 1.4901161193848e-08 will be enough. Anyway, I will come back to this later, maybe a dig into the v8 issue list can bring some insight into this.
Exception 14 and Proc.new [126.96.36.199.1]
Both the exception and proc tests crash node.js, and they both use a begin ... rescue ... end statement with a method call in the begin clause. A simple guess is that they are due to the same reason.
The original gist also comes with logs running this natively or via emscripten. With a stack manipulating setjmp/longjmp, the longjmp would erase the stack for level 1 calling of stack_manipulate_func. The program would only call the exiting printf once. However, with the current implementation of setjmp/longjmp in emscripten, the stack is not changed, the exiting printf will be call by both the level 0 and level 1 version of stack_manipulate_func.
Quick Update: Actually Alon confirms that this is just a bug and it is fixed. So maybe we can still have a C99 solution on this problem. Interesting, I will take a look at this later. I really feel sick about using a C++ compiler, that may bring a lot of evil stuff when working on the later parts involving more C code, such as C function calling interface.
Anyway, now all the tests have passed. Not only in node.js but also in browsers. However, the time to run tests differ greatly:
Chrome 23: 1.682s
Firefox Aurora 18: 5.294s
Safari 6: 0.394s
This is interesting. Safari is so fast that one can think something went wrong for the other two.
A Little Background
I really love Ruby, yet I'm not that much into Rails. Rails is great indeed, I really learned so much while playing around with Rails. In fact, more than half of my knowledge using Ruby was learned reading the Rails source code. However, I just feel bored writing again and again applications working on a database-_- Whatever new features my app has and whatever facinating features Rails provide, basically I just keep writing code to creating entries in db, reading these entries out, updating and deleting them occasionally(Thank you, CRUD!). After all, the MVC architecture is there. It is simply not fun for me>_<
So I always wonder around looking for interesting stuffs outside of the Rails world. Celluloid, Gosu, Fluentd are all fun to play with. The latest toy around is mruby, a lightweight implementation Ruby. It works for embedded system and can be linked into existing softwares. Well, that sounds like Lua, but as a former-Lisp fan, Ruby is more interesting, isn't it? Sorry I've gone a little off-topic, I could write another post describing my feelings on different languages, but that's not the point today:)
The current mruby compiling process works like this:
Use bison to parse src/parse.y into src/y.tab.c.
Compile every c source in src, the generated object files are then archived into lib/libmruby_core.a.
Compile tools/mrbc/mrbc.c and link with lib/libmruby_core.c to generate the core mruby compiler bin/mrbc.
Use the mruby compiler to standard libraries in mrblib, the generated bytecode is attached as an array to mrblib/init_mrblib.c. The newly created source code is called mrblib/mrblib.c.
Compile mrblib/mrblib.c and add to lib/libmruby_core.a. The result is called lib/libmruby.a.
We need to compile all source code src together with the generated file src/y.tab.c and mrblib/mrblib.c. For simplicity and the unstable state of mruby, I just require building the entire mruby first in my Makefile:
C compiler vs C++ compiler
In short, it was a tough and mysterious road-_-
The first version compiles okay, but it keeps running into RangeError: Maximum call stack size exceeded error. While browsing for wiki page of emscripten, one sentence in CodeGuidlinesAndLimitations caught my eye.
"Nonportable code that uses low-level features of the native environment, like native stack manipulation (e.g. in conjunction with setjmp/longjmp. We support normal setjmp/longjmp, but not with stack replacing etc.)."
I suspect this may be the reason, what supports my idea is one commit in emscripted-ruby, which is a port for ruby 1.8.7 onto the browser. It basically eliminates all setjmp/longjmp calls with c++ exception. So I decided this may be the direction and started coding right way. Well, now it turns out this is a mistake, I should've done more investigation. Anyway, you will see the mistakes I made.
Then I spent around ten hours figuring out a way to patch mruby to use c++ exception instead of setjmp/longjmp with all tests passed. The compiler is also changed from emcc to em++. Well, now the mruby compiles and works:
This was Wednesday night and I went to read the FreeBSD book for exam. On Thursday morning, I accidentally compiles the code with em++ but without my patches for setjmp. The code happened to work! Even if I revert the code back to the initial revision with all original build settings, I still cannot reproduce the RangeError error. Well, it remains a mystery and all I can guess is that something is wrong with my installation of Node.js at that day. It might use a different setting that day...
Then another problem came: when compiling with emcc, the code compiles okay, but would run forever without terminating; It would only work when compiling with em++... This is also strange, I will look into this later.
So the lesson I learned is: do more investigation before pumping out coding. There may be another reason.
Well, this is really fun. I've sent a post on emscripten-discuss. And I will spend some time on this.
Besides that, a few interesting follow-up works are already in my head:
OpenGL ES 2/WebGL binding
Repl in the browser
MobiRuby for the browser(well, this may be too ambitious)
Anyway, it is really fun working on this.
If you read all the way to here(I would not read such a long article myself-_-), I want to say sorry for my English. I'm only fluent in Chinese, but I just want more people to be able to read about this:)