Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Developing U++ » U++ Developers corner » JavaScriptCore
JavaScriptCore [message #27274] Sat, 10 July 2010 14:59 Go to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Hello friends!

I happened to be stuck for a few days away from my computer, with just an old windows pc. So having nothing better to do, I installed theide and started to dig into WebKits JavaScriptCore.

After few days of head banging and hair pulling Smile , I managed to get all the code to compile and link with mingw. And what is more important, it can be used within an U++ application.

As of now, I have a simple interactive console parser (think of calculator in theide, just with JS instead of Esc) based on jsc from WebKits tree (they use it for testing) rewritten in U++. The parser is small, about 100 lines of code, but JavaScriptCore is way too big post all the code here. If someone is interested I can post instructions how to get the code and necessary patches later. For now, you can download the binary files here (8MB). Also, there are still some obstacles, like dependence on ICU library (17MB of dlls Confused ), but I believe it can be solved in near future (some WebKit ports work without it, so I guess we can too).

As soon as I get better feeling about how this thing work, I plan to write some nicer U++ style interface around the library, so that anyone can use it from within their apps. And of course, as soon as I get back to my computer, I try the same on linux.

I hope that having JavaScriptCore available in U++ will be useful for someone. But most importantly, it is the first step towards WebKit in U++ Wink

Best regards,
Honza
Re: JavaScriptCore [message #27275 is a reply to message #27274] Sat, 10 July 2010 16:23 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
dolik.rce wrote on Sat, 10 July 2010 08:59

Hello friends!

I happened to be stuck for a few days away from my computer, with just an old windows pc. So having nothing better to do, I installed theide and started to dig into WebKits JavaScriptCore.

After few days of head banging and hair pulling Smile , I managed to get all the code to compile and link with mingw. And what is more important, it can be used within an U++ application.

As of now, I have a simple interactive console parser (think of calculator in theide, just with JS instead of Esc) based on jsc from WebKits tree (they use it for testing) rewritten in U++. The parser is small, about 100 lines of code, but JavaScriptCore is way too big post all the code here. If someone is interested I can post instructions how to get the code and necessary patches later. For now, you can download the binary files here (8MB). Also, there are still some obstacles, like dependence on ICU library (17MB of dlls Confused ), but I believe it can be solved in near future (some WebKit ports work without it, so I guess we can too).

As soon as I get better feeling about how this thing work, I plan to write some nicer U++ style interface around the library, so that anyone can use it from within their apps. And of course, as soon as I get back to my computer, I try the same on linux.

I hope that having JavaScriptCore available in U++ will be useful for someone. But most importantly, it is the first step towards WebKit in U++ Wink

Best regards,
Honza


Sounds very very good.

Webkit itself, any chances? Anybody?

Mirek

[Updated on: Sat, 10 July 2010 16:24]

Report message to a moderator

Re: JavaScriptCore [message #27277 is a reply to message #27275] Sat, 10 July 2010 19:29 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

luzr wrote on Sat, 10 July 2010 16:23

Sounds very very good.

Webkit itself, any chances? Anybody?


Hi Mirek,

I was hoping this will drag someones attention so I don't have to do the WebKit port all by myself Wink But in worst case, I will definitely at least try...

I cleaned the things up a bit, so here are the "How to do this at home" instructions Smile

  1. Get the WebKit sources from SVN:
    cd C:\upp
    svn checkout http://svn.webkit.org/repository/webkit/trunk/JavaScriptCore WebKit\JavaScriptCore
    You could also get the full trunk, but for now it is not necessary
  2. Patch the sources: Download the attachment of this message, inside is directory JavaScriptCore, which contains .upp file and three files that need to be patched. The directory structure is the same, so it should be save just to copy it over the original directory tree.
  3. Copy API headers: Because of the way how WebKit is normally built, it expects some of the headers in JavaScriptCore\JavaScriptCore, so we have to copy it in there.
    cd C:\upp\WebKit\JavaScriptCore
    copy API\*.h JavaScriptCore
  4. Create look-up tables: Some parts of JavaScript core use look-up tables which are usually generated in makefile, but we have to do it ourselves for now.
    python create_regex_tables > yarr/RegExpTables.h
    perl create_hash_table parser/Keywords.table > parser/Lexer.lut.h
    perl create_hash_table runtime/RegExpConstructor.cpp -i > runtime/RegExpConstructor.lut.h
    perl create_hash_table runtime/MathObject.cpp -i > runtime/MathObject.lut.h
    perl create_hash_table runtime/JSONObject.cpp -i > runtime/JSONObject.lut.h
    perl create_hash_table runtime/DatePrototype.cpp -i > runtime/DatePrototype.lut.h
    perl create_hash_table runtime/ArrayPrototype.cpp -i > runtime/ArrayPrototype.lut.h
    perl create_hash_table runtime/StringPrototype.cpp -i > runtime/StringPrototype.lut.h
    perl create_hash_table runtime/RegExpObject.cpp -i > runtime/RegExpObject.lut.h
    perl create_hash_table runtime/NumberConstructor.cpp -i > runtime/NumberConstructor.lut.h
    perl pcre/dftables pcre/chartables.c
    Sorry for the slashes, I did this in cygwin so they are unix style, if you have windows perl or python, you have to rewrite them. In case you don't have perl or python I added them into the attached file as well. They are in the lut directory, just copy them to correct places.
  5. Download ICU: Download and extract the library from http://download.icu-project.org/files/icu4c/4.4.1/icu4c-4_4_ 1-Win32-msvc9.zip. If you have 64b windows, you might need to download different file. I extracted this file into C:\upp\Webkit. If you put it somewhere else, you will have to change the following paths accordingly. Now it is necessary to make mingw aware of ICU, so we add C:\upp\WebKit\icu\include to include directories and C:\upp\WebKit\icu\lib in lib directories in build methods.
  6. Setup assembly: Create new assembly from the C:\upp\WebKit and C:\upp\uppsrc nests.
  7. Get jsc package: The last directory in the attachment file is jsc. That is the interpreter package. Just copy it into C:\upp\WebKit.

Doesn't look really difficult, right? The worst part was getting all the switches in wtf/Platform.h right Smile If I didn't forget something, you should be now able to compile, link and run jsc. If I did forget something, let me know Wink

Honza

EDIT: Corrected paths in upp file and reuploaded the attachment.
EDIT2: I knew I forgot something Smile One more lookup table, now added.
EDIT3: Hopefully last one... Forgot to mention ICU dlls (from C:\upp\WebKit\icu\bin) must be either on PATH or in the same directory as the final executable. But you all probably know that Wink
  • Attachment: jsc_all.zip
    (Size: 35.04KB, Downloaded 251 times)

[Updated on: Sat, 10 July 2010 21:39]

Report message to a moderator

Re: JavaScriptCore [message #27283 is a reply to message #27277] Sun, 11 July 2010 00:05 Go to previous messageGo to next message
tojocky is currently offline  tojocky
Messages: 607
Registered: April 2008
Location: UK
Contributor

What About v8 from google ( code.google.com/p/v8 )?
Re: JavaScriptCore [message #27284 is a reply to message #27283] Sun, 11 July 2010 00:15 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

tojocky wrote on Sun, 11 July 2010 00:05

What About v8 from google ( code.google.com/p/v8 )?

Hi Ion,
It should be possible. When building Chromium port of WebKit, it is just one #define in config.h to switch between JavaScriptCore and V8. But I haven't got that far yet.

Honza
Re: JavaScriptCore [message #27300 is a reply to message #27284] Sun, 11 July 2010 14:35 Go to previous messageGo to next message
tojocky is currently offline  tojocky
Messages: 607
Registered: April 2008
Location: UK
Contributor

dolik.rce wrote on Sun, 11 July 2010 01:15

tojocky wrote on Sun, 11 July 2010 00:05

What About v8 from google ( code.google.com/p/v8 )?

Hi Ion,
It should be possible. When building Chromium port of WebKit, it is just one #define in config.h to switch between JavaScriptCore and V8. But I haven't got that far yet.

Honza


I was compiled v8 in u++. I tested only v8 stand alone simple code. I think it is not a problem to create a v8 wrap plug-in.
Goo wrapper is v8-juice (http://code.google.com/p/v8-juice/) We can try. I'm interested in this project!

Ion Lupascu (tojocky)
Re: JavaScriptCore [message #27302 is a reply to message #27300] Sun, 11 July 2010 16:24 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

tojocky wrote on Sun, 11 July 2010 14:35

dolik.rce wrote on Sun, 11 July 2010 01:15

tojocky wrote on Sun, 11 July 2010 00:05

What About v8 from google ( code.google.com/p/v8 )?

Hi Ion,
It should be possible. When building Chromium port of WebKit, it is just one #define in config.h to switch between JavaScriptCore and V8. But I haven't got that far yet.

Honza


I was compiled v8 in u++. I tested only v8 stand alone simple code. I think it is not a problem to create a v8 wrap plug-in.
Goo wrapper is v8-juice (http://code.google.com/p/v8-juice/) We can try. I'm interested in this project!

Ion Lupascu (tojocky)


I guess we could make a common interface with two different backends, allowing user to switch between them. Each of those engines have it's advantages. If I'm not mistaken, V8 has better performance, while JSC supports more platforms. And probably even more key differences.

Honza
Re: JavaScriptCore [message #27305 is a reply to message #27302] Sun, 11 July 2010 20:32 Go to previous messageGo to next message
tojocky is currently offline  tojocky
Messages: 607
Registered: April 2008
Location: UK
Contributor

dolik.rce wrote on Sun, 11 July 2010 17:24


I guess we could make a common interface with two different backends, allowing user to switch between them. Each of those engines have it's advantages. If I'm not mistaken, V8 has better performance, while JSC supports more platforms. And probably even more key differences.

Honza


Please inform me about your actions.

Ion Lupascu (tojocky).
Re: JavaScriptCore [message #27351 is a reply to message #27274] Tue, 13 July 2010 20:39 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Small update: Above described process almost works on linux as well. The only differences are that step 5 can be skipped, since libicu is present in most distributions by default and that you must link with icui18n instead of icuin.

The example interpreter builds and works fine, but there is a small memory leak... Weird is that the code is the same as on windows. I'd be thankfull for any hints on why there can be difference between win32 and linux.

Honza
Re: JavaScriptCore [message #27366 is a reply to message #27274] Wed, 14 July 2010 17:01 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Hello all,

I put together an U++ style interface for JavaScriptCore, as I promised.

It's public part is super simple (well, actually the internals are very simple too Smile ):
class JavaScript{
public:
	JavaScript();
	~JavaScript();
	void AddNativeFunction(const char* name,NativeFunction function);
	bool Execute(const String& code,String& result);
};

Even though it is simple, it still should be usefull. I never used much more than this when scripting with Esc.

If anyone has any ideas how to improve or extend this interface, let me know. Maybe Execute(filename, result)?

Attachment contains new version of jsc package. If used as main package it compiles the demonstration interpreter. If included in other package, it just provides the interface.

Best regards,
Honza

EDIT: Removed the file in favor of the link below.

[Updated on: Thu, 15 July 2010 19:12]

Report message to a moderator

Re: JavaScriptCore [message #27371 is a reply to message #27366] Thu, 15 July 2010 08:25 Go to previous messageGo to next message
tojocky is currently offline  tojocky
Messages: 607
Registered: April 2008
Location: UK
Contributor

dolik.rce wrote on Wed, 14 July 2010 18:01

Hello all,

I put together an U++ style interface for JavaScriptCore, as I promised.

It's public part is super simple (well, actually the internals are very simple too Smile ):
class JavaScript{
public:
	JavaScript();
	~JavaScript();
	void AddNativeFunction(const char* name,NativeFunction function);
	bool Execute(const String& code,String& result);
};

Even though it is simple, it still should be usefull. I never used much more than this when scripting with Esc.

If anyone has any ideas how to improve or extend this interface, let me know. Maybe Execute(filename, result)?

Attachment contains new version of jsc package. If used as main package it compiles the demonstration interpreter. If included in other package, it just provides the interface.

Best regards,
Honza



Can you upload all packages?
Re: JavaScriptCore [message #27407 is a reply to message #27371] Thu, 15 July 2010 19:00 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

tojocky wrote on Thu, 15 July 2010 08:25

Can you upload all packages?

Hi Ion,

Sure I can. It is just too big to put it on the forum. I've put a zip file up for download elsewhere (11MB). It contains JavaScriptCore and the interface package. Only additional thing needed is the ICU, which is linked above (only on windows,linux should have it).

Best regards,
Honza
Re: JavaScriptCore [message #27422 is a reply to message #27407] Sat, 17 July 2010 09:52 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
dolik.rce wrote on Thu, 15 July 2010 13:00

tojocky wrote on Thu, 15 July 2010 08:25

Can you upload all packages?

Hi Ion,

Sure I can. It is just too big to put it on the forum. I've put a zip file up for download elsewhere (11MB). It contains JavaScriptCore and the interface package. Only additional thing needed is the ICU, which is linked above (only on windows,linux should have it).

Best regards,
Honza


Ideally, we should have any such "plugin" managed the same way, which means converted to source based package (like e.g. plugin/jpg).

Do you think anything like that would be possible for any of libraries mentioned here?

Mirek
Re: JavaScriptCore [message #27428 is a reply to message #27422] Sat, 17 July 2010 10:55 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

luzr wrote on Sat, 17 July 2010 09:52

Ideally, we should have any such "plugin" managed the same way, which means converted to source based package (like e.g. plugin/jpg).

Do you think anything like that would be possible for any of libraries mentioned here?

Mirek

Hi Mirek,

I like the plugin philosophy for foreign code in U++, so yes, I will try to achieve that. Either I create a package for ICU or, which would be even better, try to get it to compile without it. I hope U++ knows enough unicode to substitute it. I will have a look at the Qt port, hopefully it will give me some hints how they do it.

Anyway, my goal is to keep the JSC code as vanilla as possible to let us upgrade the original sources easily. Ideally just with "svn up".

Honza
Re: JavaScriptCore [message #27440 is a reply to message #27274] Sat, 17 July 2010 23:24 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Good news: I managed to get rid of the ICU library. Bad news: I replaced it with gliband pango. We link with those two on Linux with GUI anyway, but I don't like this solution because it still means supplying those libs on windows and in console and NOGTK builds.

Another good news
is, that I actually have even fully U++ solution, without external libs. The only problem is that I had to use some function stubs to get it working. To get it into production quality, I need to implement (in unicode safe manner):
case folding,
ToLower,
ToUpper,
comparation,
determining the character category (as described for example here).

I suspect that for win32 there should be some native functions to do this, but I am not familiar with windows at all... For Linux, I'm not really sure where to start if I don't wan't to read the whole unicode reference. So I'll be very thankfull for any help or hints where to look.

Honza
Re: JavaScriptCore [message #27442 is a reply to message #27440] Sun, 18 July 2010 08:59 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
dolik.rce wrote on Sat, 17 July 2010 17:24

Good news: I managed to get rid of the ICU library. Bad news: I replaced it with gliband pango. We link with those two on Linux with GUI anyway, but I don't like this solution because it still means supplying those libs on windows and in console and NOGTK builds.

Another good news
is, that I actually have even fully U++ solution, without external libs. The only problem is that I had to use some function stubs to get it working. To get it into production quality, I need to implement (in unicode safe manner):
case folding,
ToLower,
ToUpper,
comparation,
determining the character category (as described for example here).



U++ ToLower/Upper are unicode, at least they are intended to be.

We have some char properties too (actually, it is implemented with the same table:)

We can extend that.

Mirek
Re: JavaScriptCore [message #27454 is a reply to message #27442] Sun, 18 July 2010 23:04 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

luzr wrote on Sun, 18 July 2010 08:59

U++ ToLower/Upper are unicode, at least they are intended to be.

We have some char properties too (actually, it is implemented with the same table:)

We can extend that.

Mirek


Hi Mirek,

I looked at the tables, and I think I got the idea. But it would save me a lot of time if you could point me what information are present in uni__info elements and also on how are they encoded. Is it some standard way, or is it U++ specific? It looks both space and speed efficient, so I guess it is U++ invention Smile

Also, why is there only first 2048 characters? Do you think the next 20000 in the unicode standard are not really important so it is safe to save space in the resulting binaries by omitting them?

I also looked at the Unicode character database. It seems to contain all the information we need in quite friendly formats. It should not be too difficult to convert those files into something useful.

But the main question I have at this moment is: Should we even try to support all this? Everyone using U++ without all those Unicode characters and properties seemed to be quite happy so far. If it would just bloat the library without any added value, than it is useless, right? In such case, I would be more than happy to implement the functions in JavaScriptCore to the same extent as the rest of U++ and never talk about it again Smile

Honza
Re: JavaScriptCore [message #27461 is a reply to message #27454] Mon, 19 July 2010 09:10 Go to previous messageGo to next message
cbpporter is currently offline  cbpporter
Messages: 1401
Registered: September 2007
Ultimate Contributor
Well, I have had a lot of contact with Unicode both in U++ and outside. From a pure completionist and compatibility point of view, when compared to the Unicode standard, U++ is around version 1.1, with some features from latter version but fewer characters and lacking some key features. This would seem very bad, but in practice this is not that bad because everything has very poor Unicode support. As you noticed, there are only 2048 characters in that table, but I think it is safe to assume that most of our users have not hit this barrier. If you live in the United States you are covered. Also, support for most of Europe is very good. Also, a lot of business software has very poor Unicode support. Delphi versions have only recently started using Unicode if I'm not mistaken.

The problem of Unicode is very simple to mask because if you do not want to process the text, only show if, Windows, especially Vista and probably 7 also, is very good at showing it. So your application may not know a thing about Unicode, but you will probably be able to render all text if you have fonts for it. Windows' support is not perfect either. Character composition and ligatures are generally sub par.

In the past I tried to improve Unicode compatibility, but I had little support in my efforts and rightly so because I was having a very specific Unicode issue and I realized that my need was not general.

The problem is that Unicode is very complex. It is not about the character tables. Even implementing a proper ToLower for only the first 2048 characters is a lot more complicated that the current implementation. In Linux it is exponentially more complicated because of the need for very complex character escaping because font support is very poor.

So at this point I believe we have two choices:
1. Continues as we have in the past, having good practical support for common need of Latin alphabet (and variations) using languages, but very poor theoretical compliance to the standard.
2. Go full monty, and add very good support, but the rest the ecosystem being as it is, we will probably be the only ones. Right now in 2010, KDE4, Windows (in general only at rendering) and a few specific scholarly tools have "good" support of Unicode. Standard C++ libs have abyssal support for it. std::wstring is not even Unicode. KDE4 was very good at displaying Unicode in my limited, good at allowing the user to input it, and probably has some potent methods for processing in Qt. Gnome is OK, but not great or as great as KDE. If you live in Europe your software has poor support for anything exotic, and might choke on a few common Unicode strings in various states of normalization, especially when using Mac strings. Probably very few people notice or need better.
Re: JavaScriptCore [message #27585 is a reply to message #27274] Sat, 24 July 2010 15:26 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Hello all,

It took me some time, but I think I am getting closer to the goal. I think this could be called beta release Smile

The attached archive should contain all necessary files for JavaScriptCore and three simple example apps (actually two, jsc_test was created just to be compatible with webkits test suite).

I used mozilla test suite included in webkits source tree to test the quality of this U++ port. It introduces 15 regressions, but hopefully nothing really bad. Most of the failed tests can be traced to the suboptimal unicode handling.

I didn't do any tests on windows lately, but I hope it should work there too (or with minor changes).

Known problems:
Heap leaks in webkits code. (Is there some way to disable them for part of the code?)
The regressions in unicode handling as mentioned.
The file jsc.h must be included before Core.h. (Might be possible to fix with some preprocessor magic)

I'm looking forward to your feedback.

To cbporter: Thanks for your summary on unicode. It helped me a lot. I decided to stick to the option 1 for now, supporting the current level of unicode implementation in U++. Getting full support would take too much time, maybe I will do it as a summer project next year...

Best regards,
Honza
  • Attachment: jsc.zip
    (Size: 1.22MB, Downloaded 195 times)
Re: JavaScriptCore [message #27618 is a reply to message #27585] Mon, 26 July 2010 11:46 Go to previous messageGo to previous message
cbpporter is currently offline  cbpporter
Messages: 1401
Registered: September 2007
Ultimate Contributor
dolik.rce wrote on Sat, 24 July 2010 16:26


To cbporter: Thanks for your summary on unicode. It helped me a lot. I decided to stick to the option 1 for now, supporting the current level of unicode implementation in U++. Getting full support would take too much time, maybe I will do it as a summer project next year...

Best regards,
Honza

No problem. Please let me know if you need more information. And with regards of adding Unicode support, as far as I can tell, you only need it for the native function hooks in Java Script. Making a few functions fully Unicode aware (like ToUpper) coupled with input/output methods should do the trick. There is no need to improve U++'s Unicode support. Especially not the GUI part, which is the most time consuming. So basically doing the least amount of work in order for JS to not regress.
Previous Topic: MultipartForm Class for use w/HttpClient
Next Topic: Disable library functions
Goto Forum:
  


Current Time: Fri Mar 29 01:10:11 CET 2024

Total time taken to generate the page: 0.02732 seconds