Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Developing U++ » U++ Developers corner » CodeEditor with user-definable syntax highlighting
CodeEditor with user-definable syntax highlighting [message #36655] Sun, 24 June 2012 10:01 Go to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Few days ago, I thought that it might be fun to reimplement CodeEditor to support user-defined highlighting as an excersise to avoid my brain slowly rotting from all that Java code I have to write at work Smile I even started working on it in my free time, and here is a little tasting of the work done so far:
index.php?t=getfile&id=3785&private=0
The image shows CodeEditor using the highlight definition file specifying highlighting of higlighting definition files to highlight highlight definition file specifying highlighting of higlighting definition files. Pretty meta, right? Very Happy

It is based on pcre and caching. Basic highlighting works just fine, some of the higher level features (scope highlighting, matching brackets, ...) are not yet implemented and some are using the old CodeEditor code. The goal is to allow loading user specified highlighters, both at compile time (by include) and at run-time. It requires some pcre skill (or copy&paste skill Smile ) to write one, but otherwise it is very simple format as you can see above - it takes only 16 lines to describe C++ Smile This file imitates current CodeEditor:
Quote:

HL_PATTERN(INK_OPERATOR, "[\\Q!+-*/\%&|=[]:?<>.~^\\E]")
HL_PATTERN(INK_KEYWORD, " \\b(__(asm|cdecl|declspec|except|fastcall|finally|inline|int (16|32|64|8)|leave|stdcall|try|uuidof)|auto|bool|break|case| catch|char|class|const|const_cast|continue|default|delete|dl l(ex|im)port|do|double|dynamic_cast|else|enum|explicit|exter n|false|float|for|force_inline|friend|goto|if|inline|int|lon g|mutable|namespace|new|operator|private|protected|public|re gister|reinterpret_cast|return|short|signed|sizeof|static|st atic_cast|struct|switch|template|this|thread|throw|true|try| typedef|typeid|typename|union|unsigned|using|virtual|void|vo latile|while)\\b ")
HL_PATTERN(INK_UPPER, "\\b[A-Z][A-Z_0-9]*\\b")
HL_PATTERN(INK_UPP, "\\bupp\\b")
HL_PATTERN(INK_MACRO, " ^[[:space:]]*#[[:space:]]*(define|include(_next)?|(un|ifn?)d ef|endif|el(if|se)|if|warning|error|pragma|using)\\b ")
HL_PATTERN(INK_UPPMACROS, " \\b(CLASSNAME|(THIS|PTE)BACK([1-4]?)|QUOTE|X?ASSERT|X?NEVER| X?CHECK|NAMESPACE_UPP|END_UPP_NAMESPACE|NEVER_|ASSERT_)\\b ")
HL_PATTERN(INK_UPPLOGS, " \\b([DLR]?DUMP(C|CC|CC|HEX|M)?|[DLR]?LOG(F|HEX|BEGIN|END|BLO CK|HEXDUMP|SRCPOS)?|[DLR]?TIMING|DEBUGCODE|RQUOTE)\\b ")
HL_PATTERN(INK_CONST_INT, "\\b[0-9]+\\b")
HL_PATTERN(INK_CONST_HEX, "\\b0(x|X)[[:xdigit:]]+\\b")
HL_PATTERN(INK_CONST_OCT, "\\b0[0-7]+\\b")
HL_PATTERN(INK_CONST_FLOAT, "\\b[0-9]+\\.[0-9]*\\b")
HL_PATTERN(INK_CONST_STRINGOP, "TODO")
HL_PATTERN(INK_CONST_STRING, "[\"](\\.|[^\"])*[\"]")
HL_PATTERN(INK_COMMENT, "//.*")
HL_PATTERN_ML(INK_COMMENT, "/\\*", "\\*/")
HL_SCOPE("[{]", "[}]")


Best regards,
Honza

PS: Reposted from this thread.
Re: CodeEditor with user-definable syntax highlighting [message #36657 is a reply to message #36655] Sun, 24 June 2012 10:09 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

koldo wrote on Sun, 24 June 2012 09:20

Hello Honza

It is interesting for me. I use a special language and I implemented some very basic syntax highlighting but I would like to use something much better.

Thank you Smile.


Hi Koldo,

Would you like to use it in your application or in TheIDE? I plan to keep the API compatible with current CodeEditor, so they are switchable, but I haven't tried to compile it in TheIDE yet.

There are still some bugs that need to be fixed before it is usable at all. E.g: It does only work in Debug mode now, Optimal breaks it Smile But considering that the main algorithms were written in one night, it is not that bad... I believe it could work reasonably well in couple weeks, if I have enough time.

I'll post updates here to keep you informed and I will upload the code when it gets into shape where I'm not ashamed for it Wink

Honza
Re: CodeEditor with user-definable syntax highlighting [message #36658 is a reply to message #36655] Sun, 24 June 2012 10:25 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Didier wrote on Sun, 24 June 2012 09:55

Hi Honza,

Very intersting Wink

Although I don't need it for the moment, it can get handy when using a DSL or wrting a new one.

What is the perfomance on large files ?


Hi Didier,

The performance is much better than I expected Smile

The worst text I tested it on was uppsrc/plugin/sqlite3/lib/sqlite3.c, which is a 131071 lines long beast. Even current CodeEditor chokes on it quite badly on my system (Intel Atom N280 @1.66GHz, 1Gb RAM) Smile

It tears a lot when scrolling a too quick (i.e. thousands of lines at once), but so does current CodeEditor. This one might be slightly slower according to my benchmarks, but the majority of time is still spent in underlaying LineEditor code in both cases, so the difference is not really noticeable. Id say that subjectively the current and my new implementation are pretty much the same performance-wise.

Also, I haven't done much optimizations yet, only the basics. So it will probably get better over time (compared to current state, not to current CodeEditor code) Wink

Honza
Re: CodeEditor with user-definable syntax highlighting [message #36659 is a reply to message #36658] Sun, 24 June 2012 12:42 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
I think we should definitely add this to theide in the next development cycle...

The question is, can this support something like html? (from the first naive evaluation, I guess not).

Speaking about it, I think that current hightlighting code became a little bit outdated now, I plan to do quite a bit refactoring into more modular approach..

Mirek
Re: CodeEditor with user-definable syntax highlighting [message #36672 is a reply to message #36659] Sun, 24 June 2012 16:40 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

mirek wrote on Sun, 24 June 2012 12:42

I think we should definitely add this to theide in the next development cycle...
Ok, why not. If my code is stable enough by then, of course.

mirek wrote on Sun, 24 June 2012 12:42

The question is, can this support something like html? (from the first naive evaluation, I guess not).
I plan to add support for "scoped matches", mainly as the mechanism behind the scope highlighting. It should be definitely possible to make it work for simple xml files with fixed number of known tags. The more general xml and html-like syntaxes might be little more tricky, but perhaps we can find some tricks to do that as well (it might work with backreferences, but haven't tried it in practice yet).

mirek wrote on Sun, 24 June 2012 12:42

Speaking about it, I think that current hightlighting code became a little bit outdated now, I plan to do quite a bit refactoring into more modular approach..
The current highlighting code and the rest of the CodeEditor are bit too rigid for this I think. Letting user specify the rules opens much more possibilities, but it comes with it's own price. The CParser based approach is probably more efficient, but I have few ideas how to make the pcre perform a bit better too (I'll submit some patches soon Wink ).

Honza
Re: CodeEditor with user-definable syntax highlighting [message #36674 is a reply to message #36657] Sun, 24 June 2012 18:22 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
dolik.rce wrote on Sun, 24 June 2012 10:09

koldo wrote on Sun, 24 June 2012 09:20

Hello Honza

It is interesting for me. I use a special language and I implemented some very basic syntax highlighting but I would like to use something much better.

Thank you Smile.


Hi Koldo,

Would you like to use it in your application or in TheIDE? I plan to keep the API compatible with current CodeEditor, so they are switchable, but I haven't tried to compile it in TheIDE yet.

There are still some bugs that need to be fixed before it is usable at all. E.g: It does only work in Debug mode now, Optimal breaks it Smile But considering that the main algorithms were written in one night, it is not that bad... I believe it could work reasonably well in couple weeks, if I have enough time.

I'll post updates here to keep you informed and I will upload the code when it gets into shape where I'm not ashamed for it Wink

Honza

Hello Honza

In my case I want to include it in separate programs. However if it is enough good it could match in TheIDE.


Best regards
Iñaki
Re: CodeEditor with user-definable syntax highlighting [message #36700 is a reply to message #36674] Tue, 26 June 2012 20:37 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
BTW, looking for the list of css keywords, I have stumbled upon this:

http://code.google.com/p/google-code-prettify/source/browse/ #svn%2Ftrunk%2Fsrc

Could be perhaps helpful for implementing support for other languages...

Mirek
Re: CodeEditor with user-definable syntax highlighting [message #36805 is a reply to message #36655] Sat, 07 July 2012 10:32 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Hi everyone,

As promised, I uploaded the first partly functional version of "CodeEditor2". You can find it in sandbox along with a simple test application that demonstrates an ability to tweak the highlighting at runtime.

There are still some rough corners and some missing functionality, but overall it seems to do the job. Ideas and suggestions are welcomed Wink

Honza

Re: CodeEditor with user-definable syntax highlighting [message #36818 is a reply to message #36805] Sun, 08 July 2012 21:33 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
Great Honza

I am going to use immediately Smile.


Best regards
Iñaki
Re: CodeEditor with user-definable syntax highlighting [message #36819 is a reply to message #36818] Sun, 08 July 2012 22:36 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

koldo wrote on Sun, 08 July 2012 21:33

Great Honza

I am going to use immediately Smile.

Thanks Koldo, but please keep in mind that it is definitely not a stable code. I would not recommend you to use it in production environment. It quite possibly still crashes sometimes and also the code will definitely change a lot before everything is finished.

Honza
Re: CodeEditor with user-definable syntax highlighting [message #36820 is a reply to message #36819] Mon, 09 July 2012 02:22 Go to previous message
koldo is currently offline  koldo
Messages: 3355
Registered: August 2008
Senior Veteran
Oh well. So I will wait ... Rolling Eyes

Best regards
Iñaki
Previous Topic: Console decoration for DOS (windows)
Next Topic: U++ and MinGW 4.7 (-std=c++11) issues
Goto Forum:
  


Current Time: Thu Mar 28 19:38:54 CET 2024

Total time taken to generate the page: 0.01609 seconds