Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Developing U++ » U++ Developers corner » theide with libclang
Re: theide with libclang, first milestone finished [message #58659 is a reply to message #58655] Mon, 11 July 2022 18:40 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
OK, so I have tried to install llvm-14 in my linux distro and linked theide with its libclang - and things started to work even with preamble, totally fine, fast, just as in Windows (except some fixing of include paths to get autocomplete of std:: stuff, but nothing serious; that part is under development anyway).

(Note - if you wonder why I am speaking about preamble so much - preamble is a flag that you give to libclang that makes it create implicit precompiled header in the first file parse. This is very important for autocomplete because we do that parse in background thread and when user needs the autocomplete, all is ready and fast. So it is really important for comfort - using preamble, autocomplete is almost instant (~400ms). Without it can take 1-5 seconds to popup).

So my conclusion: I will only use preamble only when I detect libclang 14, I do not care about outdated distros. Autocomplete will be slow if libclang 14 is not available.

Latest ubuntu has 14, so... Maybe one last thing to try is whether libclang 13 works, if somebody has time to test.

This will be hell in the installation script (to get the most fresh libclang for given distro or even disable libclang completely), but what can we do...

I really would appreciate help with installation script BTW!

Mirek

[Updated on: Mon, 11 July 2022 18:42]

Report message to a moderator

Re: theide with libclang, first milestone finished [message #58660 is a reply to message #58659] Mon, 11 July 2022 21:57 Go to previous messageGo to next message
Klugier is currently offline  Klugier
Messages: 1076
Registered: September 2012
Location: Poland, Kraków
Senior Contributor
Hello Mirek,

libclang-14 requierments explains a lot. On Manjaro current available version is libclang-13, so maybe the issues mentioned by me in my previous message are caused by insufficient libclang library version. Manjaro is rolling release, it means that it has relatively new packages. So, the general adoption of libclang-14 in the Linux word is low.

Quote:
So it is really important for comfort - using preamble, autocomplete is almost instant (~400ms). Without it can take 1-5 seconds to popup).

I can confirm that CTRL+Space take some time to appear on the screen. Is there a way to optimize it for example by pre generating auto complete or using caching? Also, I think user should be somehow inform that something is generating under the hood. For me it looks like TheIDE is unresponsive after using Ctrl+Space. Moreover, ~400ms might be too much. You need to take into consideration that it calculates that time on your CPU. With weaker CPU's this value might be bigger. So, maybe we could also optimize this using similar techniques I proposed.

Could you also tell me from where on Windows I can download libclang-14? I would like to have comparison.

Klugier


U++ - one framework to rule them all.

[Updated on: Mon, 11 July 2022 21:59]

Report message to a moderator

Re: theide with libclang, first milestone finished [message #58661 is a reply to message #58660] Mon, 11 July 2022 22:27 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Klugier wrote on Mon, 11 July 2022 21:57
Hello Mirek,

libclang-14 requierments explains a lot.


OK, some good news. I have finally figured it out and it now works fine with libclang 10 as well Smile (commit 046f816b5a12aae4813cf4a7a83e4b6f11c13ca9)

The problem was actually in those headers, sort of as expected. In windows, I need to add compiler options for header paths even for standard headers. I left that feature in linux as well and it turns out that this is the primary cause of the problem. I guess the real issue is there is too many header paths or maybe duplicating standard paths is confusing clang. All in all, it seems to work great in linux now.

Mirek

[Updated on: Mon, 11 July 2022 22:30]

Report message to a moderator

Re: theide with libclang, first milestone finished [message #58662 is a reply to message #58661] Mon, 11 July 2022 22:49 Go to previous messageGo to next message
Klugier is currently offline  Klugier
Messages: 1076
Registered: September 2012
Location: Poland, Kraków
Senior Contributor
Hello Mirek,

I tested on the latest commit and from my perspective the performance is better, however my feelings are that Assist++ is not supper responsive and you can fell that it takes some time to appear. Could you add a release log file that will measure the Assist++ opening time? From request to appear.

For the rest of the issues I have mentioned in my previous message, everything is the same. Here is screenshot from String without AString methods:
index.php?t=getfile&id=6631&private=0

The console.${Assist++} doesn't open assist.

Klugier


U++ - one framework to rule them all.

[Updated on: Mon, 11 July 2022 22:57]

Report message to a moderator

Re: theide with libclang, first milestone finished [message #58663 is a reply to message #58662] Mon, 11 July 2022 23:00 Go to previous messageGo to next message
Klugier is currently offline  Klugier
Messages: 1076
Registered: September 2012
Location: Poland, Kraków
Senior Contributor
Hello Mirek,

I think it would be nice to add "No suggestions" information when you explicitly hit CTRL+Space. Right now it is difficult to distinguish whenever Assist++ can not find any symbols or it is working under the hood.

Here is the feature from VSCode ("Brak sugesti." = "No sugesttions."):
index.php?t=getfile&id=6632&private=0

Klugier


U++ - one framework to rule them all.
Re: theide with libclang, first milestone finished [message #58746 is a reply to message #58663] Mon, 15 August 2022 10:44 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
So I guess I have reached another (and sort of final) milestone - for more than week I am now working with theide with clang based navigator (that means, it is running 'indexer' in background - the code that continually maintains the database of the whole project). BTW, it takes 20 seconds to reindex theide sources on my machine - BLITZ and 16 cores in action....

Not everything is happy valley though, theide requires far more powerful HW than before. Reindexing consumes a lot of memory (around 6GB) - that is understandable considering we are running 32 C++ compilers in parallel to do it. Also autocomplete is sort of lazy in large projects as compared to my in-house parser, but that is the price to pay I guess...

There is a lot of things to fix and mature yet, my defect list has around 70 entries... Smile Simultaneously, I will be adding nice new shiny features; new theide already got "Search for usages" that seems to be pretty useful.

I have also changed win32 nightly to contain all .dll and headers to be able to compile this llbm_ide branch directly, all you need to make sure is that new ide.exe is placed into the same directory as installed one (you can actually even use "Setup/Upgrade theide.." I think).

The remaining major issue is how to install this in linux....
Re: theide with libclang, first milestone finished [message #58747 is a reply to message #58746] Mon, 15 August 2022 18:44 Go to previous messageGo to next message
Klugier is currently offline  Klugier
Messages: 1076
Registered: September 2012
Location: Poland, Kraków
Senior Contributor
Hello Mirek,

Thanks for the update, I will play with llbm_ide branch.

Quote:
Not everything is happy valley though, theide requires far more powerful HW than before. Reindexing consumes a lot of memory (around 6GB) - that is understandable considering we are running 32 C++ compilers in parallel to do it. Also autocomplete is sort of lazy in large projects as compared to my in-house parser, but that is the price to pay I guess...

As, I understand the number of C++ compiler jobs depends on HYDRA threads parameter. Could it be limited to one thread only if HYDRA threads are set to one? It would be good if TheIDE will still work on all these ARM devices that has single CPU. With only one thread running, the memory requirements should also be lower.

We have a lot of users from countries where there are difficulties to have access to modern hardware. In such cases TheIDE should work fine, even if indexing will take longer.

Scanning operation takes some time, it would be good to let user know that something is happening under the hood. We should avoid situation when user fells confusion. I think we can introduce following things:
- scanning progress is present in status bar (A lot of ide's are using this option...). Status bar is visible by default.
- CTRL+SPACE should indicate that the scanning is in progress and symbols are not ready yet. The information can be displayed in tooltip window.
- CTRL+SPACE should indicate that there are no symbols after scan (please back to my previous message). The same situation it can be displayed in tooltip window. This is exactly what I have presented in my previous message.

Klugier


U++ - one framework to rule them all.

[Updated on: Mon, 15 August 2022 18:48]

Report message to a moderator

Re: theide with libclang, first milestone finished [message #58748 is a reply to message #58747] Tue, 16 August 2022 08:04 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Klugier wrote on Mon, 15 August 2022 18:44
- CTRL+SPACE should indicate that there are no symbols after scan (please back to my previous message). The same situation it can be displayed in tooltip window. This is exactly what I have presented in my previous message.
Klugier


Well, there are always some symbols to display after CTRL+SPACE (unless you are in the completely empty file)...

Better indication of autocomplete in progress is coming..

Mirek


Re: theide with libclang [message #58749 is a reply to message #58579] Tue, 16 August 2022 19:18 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 1358
Registered: December 2006
Ultimate Contributor
Linux. Branch: llbm_ide
Command: make -f umkMakefile
In file included from ./uppsrc/CtrlCore/CtrlCore.h:49,
                 from ./uppsrc/CtrlLib/CtrlLib.h:4,
                 from ./uppsrc/RichEdit/RichEdit.h:4,
                 from ./uppsrc/ide/ide.h:4,
                 from ./uppsrc/ide/Core/Ide.cpp:1:
./uppsrc/CtrlCore/Gtk.h:12:10: fatal error: gtk/gtk.h: No such file or directory
   12 | #include <gtk/gtk.h>
      |          ^~~~~~~~~~~

umk is a console app ...


Regards,
Novo
Re: theide with libclang [message #58750 is a reply to message #58579] Tue, 16 August 2022 19:25 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 1358
Registered: December 2006
Ultimate Contributor
Linux. Branch: llbm_ide
Command: ./umk uppsrc ide CLANG -arbus
Problem: Fatal error Invalid memory access! (on opening a project)


Regards,
Novo
Re: theide with libclang [message #58752 is a reply to message #58749] Fri, 19 August 2022 13:03 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Novo wrote on Tue, 16 August 2022 19:18
Linux. Branch: llbm_ide
Command: make -f umkMakefile
In file included from ./uppsrc/CtrlCore/CtrlCore.h:49,
                 from ./uppsrc/CtrlLib/CtrlLib.h:4,
                 from ./uppsrc/RichEdit/RichEdit.h:4,
                 from ./uppsrc/ide/ide.h:4,
                 from ./uppsrc/ide/Core/Ide.cpp:1:
./uppsrc/CtrlCore/Gtk.h:12:10: fatal error: gtk/gtk.h: No such file or directory
   12 | #include <gtk/gtk.h>
      |          ^~~~~~~~~~~

umk is a console app ...


Fixed.
Re: theide with libclang, first milestone finished [message #58753 is a reply to message #58747] Fri, 19 August 2022 13:04 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Klugier wrote on Mon, 15 August 2022 18:44

- scanning progress is present in status bar (A lot of ide's are using this option...). Status bar is visible by default.


Working on it all, however the idea is that scanning will be represented by left bar changing color (which makes sense as there are annotation that get updated).
Re: theide with libclang, first milestone finished [message #58754 is a reply to message #58753] Fri, 19 August 2022 13:05 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Hopefully fixed Linux crashing in release mode.
Re: theide with libclang, first milestone finished [message #58758 is a reply to message #58754] Sat, 20 August 2022 20:00 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 1358
Registered: December 2006
Ultimate Contributor
mirek wrote on Fri, 19 August 2022 07:05
Hopefully fixed Linux crashing in release mode.

Thanks. TheIde works fine on Linux.


Regards,
Novo
Re: theide with libclang, first milestone finished [message #58759 is a reply to message #58758] Sat, 20 August 2022 20:33 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 1358
Registered: December 2006
Ultimate Contributor
CPU usage.

TheIde is heavily using CPU even if I change one character.
This is a problem for my ten-years-old laptop.
Is it possible to add an options, which will
1) disable libclang completely.
2) limit reindexing to a "Save" command.
3) keep libclang running all the time.

TIA.


Regards,
Novo
Re: theide with libclang, first milestone finished [message #58760 is a reply to message #58754] Sat, 20 August 2022 20:34 Go to previous messageGo to next message
Tom1
Messages: 1212
Registered: March 2007
Senior Contributor
Hi,

I had to move #include <X11/Xlib.h> in GtkApp.cpp in order to compile in Windows with clang:
#include <CtrlCore/CtrlCore.h>


#ifdef GUI_GTK
#include <X11/Xlib.h>
#define CATCH_ERRORS 1


Additionally, compilation fails to missing include in MSBT22x64:
C:\upp-16323\upp.src\uppsrc\ide/clang/clang.h (5): fatal error C1083: Cannot open include file: 'clang-c/Index.h': No such file or directory


Best regards,

Tom
Re: theide with libclang, first milestone finished [message #58761 is a reply to message #58760] Sun, 21 August 2022 00:42 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Tom1 wrote on Sat, 20 August 2022 20:34
Hi,

I had to move #include <X11/Xlib.h> in GtkApp.cpp in order to compile in Windows with clang:
#include <CtrlCore/CtrlCore.h>


#ifdef GUI_GTK
#include <X11/Xlib.h>
#define CATCH_ERRORS 1


Additionally, compilation fails to missing include in MSBT22x64:
C:\upp-16323\upp.src\uppsrc\ide/clang/clang.h (5): fatal error C1083: Cannot open include file: 'clang-c/Index.h': No such file or directory


Best regards,

Tom


Compilation of theide with MSC is not supported at the moment. Not sure it will be possible. Use CLANG64...

Mirek
Re: theide with libclang, first milestone finished [message #58762 is a reply to message #58759] Sun, 21 August 2022 00:46 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Novo wrote on Sat, 20 August 2022 20:33
CPU usage.

TheIde is heavily using CPU even if I change one character.
This is a problem for my ten-years-old laptop.
Is it possible to add an options, which will
1) disable libclang completely.
2) limit reindexing to a "Save" command.
3) keep libclang running all the time.

TIA.


Disable is needed. However, reindexing is already issued only in specific moments (save being one of them). The problem is that initially it has to do it all.

Also, changing headers can trigger reindexing of a lot of files.

This is the price to pay.... (this is the reason I was reluctant to do abandon my parser....)

Mirek
Re: theide with libclang, first milestone finished [message #58780 is a reply to message #58762] Tue, 30 August 2022 16:04 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Good news: I have found a bug in my code (unitialised variable for all things) that was making autocomplete slow. It now seems to be superfast after the initial parse.

Things appear to fall in place. The last major hurdle is to figure out Linux installation. The new idea to try is to use runtime .so loading...

Mirek
Re: theide with libclang, first milestone finished [message #58784 is a reply to message #58780] Wed, 31 August 2022 10:00 Go to previous messageGo to previous message
Tom1
Messages: 1212
Registered: March 2007
Senior Contributor
Hi Mirek,

Each time I have e.g.:
	void LeftDown(Point p, dword keyflags) override {

I get a row: "WARNING: EMIT" in my Navigator. If I remove the override, it goes away. (I do not think this is anything new, but it shows up in the latest version too.)

Second, there are some warnings (at least on MSBT):
C:\upp-git\upp.src\uppsrc\Core\Other.h(123): warning C4267: 'argument': conversion from 'size_t' to 'int', possible loss of data
C:\upp-git\upp.src\uppsrc\Core\Other.h(143): warning C4267: 'argument': conversion from 'size_t' to 'int', possible loss of data
C:\upp-git\upp.src\uppsrc\CtrlLib/DropChoice.h(83): warning C4099: 'Upp::PopUpList::Popup': type name first seen using 'struct' now seen using 'class'
C:\upp-git\upp.src\uppsrc\CtrlLib/DropChoice.h(54): note: see declaration of 'Upp::PopUpList::Popup'


Best regards,

Tom
Previous Topic: Impressive improvement in stl::vector when dealing with raw memory.
Next Topic: about unicode supporting
Goto Forum:
  


Current Time: Sun Apr 28 12:39:18 CEST 2024

Total time taken to generate the page: 0.03843 seconds