Home » Developing U++ » U++ Developers corner » theide with libclang
| theide with libclang [message #58579] |
Sun, 26 June 2022 09:32  |
 |
mirek
Messages: 14290 Registered: November 2005
|
Ultimate Member |
|
|
In https://github.com/ultimatepp/ultimatepp/tree/llbm_ide branch there lives my recent development effort of integrating theide with libclang.
Now I can proudly announce that I have reached the first milestone: libclang autocomplete is now integrated well enough for me to actually start using this version of theide. Did not tested/fixed outside of windows yet though... (that said, windows seems the most complicated as we have to provide libclang.dll to the release).
Anyway, I would appreciate some more testers for this. In order to compile/use this version in windows, you need libclang.dll from official llvm release and you will also need to add include/lib paths to build method (in future, this will be coming with theide).
Mirek
[Updated on: Thu, 07 July 2022 15:29] Report message to a moderator
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58580 is a reply to message #58579] |
Sun, 26 June 2022 12:53   |
Oblivion
Messages: 1264 Registered: August 2007
|
Senior Contributor |
|
|
Hello Mirek,
I've succesfully compiled the branch on Linux, however, it doesn't seem to work at all. (No assist/code completion, etc...)
(Linux 5.18, GNOME 42.1 (Wayland), LLVM/CLANG 13.0.1 (x64))
Keep in mind that this was just a quick test, Later I'll do some more...
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58581 is a reply to message #58580] |
Sun, 26 June 2022 14:18   |
 |
mirek
Messages: 14290 Registered: November 2005
|
Ultimate Member |
|
|
Oblivion wrote on Sun, 26 June 2022 12:53Hello Mirek,
I've succesfully compiled the branch on Linux, however, it doesn't seem to work at all. (No assist/code completion, etc...)
(Linux 5.18, GNOME 42.1 (Wayland), LLVM/CLANG 13.0.1 (x64))
Keep in mind that this was just a quick test, Later I'll do some more...
Best regards,
Oblivion
Probably RedefineMacros() function problems - in linux, I believe it should not be there at all.
|
|
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58583 is a reply to message #58582] |
Sun, 26 June 2022 18:07   |
 |
mirek
Messages: 14290 Registered: November 2005
|
Ultimate Member |
|
|
OK, good news and bad news (for now):
The problem is caused by CXTranslationUnit_PrecompiledPreamble flag for clang parser. For some reason, when I specify this in Linux, libclang ignores #includes.
Without this flag, it now works. Downside is that it is much slower than in Win32 where CXTranslationUnit_PrecompiledPreamble works...
Now I am pretty sure this must be some problem of my code, as you are actually supposed to use this and I bet other ides are using this flag. But so far I was not able to crack it...
Anyway, you should now be able to test it, hopefully, for initial impressions.
[Updated on: Sun, 26 June 2022 18:08] Report message to a moderator
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58612 is a reply to message #58579] |
Sun, 03 July 2022 16:19   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
I checked a clang-enabled version of TheIDE in Linux (debug version).
It is using 100% of CPU all the time. Initially, I thought that this is Clang indexing UPP files, but after waiting for 50 minutes I figured out that this must be a bug with TheIDE.
Regards,
Novo
|
|
|
|
|
|
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58628 is a reply to message #58626] |
Thu, 07 July 2022 16:33   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
mirek wrote on Thu, 07 July 2022 07:23Can please somebody retest in Linux? I believe autocomplete works now great (I mean, fast enough).
"make -f umkMakefile" - works.
Regular "make" (build TheIDE) didn't work (problem with linking).
Problem with CPU is fixed.
Autocomplete doesn't work for me. For example:
Dot won't be autocompled. Ctrl+Space doesn't help either. I also tried autocomplete with TAB.
To be fair, I never used autocomplete with TheIDE before, but I use it with VIM all the time.
Regards,
Novo
|
|
|
|
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58631 is a reply to message #58629] |
Thu, 07 July 2022 18:00   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
mirek wrote on Thu, 07 July 2022 11:01
Are you testing latest llbm_ide branch, right? (make reference is confusing me a bit)
Yes, I'm using the llbm_ide branch. Last time I tested against rev. 14dad3e.
The only possible problem I can imagine is that I copied .config (all of it) from the previous version of TheIDE.
Regards,
Novo
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58632 is a reply to message #58630] |
Thu, 07 July 2022 18:41   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
mirek wrote on Thu, 07 July 2022 11:02Can you test removing "CXTranslationUnit_PrecompiledPreamble" from sources (you will easily find where)?
I checked that. Removing "CXTranslationUnit_PrecompiledPreamble" doesn't make any difference.
Ctrl+Space uses string on the left as a prefix for search.
Dot doesn't trigger method autocomplition.
Interestingly, umk doesn't check dependencies anymore. I need to "rebuild all" each time.
Regards,
Novo
|
|
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58634 is a reply to message #58633] |
Thu, 07 July 2022 18:55   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
mirek wrote on Thu, 07 July 2022 12:49So Ctrl+Space does show something?
You can also try something selfcontained (no headers) like
struct Foo { int x; };
void foo() { Foo(). };
Mirek
With or without CXTranslationUnit_PrecompiledPreamble?
Regards,
Novo
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58635 is a reply to message #58632] |
Thu, 07 July 2022 18:58   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
Novo wrote on Thu, 07 July 2022 12:41
Interestingly, umk doesn't check dependencies anymore. I need to "rebuild all" each time.
This happens after "git restore". Maybe, git is getting time of last change and setting it on a file ...
It is hard to tell what is going on here.
Regards,
Novo
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58636 is a reply to message #58633] |
Thu, 07 July 2022 19:05   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
mirek wrote on Thu, 07 July 2022 12:49So Ctrl+Space does show something?
You can also try something selfcontained (no headers) like
struct Foo { int x; };
void foo() { Foo(). };
Mirek
In this case it works (with CXTranslationUnit_PrecompiledPreamble and no headers at all).
Regards,
Novo
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58637 is a reply to message #58636] |
Thu, 07 July 2022 19:10   |
 |
mirek
Messages: 14290 Registered: November 2005
|
Ultimate Member |
|
|
Novo wrote on Thu, 07 July 2022 19:05mirek wrote on Thu, 07 July 2022 12:49So Ctrl+Space does show something?
You can also try something selfcontained (no headers) like
struct Foo { int x; };
void foo() { Foo(). };
Mirek
In this case it works (with CXTranslationUnit_PrecompiledPreamble and no headers at all).
Now probably stupid question, I am just asking because I was there before: Are you sure your test code is syntactically correct? Does it compile? Is not it e.g. in #if 0 block?
Also: What happens if you actually add some header? Does it break?
Mirek
[Updated on: Thu, 07 July 2022 19:10] Report message to a moderator
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58638 is a reply to message #58637] |
Thu, 07 July 2022 19:39   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
mirek wrote on Thu, 07 July 2022 13:10
Now probably stupid question, I am just asking because I was there before: Are you sure your test code is syntactically correct? Does it compile? Is not it e.g. in #if 0 block?
I tested with this app.
It is a real app. And autocomplition doesn't work at any place.
I checked with another app. Dot works there ...
Another problem: Ctrl+Space ignores locally defined symbols.
Regards,
Novo
|
|
|
|
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58641 is a reply to message #58640] |
Thu, 07 July 2022 20:44   |
 |
mirek
Messages: 14290 Registered: November 2005
|
Ultimate Member |
|
|
Novo wrote on Thu, 07 July 2022 20:15I'm testing TheIDE on Void Linux (rolling distro), which is using clang version 12.0.1 at this time.
Well, in Windows I am at clang 14, in Linux clang 10 so that is unlikely the issue.
That said, you still have not told me what Ctrl+Space does. Obviously it shows something based on your posts.
Does it show U++ functions e.g.?
Or, opposite, does autocomplete work with std::string? (with proper includes, of course) (in that case there would be a problem with include paths)
Mirek
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58643 is a reply to message #58641] |
Thu, 07 July 2022 21:35   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
mirek wrote on Thu, 07 July 2022 14:44
That said, you still have not told me what Ctrl+Space does. Obviously it shows something based on your posts.
Does it show U++ functions e.g.?
App, where autocompletion works:
bool amap::AnalyzeMap(String fn) {
String map = LoadFile(fn);
char nm[1024];
if (IsNull(map))
return false;
MapFileParser::MSVC MSVC(map);
MapFileParser::CodeWarriorW CWW(map);
MapFileParser::CodeWarrior CW(map);
MapFileParser::PS3 PS3(map);
MapFileParser::GCC3 GCC3(map, mcr);
int addr;
int sect_len;
int prev_size = 0;
uint64 pref_addr;
int size;
int modulen;
int filen;
String sect_name;
String sect_class;
String funct_name;
String module_name;
String file_name;
String app_name;
String time_stamp;
One<MFile> cur_mfile(new MFile);
if (map.StartsWith(elf::ElfMagic, 4)) {
// Elf file.
is32bit = map[4] == elf::ELFCLASS32;
isLittleEndian = map[5] == elf::ELFDATA2LSB;
const uint8 osABI = map[6];
const uint8 abiVersion = map[7];
const elf::Elf32_Ehdr* hdr = reinterpret_cast<const elf::Elf32_Ehdr*>(map.Begin());
If I enter "file_" and press Ctrl+Space I get "file_handle". file_name, which is defined locally, is ignored.
I do not know what Ctrl+Space is supposed to do, but it definitely not doing anything useful to me.
Regards,
Novo
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58644 is a reply to message #58641] |
Thu, 07 July 2022 21:42   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
mirek wrote on Thu, 07 July 2022 14:44
Or, opposite, does autocomplete work with std::string? (with proper includes, of course) (in that case there would be a problem with include paths)
std::string works. Below is my test case.
#include <string>
struct Foo { int x; };
void foo() { Foo(); };
int main(int argc, const char *argv[])
{
std::string str;
str.
return 0;
}
Regards,
Novo
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58645 is a reply to message #58641] |
Thu, 07 July 2022 22:04   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
mirek wrote on Thu, 07 July 2022 14:44
That said, you still have not told me what Ctrl+Space does. Obviously it shows something based on your posts.
Does it show U++ functions e.g.?
In an app, where "dot" doesn't work Ctrl+Space shows functions from the top level (U++ functions). That works.
Regards,
Novo
|
|
|
|
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58648 is a reply to message #58647] |
Sat, 09 July 2022 19:44   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
mirek wrote on Sat, 09 July 2022 07:11
Now it is definitely a temporary solution as it is quite slow that way, hopefully we will find a fix soon.
I tested latest source code.
I do not see any changes.
Basically, source code where autocompletion doesn't work:
bool App::Parse(const String& file_name) {
using namespace conllu;
FileMapping s;
if (!s.Open(file_name))
return false;
if (!s.Map(0, static_cast<dword>(s.GetFileSize())))
return false;
arr.Clear();
sent.Clear();
conlluTree.Clear();
sentv.Clear();
Parser p(reinterpret_cast<const char*>(s.Begin()), reinterpret_cast<const char*>(s.End()), sentv);
while (p.Sentence()) ;
for (const drtree::Sentence& v: sentv)
sent.Add(AsString(v.GetWordV().GetCount()), v.GetText());
return true;
}
In case of "arr.", "sent.", "conlluTree.", e.t.c. autocompletion doesn't work.
In case of "s." and "p." it does.
The only reason LibClang is needed is "name resolution". It is much harder to develop a name resolution algorithm than to make a C++ parser itself. If name resolution doesn't work for some reason, then there is no reason to use LibClang. There are better solutions for parsing.
Regards,
Novo
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58650 is a reply to message #58648] |
Sun, 10 July 2022 00:49   |
 |
mirek
Messages: 14290 Registered: November 2005
|
Ultimate Member |
|
|
Novo wrote on Sat, 09 July 2022 19:44mirek wrote on Sat, 09 July 2022 07:11
Now it is definitely a temporary solution as it is quite slow that way, hopefully we will find a fix soon.
I tested latest source code.
I do not see any changes.
Basically, source code where autocompletion doesn't work:
bool App::Parse(const String& file_name) {
using namespace conllu;
FileMapping s;
if (!s.Open(file_name))
return false;
if (!s.Map(0, static_cast<dword>(s.GetFileSize())))
return false;
arr.Clear();
sent.Clear();
conlluTree.Clear();
sentv.Clear();
Parser p(reinterpret_cast<const char*>(s.Begin()), reinterpret_cast<const char*>(s.End()), sentv);
while (p.Sentence()) ;
for (const drtree::Sentence& v: sentv)
sent.Add(AsString(v.GetWordV().GetCount()), v.GetText());
return true;
}
In case of "arr.", "sent.", "conlluTree.", e.t.c. autocompletion doesn't work.
In case of "s." and "p." it does.
The only reason LibClang is needed is "name resolution". It is much harder to develop a name resolution algorithm than to make a C++ parser itself. If name resolution doesn't work for some reason, then there is no reason to use LibClang.
It works 100% with clang 14 in Windows, hence I think the problem is somewhere else than in clang. It works without preamble in clang 10 for me too. Too soon to try something else at this point.
Now this is one of troubles I expected before going clang - we are now victim of external incompatibilities. But what is done is done.
Quote:There are better solutions for parsing.
And these are?
Mirek
[Updated on: Sun, 10 July 2022 00:52] Report message to a moderator
|
|
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58652 is a reply to message #58650] |
Sun, 10 July 2022 02:53   |
Novo
Messages: 1431 Registered: December 2006
|
Ultimate Contributor |
|
|
mirek wrote on Sat, 09 July 2022 18:49
It works 100% with clang 14 in Windows, hence I think the problem is somewhere else than in clang. It works without preamble in clang 10 for me too. Too soon to try something else at this point.
IMHO, you just haven't tested it very well yet. I was just "lucky" to hit a bug with my first test. My code is not that complicated. It is mostly C++98, but I use a lot of simple templates, but nothing fancy like SFINAE.
mirek wrote on Sat, 09 July 2022 18:49
Now this is one of troubles I expected before going clang - we are now victim of external incompatibilities. But what is done is done.
I do not think this is a Windows/Linux (or version of clang) incompatibility problem. I'm getting the same behavior with VIM and ccls (which is similar to clangd), and with Kakoune and kak-lsp + ccls. I use this technology for several years and I never rely on clang autocompletion, but "goto definition" works very well for me. Even in this broken app "goto definition" of sentv work fine, but for some reason language server is unable to figure out a type of sentv. This is weird. This shouldn't be a problem.
So, I personally feel fine with using LibClang + VIM because VIM has at least half a dozen of different builtin autocompletion tools. I do not care even when LibClang-related tools crash from time to time because they are out of process tools. I just do not know which part of the system (libclang or ccls) is responsible for all these problems.
But in case of TheIDE this can be a bigger problem, IMHO.
Regards,
Novo
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58653 is a reply to message #58652] |
Sun, 10 July 2022 08:46   |
 |
mirek
Messages: 14290 Registered: November 2005
|
Ultimate Member |
|
|
Novo wrote on Sun, 10 July 2022 02:53mirek wrote on Sat, 09 July 2022 18:49
It works 100% with clang 14 in Windows, hence I think the problem is somewhere else than in clang. It works without preamble in clang 10 for me too. Too soon to try something else at this point.
IMHO, you just haven't tested it very well yet. I was just "lucky" to hit a bug with my first test. My code is not that complicated. It is mostly C++98, but I use a lot of simple templates, but nothing fancy like SFINAE.
In Windows, I am using autocomplete for real work (really complex C++17 rich project) for about 14 days. Works flawlessly (of course, gradually maturing some things, but I have not encountered moment when it would fail to provide correct list of methods/variables).
I could try with your code if I could download it somewhere. If I delete "unknown" stuff from short snippets you are posting, it works fine in Windows/libclang 14.
Mirek
|
|
|
|
| Re: theide with libclang, first milestone finished [message #58654 is a reply to message #58653] |
Sun, 10 July 2022 11:06   |
 |
Klugier
Messages: 1117 Registered: September 2012 Location: Poland, Kraków
|
Senior Contributor |
|
|
Hello Mirek,
Few remarks from me while testing new implementation with uppsrc/ide project on Linux.
It doesn't assist with member variables. For example in idewin.cpp (Ide::ConsolePaste()) with following code:
void Ide::ConsolePaste()
{
String s = ReadClipboardText();
if(!IsNull(s)) {
s.Insert(0, '\n');
int len = console.GetLength();
console.Insert(len, s.ToWString());
console.SetCursor(len + 1);
console. // <- No help at all
}
}
The other problem within the same method is that CTRL+Space work with s. It doesn't show functionality with AString, which is essential for working with String:
void Ide::ConsolePaste()
{
String s = ReadClipboardText();
if(!IsNull(s)) {
s.Insert(0, '\n');
int len = console.GetLength();
console.Insert(len, s.ToWString());
console.SetCursor(len + 1);
s. // <- No AString method, only derived method from String
}
}
In the same place when you "include <string>" from standard library, assist will do not show anything:
void Ide::ConsolePaste()
{
String s = ReadClipboardText();
if(!IsNull(s)) {
s.Insert(0, '\n');
int len = console.GetLength();
console.Insert(len, s.ToWString());
console.SetCursor(len + 1);
std::string str2;
str2. // <- No assist...
}
}
Also, I just noticed that CTRL+Space for String is slow for example in release mode. It is not immediate and it takes some time until dialog appear on the screen. The lag is perceptible.
--------------------------
The code navigation works correct and it is fast (CTRL+Left Mouse Button) and for me it looks like improvement.
Klugier
U++ - one framework to rule them all.
[Updated on: Sun, 10 July 2022 11:08] Report message to a moderator
|
|
|
|
|
|
Goto Forum:
Current Time: Sat Apr 25 13:02:29 GMT+2 2026
Total time taken to generate the page: 0.01663 seconds
|