|
|
Home » U++ TheIDE » U++ TheIDE: Other Features Wishlist and/or Bugs » [Fixed] TheIde continuously using CPU
[Fixed] TheIde continuously using CPU [message #60143] |
Wed, 13 September 2023 15:39 |
jjacksonRIAB
Messages: 225 Registered: June 2011
|
Experienced Member |
|
|
When I run TheIDE it will never stop using CPU while it's running - using about 2-3%. It did this even when I ran an empty project with no files at all and libclang disabled. By comparison a video player that is actively playing a video uses about 1.5% of CPU on my system. Any idea what it's doing? I run strace and see thousands and thousands of polling calls (resource unavailable).
[Updated on: Wed, 20 September 2023 22:45] Report message to a moderator
|
|
|
Re: TheIde continuously using CPU [message #60144 is a reply to message #60143] |
Wed, 13 September 2023 16:53 |
jjacksonRIAB
Messages: 225 Registered: June 2011
|
Experienced Member |
|
|
After running it I saw the event loop, that was not the issue. I checked the other threads:
ide/background.cpp
void GatherAllFiles(const String& path, Index<String>& filei, VectorMap<String, String>& file)
{
Sleep(0); // This is supposed to be superlazy
for(FindFile ff(path + "/*.*"); ff && !Thread::IsShutdownThreads(); ff.Next())
if(ff.IsFolder() && *ff.GetName() != '.')
GatherAllFiles(ff.GetPath(), filei, file);
else
if(ff.IsFile()) {
String p = NormalizePath(ff.GetPath());
String lp = ToLower(p);
if(filei.Find(lp) < 0) {
filei.Add(lp);
file.Add(GetFileName(p), p);
}
}
}
That Sleep(0) seems to be not very superlazy at all. If I set it to 10 the CPU usage drops. Could this be that Linux handles sleep differently from Windows? I can see the unix version is using nanosleep and I'm not sure what that does with a param of 0 but I'm assuming from what I've read it does nothing... which instead of making it lazy makes it expensive.
[Updated on: Wed, 13 September 2023 19:05] Report message to a moderator
|
|
|
Re: TheIde continuously using CPU [message #60148 is a reply to message #60144] |
Thu, 14 September 2023 11:43 |
|
mirek
Messages: 14155 Registered: November 2005
|
Ultimate Member |
|
|
jjacksonRIAB wrote on Wed, 13 September 2023 16:53After running it I saw the event loop, that was not the issue. I checked the other threads:
ide/background.cpp
void GatherAllFiles(const String& path, Index<String>& filei, VectorMap<String, String>& file)
{
Sleep(0); // This is supposed to be superlazy
for(FindFile ff(path + "/*.*"); ff && !Thread::IsShutdownThreads(); ff.Next())
if(ff.IsFolder() && *ff.GetName() != '.')
GatherAllFiles(ff.GetPath(), filei, file);
else
if(ff.IsFile()) {
String p = NormalizePath(ff.GetPath());
String lp = ToLower(p);
if(filei.Find(lp) < 0) {
filei.Add(lp);
file.Add(GetFileName(p), p);
}
}
}
That Sleep(0) seems to be not very superlazy at all. If I set it to 10 the CPU usage drops. Could this be that Linux handles sleep differently from Windows? I can see the unix version is using nanosleep and I'm not sure what that does with a param of 0 but I'm assuming from what I've read it does nothing... which instead of making it lazy makes it expensive.
That is weird. If you look one level up
void IdeBackgroundThread()
{
while(!Thread::IsShutdownThreads()) {
VectorMap<String, String> file;
Index<String> dir;
Index<String> filei;
for(FindFile ff(ConfigFile("*.var")); ff && !Thread::IsShutdownThreads(); ff.Next()) {
VectorMap<String, String> var;
LoadVarFile(ff.GetPath(), var);
for(String d : Split(var.Get("UPP", ""), ';'))
dir.FindAdd(NormalizePath(d));
Sleep(0);
}
for(String d : dir)
GatherAllFiles(d, filei, file);
{
Mutex::Lock __(s_allfiles_lock);
s_allfiles = pick(file);
s_allnests = dir.PickKeys();
}
for(int i = 0; i < 10 && !Thread::IsShutdownThreads(); i++)
Sleep(100);
}
}
It should wait before doing this for 1 second. The whole purpose of the exercise is to have somewhat actual list of all files of all assemblies (this is then used in comparison menu where now all files with the same name that are somewhat accessible through any assembly are listed - simplifies comparison sources between branches/versions). Putting Sleep(10) into GatherFiles would make it too long to happen.
Can you experiment with that loop at the end? IDK, maybe IsShutdownThreads is broken?
BTW, the idea behing Sleep(0) is to give up CPU is there is more important work to do (this is Thread::StartNice).
Mirek
[Updated on: Thu, 14 September 2023 11:45] Report message to a moderator
|
|
|
|
Re: TheIde continuously using CPU [message #60153 is a reply to message #60149] |
Thu, 14 September 2023 16:04 |
|
mirek
Messages: 14155 Registered: November 2005
|
Ultimate Member |
|
|
jjacksonRIAB wrote on Thu, 14 September 2023 12:18Mirek,
If I set it back to Sleep(0) and then change Sleep(100) to Sleep(1000) in IsBackgroundThread() I get a spike of up to 3-5% (22-33% of a core) every few seconds and then it drops back to nothing.
Well, with Sleep(1000) you should be getting spike each 10 seconds (since the last spike).
With Sleep(100), it should be spike every second.
The length of spike should depend basically on the number of files accessible through all your assemblies.
Well, I was thinking this is innocent and that updated data is more important than small spike every second. But we can increase it a bit I guess..
Would Sleep(1000) work for you?
Mirek
[Updated on: Thu, 14 September 2023 16:04] Report message to a moderator
|
|
|
|
Re: TheIde continuously using CPU [message #60155 is a reply to message #60153] |
Fri, 15 September 2023 03:46 |
jjacksonRIAB
Messages: 225 Registered: June 2011
|
Experienced Member |
|
|
Any chance of using inotify instead? I wrote some example code that appears to work but likely has bugs and could probably be written better.
#include <Core/Core.h>
#include <ide/Core/Core.h>
#include <sys/inotify.h>
using namespace Upp;
void GatherAllFiles(const String& path, Index<String>& filei, VectorMap<String, String>& file)
{
Sleep(0); // This is supposed to be superlazy
for(FindFile ff(path + "/*.*"); ff && !Thread::IsShutdownThreads(); ff.Next())
if(ff.IsFolder() && *ff.GetName() != '.')
GatherAllFiles(ff.GetPath(), filei, file);
else
if(ff.IsFile()) {
String p = NormalizePath(ff.GetPath());
String lp = ToLower(p);
if(filei.Find(lp) < 0) {
filei.Add(lp);
file.Add(GetFileName(p), p);
}
}
}
class INotify {
int fd {};
Index<String> directories {};
Buffer<char> buffer {};
Index<int> watches {};
const int BUFFER_SIZE = 4096;
Vector<String> add;
Vector<String> remove;
public:
void AddWatch(String directory) {
add.Add(directory);
if(directories.Find(directory) < 0) {
auto wd = inotify_add_watch(fd, directory, IN_CREATE | IN_DELETE | IN_DELETE_SELF | IN_MOVED_TO | IN_MOVED_FROM | IN_CLOSE_WRITE);
if(wd == -1) {
LOG("watch failed!");
return;
}
directories.Add(directory);
watches.Add(wd);
}
}
void AddWatchRecursive(String path) {
AddWatch(path);
for(FindFile ff(path + "/*.*"); ff && !Thread::IsShutdownThreads(); ff.Next()) {
if(ff.IsFolder() && *ff.GetName() != '.') {
AddWatchRecursive(ff.GetPath());
}
}
}
void RemoveWatch(String directory) {
auto idx = directories.Find(directory);
if(idx >= 0) {
inotify_rm_watch(fd, watches[idx]);
directories.Unlink(idx);
watches.Unlink(idx);
}
}
void Run() {
const size_t EVENT_SIZE = ( sizeof (struct inotify_event) );
const size_t EVENT_BUF_LEN = ( 1024 * ( EVENT_SIZE + 16 ) );
ssize_t numBytes = read(fd, buffer.begin(), BUFFER_SIZE);
if(numBytes == -1 && errno != EAGAIN) {
LOG("read failed!");
}
struct inotify_event* event = reinterpret_cast<struct inotify_event*>(buffer.begin());
int i = 0;
while(i < numBytes) {
struct inotify_event *event = ( struct inotify_event * ) &buffer[i];
auto idx = watches.Find(event->wd);
if(idx < 0) break;
String name = directories[idx];
name.Cat("/");
name.Cat(event->name);
LOG(name);
if(event->mask & IN_ISDIR) {
if(event->mask & IN_CREATE) {
DirCreated(name);
}
else if(event->mask & IN_DELETE) {
DirDeleted(name);
}
else if(event->mask & IN_DELETE_SELF) {
DirDeleted(name);
}
else if(event->mask & IN_MOVED_FROM) {
DirMovedFrom(name);
}
else if(event->mask & IN_MOVED_TO) {
DirMovedTo(name);
}
}
else {
if(event->mask & IN_CREATE) {
FileCreated(name);
}
else if(event->mask & IN_DELETE) {
FileDeleted(name);
}
else if(event->mask & IN_MOVED_FROM) {
FileMovedFrom(name);
}
else if(event->mask & IN_MOVED_TO) {
FileMovedTo(name);
}
}
i += EVENT_SIZE + event->len;
}
directories.Sweep();
watches.Sweep();
}
INotify() : fd(inotify_init1(IN_NONBLOCK)) {
if(!fd) {
LOG("inotify failed!");
}
LOG("inotify started");
buffer.Alloc(BUFFER_SIZE);
}
~INotify() {
for(int i = 0; i < watches.GetCount(); i++) {
inotify_rm_watch(fd, watches[i]);
}
LOG("inotify closed");
close(fd);
}
Event<String> FileCreated;
Event<String> FileDeleted;
Event<String> FileMovedFrom;
Event<String> FileMovedTo;
Event<String> DirCreated;
Event<String> DirDeleted;
Event<String> DirMovedFrom;
Event<String> DirMovedTo;
};
void IdeBackgroundThread()
{
StdLogSetup(LOG_COUT);
VectorMap<String, String> file;
Index<String> dir;
Index<String> filei;
for(FindFile ff(ConfigFile("*.var")); ff && !Thread::IsShutdownThreads(); ff.Next()) {
VectorMap<String, String> var;
LoadVarFile(ff.GetPath(), var);
for(String d : Split(var.Get("UPP", ""), ';'))
dir.FindAdd(NormalizePath(d));
Sleep(0);
}
INotify notify;
notify.FileCreated = [&](String name) { LOG(Format("File Created: %s", name)); filei.Add(name); };
notify.FileDeleted = [&](String name) { LOG(Format("File Deleted: %s", name)); filei.RemoveKey(name); };
notify.FileMovedFrom = [&](String name) { LOG(Format("File MovedFrom: %s", name)); filei.RemoveKey(name); };
notify.FileMovedTo = [&](String name) { LOG(Format("File MovedTo: %s", name)); filei.Add(name); };
notify.DirCreated = [&](String name) { LOG(Format("Dir Created: %s", name)); dir.Add(name); notify.AddWatchRecursive(name); };
notify.DirDeleted = [&](String name) { LOG(Format("Dir Deleted: %s", name)); dir.RemoveKey(name); notify.RemoveWatch(name); };
notify.DirMovedFrom = [&](String name) { LOG(Format("Dir MovedFrom: %s", name)); dir.RemoveKey(name); notify.RemoveWatch(name); };
notify.DirMovedTo = [&](String name) { LOG(Format("Dir MovedTo: %s", name)); dir.Add(name); notify.AddWatchRecursive(name); };
LOG(dir);
for(String d : dir) {
GatherAllFiles(d, filei, file);
notify.AddWatch(d);
}
for(;;) {
notify.Run();
Sleep(100);
}
}
CONSOLE_APP_MAIN {
auto configFile = ConfigFile("*.var");
LOG(ConfigFile("*.var"));
IdeBackgroundThread();
}
I copied .var files over to the config directory for this project.
|
|
|
|
|
|
|
Goto Forum:
Current Time: Wed Dec 04 15:42:27 CET 2024
Total time taken to generate the page: 0.01753 seconds
|
|
|