Anyway, peak and final memory profiles would be nice to know...

(Although one problem is that only calling thread's memory is in the profile).

Well, from what I see, the real difference is that U++ "keeps" the address space...

Maybe you can strace both allocators and grep for mmap / munmap?

Also, profile after the CoWork would be nice to know.

Mirek

Profile after the CoWork was posted in this message.

I believe that is just peak profile, but I might be looking at it wrong.

strace logs seem weird a bit - I do not see enough allocations (either way) for 4GB.

mirek wrote on Sat, 22 June 2019 16:22

I believe that is just peak profile, but I might be looking at it wrong.

What is "Profile after the CoWork"? I do know only about MemoryUsedKb(), MemoryUsedKbMax(), PeakMemoryProfile() ...
In this log-file all three of them are called after CoWork.

My bad. I didn't follow the threads ...
New logs are attached.

"MemoryProfile()"

It would probably be worth to experiment with HPAGE constant... Can be any number >16.

mirek wrote on Sun, 23 June 2019 02:21

"MemoryProfile()"

Attached

Based on this, I do not see any defect or deficiency, except the choosen one (keep the memory).

mirek wrote on Mon, 24 June 2019 03:22

Based on this, I do not see any defect or deficiency, except the choosen one (keep the memory).

Well, I do not think that taking from the system 4.6Gb when the app is allocating ~400Mb is acceptable.
Default behavior of glibc's allocator is optimized for huge enterprise-level apps, but with some manual tweaking it performs fine with small apps. jemalloc would perform even better, I believe.

Well, app actually IS allocating ~4.5 GB at the peak, that is what all indicators show - or have I got that wrong?

mirek wrote on Mon, 24 June 2019 12:42

Well, app actually IS allocating ~4.5 GB at the peak, that is what all indicators show - or have I got that wrong?

Top with default options gives me 4.6GB, 4.5GB or 4.4GB. It depends on a run.

mirek wrote on Sun, 23 June 2019 04:19

It would probably be worth to experiment with HPAGE constant... Can be any number >16.

I'm afraid I cannot afford to spend time on U++'s allocator anymore.
I'm switching to the glibc's one for the time being.

Hi Mirek,

My application started to crash maybe about a week or so ago after updating uppsrc from SVN. Now I got some time to investigate and it seems my app:

- Crashes strangely in different places with current SVN if I compile MSBT19x64 Release
- Works OK with current SVN when compiling with MSBT17x64 Debug or MSBT19x64 Debug
- Works OK with current SVN if I use flag USEMALLOC and compile with MSBT19x64 Release
- Works OK with SVN 13068 compiling with MSBT17x64 Release

Any idea what's wrong? Is it related to the new allocator?

I added some RLOGs and found that it might crash even somewhere in BufferPainter during Stroke... (did not track it all the way down though). Debugger does not help as it does not crash when compiled in debug mode.

Best regards,

Tom

though). Debugger does not help as it does not crash when compiled in debug mode.

I'm getting a crash with a stack trace below when using MemoryAllocPermanent:

Upp::Heap::RemoteFlushRaw (this=<optimized out>) at HeapImp.h:481
Upp::Heap::RemoteFree (this=<optimized out>, ptr=<optimized out>, size=<optimized out>) at HeapImp.h:498
Upp::Heap::Free (this=<optimized out>, ptr=<optimized out>, page=<optimized out>, k=<optimized out>) at sheap.cpp:216
Upp::Heap::Free (this=0x7ffff7a15a10, ptr=<optimized out>) at sheap.cpp:237
Upp::MemoryFree (ptr=<optimized out>) at sheap.cpp:420
judy_close (judy=<optimized out>) at lib/judy.c:147
judy::Map<long long, long long>::~Map (this=<optimized out>) at /home/ssg/dvlp/cpp/sergey/upp/dvlp/plugin/judy/judy.h:48
ConsoleMainFn_ () at test_ht_perf.cpp:98
Upp::AppExecute__ (app=0xfffffffffffffffd) at App.cpp:343
main (argc=-3, argv=0x7ff0b0b5b000, envptr=0x7ffff7a167e8) at test_ht_perf.cpp:8

MemoryAllocPermanent seems to be a replacement for malloc.
The same code using MemoryAlloc works fine.
svn@13460, git@cb77bd58b76a15
Am I doing something wrong or is this a bug?

MemoryAllocPermanent is for allocating memory that is not to be freed (aka is "permanent"). You cannot call MemoryFree on it (as it is "permanent" .

Mirek

mirek wrote on Sun, 30 June 2019 13:11

MemoryAllocPermanent is for allocating memory that is not to be freed (aka is "permanent"). You cannot call MemoryFree on it (as it is "permanent" .

Mirek

Thanks! A couple of lines of documentation would be really helpful ...

In addtion to USEMALLOC there is now HEAPOVERRIDE flag that kicks out all Memory* definitions, allowing you to replace the whole allocator with something else than malloc.

mirek wrote on Thu, 27 June 2019 03:44

In addtion to USEMALLOC there is now HEAPOVERRIDE flag that kicks out all Memory* definitions, allowing you to replace the whole allocator with something else than malloc.

Thanks a lot!

P.S. Documentation for MemoryLimitKb has a typo ...

Novo wrote on Thu, 11 July 2019 19:55

mirek wrote on Thu, 27 June 2019 03:44

In addtion to USEMALLOC there is now HEAPOVERRIDE flag that kicks out all Memory* definitions, allowing you to replace the whole allocator with something else than malloc.

Thanks a lot!

P.S. Documentation for MemoryLimitKb has a typo ...

Must be blind, do not see it....

mirek wrote on Thu, 27 June 2019 03:44

In addtion to USEMALLOC there is now HEAPOVERRIDE flag that kicks out all Memory* definitions, allowing you to replace the whole allocator with something else than malloc.

Thanks a lot!

Novo wrote on Thu, 11 July 2019 19:55

mirek wrote on Thu, 27 June 2019 03:44

In addtion to USEMALLOC there is now HEAPOVERRIDE flag that kicks out all Memory* definitions, allowing you to replace the whole allocator with something else than malloc.

Thanks a lot!

Any chance to get allocator retested? If still unsatisfactory, send me allocation log?

Mirek

I checked "Heap implementation" article from "Help Topics" (which for some reason is not published on web), and it looks like it doesn't match current state of the allocator.

I'm trying to get minimum value of alignment of allocated memory. From a simple test below

	size_t sz = 0;
	void* m = nullptr;
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);

I got that min alignment is 32. Is this correct?
And min block size is 28. This is a little bit weird.
This message states that "the smallest allocation has size 32 and is 32 bytes aligned", which doesn't match the help topic.

Is it possible to expose allocator-related info via a public enum?
Knowing min alignment is critical. Info about block sizes is also important.

TIA