Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Community » U++ community news and announcements » Core 2019
Re: Core 2019 [message #51968 is a reply to message #51963] Fri, 28 June 2019 09:09 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12096
Registered: November 2005
Ultimate Member
I have got another idea. In order to reproduce allocation pattern, there is now

HEAPLOG

flag. If in main config, heap will produce a log of allocation (to heap.log file at exe dir) so that it can be reproduced and optimized.

heap.log is long, but should be highly compressible.
Re: Core 2019 [message #51970 is a reply to message #51968] Fri, 28 June 2019 16:56 Go to previous messageGo to next message
Tom1
Messages: 658
Registered: March 2007
Contributor
Hi Mirek,

My application started to crash maybe about a week or so ago after updating uppsrc from SVN. Now I got some time to investigate and it seems my app:

- Crashes strangely in different places with current SVN if I compile MSBT19x64 Release
- Works OK with current SVN when compiling with MSBT17x64 Debug or MSBT19x64 Debug
- Works OK with current SVN if I use flag USEMALLOC and compile with MSBT19x64 Release
- Works OK with SVN 13068 compiling with MSBT17x64 Release

Any idea what's wrong? Is it related to the new allocator?

I added some RLOGs and found that it might crash even somewhere in BufferPainter during Stroke... (did not track it all the way down though). Debugger does not help as it does not crash when compiled in debug mode.

Best regards,

Tom
Re: Core 2019 [message #51971 is a reply to message #51970] Fri, 28 June 2019 17:30 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12096
Registered: November 2005
Ultimate Member
Tom1 wrote on Fri, 28 June 2019 16:56
Hi Mirek,

My application started to crash maybe about a week or so ago after updating uppsrc from SVN. Now I got some time to investigate and it seems my app:

- Crashes strangely in different places with current SVN if I compile MSBT19x64 Release
- Works OK with current SVN when compiling with MSBT17x64 Debug or MSBT19x64 Debug
- Works OK with current SVN if I use flag USEMALLOC and compile with MSBT19x64 Release
- Works OK with SVN 13068 compiling with MSBT17x64 Release

Any idea what's wrong? Is it related to the new allocator?

I added some RLOGs and found that it might crash even somewhere in BufferPainter during Stroke... (did not track it all the way down though). Debugger does not help as it does not crash when compiled in debug mode.

Best regards,

Tom


Probably allocator, thanks for reporting. It is under active development. It would be worth quoting the revision tested - even today I have fixed / changed some things...

Mirek
Re: Core 2019 [message #51972 is a reply to message #51970] Fri, 28 June 2019 17:31 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12096
Registered: November 2005
Ultimate Member
Tom1 wrote on Fri, 28 June 2019 16:56
though). Debugger does not help as it does not crash when compiled in debug mode.


You can activate debug info for release mode. Sometimes it is very useful.

Mirek
Re: Core 2019 [message #51973 is a reply to message #51972] Fri, 28 June 2019 17:40 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12096
Registered: November 2005
Ultimate Member
For what is worth, I have quickly checked PainterExamples and all is working...
Re: Core 2019 [message #51974 is a reply to message #51973] Fri, 28 June 2019 18:38 Go to previous messageGo to next message
Tom1
Messages: 658
Registered: March 2007
Contributor
Hi Mirek,

Thanks for your reply. I will update from SVN again on Monday morning in the office. Then I will check again. I will also try enabling debug info for release mode.

Thanks and best regards,

Tom
Re: Core 2019 [message #51987 is a reply to message #51974] Sun, 30 June 2019 19:02 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 890
Registered: December 2006
Experienced Contributor
I'm getting a crash with a stack trace below when using MemoryAllocPermanent:

Upp::Heap::RemoteFlushRaw (this=<optimized out>) at HeapImp.h:481
Upp::Heap::RemoteFree (this=<optimized out>, ptr=<optimized out>, size=<optimized out>) at HeapImp.h:498
Upp::Heap::Free (this=<optimized out>, ptr=<optimized out>, page=<optimized out>, k=<optimized out>) at sheap.cpp:216
Upp::Heap::Free (this=0x7ffff7a15a10, ptr=<optimized out>) at sheap.cpp:237
Upp::MemoryFree (ptr=<optimized out>) at sheap.cpp:420
judy_close (judy=<optimized out>) at lib/judy.c:147
judy::Map<long long, long long>::~Map (this=<optimized out>) at /home/ssg/dvlp/cpp/sergey/upp/dvlp/plugin/judy/judy.h:48
ConsoleMainFn_ () at test_ht_perf.cpp:98
Upp::AppExecute__ (app=0xfffffffffffffffd) at App.cpp:343
main (argc=-3, argv=0x7ff0b0b5b000, envptr=0x7ffff7a167e8) at test_ht_perf.cpp:8

MemoryAllocPermanent seems to be a replacement for malloc.
The same code using MemoryAlloc works fine.
svn@13460, git@cb77bd58b76a15
Am I doing something wrong or is this a bug?


Regards,
Novo
Re: Core 2019 [message #51988 is a reply to message #51987] Sun, 30 June 2019 19:11 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12096
Registered: November 2005
Ultimate Member
Novo wrote on Sun, 30 June 2019 19:02
I'm getting a crash with a stack trace below when using MemoryAllocPermanent:

Upp::Heap::RemoteFlushRaw (this=<optimized out>) at HeapImp.h:481
Upp::Heap::RemoteFree (this=<optimized out>, ptr=<optimized out>, size=<optimized out>) at HeapImp.h:498
Upp::Heap::Free (this=<optimized out>, ptr=<optimized out>, page=<optimized out>, k=<optimized out>) at sheap.cpp:216
Upp::Heap::Free (this=0x7ffff7a15a10, ptr=<optimized out>) at sheap.cpp:237
Upp::MemoryFree (ptr=<optimized out>) at sheap.cpp:420
judy_close (judy=<optimized out>) at lib/judy.c:147
judy::Map<long long, long long>::~Map (this=<optimized out>) at /home/ssg/dvlp/cpp/sergey/upp/dvlp/plugin/judy/judy.h:48
ConsoleMainFn_ () at test_ht_perf.cpp:98
Upp::AppExecute__ (app=0xfffffffffffffffd) at App.cpp:343
main (argc=-3, argv=0x7ff0b0b5b000, envptr=0x7ffff7a167e8) at test_ht_perf.cpp:8

MemoryAllocPermanent seems to be a replacement for malloc.
The same code using MemoryAlloc works fine.
svn@13460, git@cb77bd58b76a15
Am I doing something wrong or is this a bug?


MemoryAllocPermanent is for allocating memory that is not to be freed (aka is "permanent"). You cannot call MemoryFree on it (as it is "permanent" Smile.

Mirek
Re: Core 2019 [message #51990 is a reply to message #51988] Sun, 30 June 2019 20:12 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 890
Registered: December 2006
Experienced Contributor
mirek wrote on Sun, 30 June 2019 13:11
MemoryAllocPermanent is for allocating memory that is not to be freed (aka is "permanent"). You cannot call MemoryFree on it (as it is "permanent" Smile.

Mirek

Thanks! A couple of lines of documentation would be really helpful ... Rolling Eyes


Regards,
Novo
Re: Core 2019 [message #51992 is a reply to message #51990] Mon, 01 July 2019 00:03 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12096
Registered: November 2005
Ultimate Member
Novo wrote on Sun, 30 June 2019 20:12
mirek wrote on Sun, 30 June 2019 13:11
MemoryAllocPermanent is for allocating memory that is not to be freed (aka is "permanent"). You cannot call MemoryFree on it (as it is "permanent" Smile.

Mirek

Thanks! A couple of lines of documentation would be really helpful ... Rolling Eyes


You are right. Done.
Re: Core 2019 [message #51996 is a reply to message #51974] Mon, 01 July 2019 09:14 Go to previous messageGo to next message
Tom1
Messages: 658
Registered: March 2007
Contributor
Hi Mirek,

I just downloaded SVN 13462 and now it all works OK again... No crashes.

Thanks and best regards,

Tom
Re: Core 2019 [message #52048 is a reply to message #51963] Thu, 11 July 2019 19:55 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 890
Registered: December 2006
Experienced Contributor
mirek wrote on Thu, 27 June 2019 03:44
In addtion to USEMALLOC there is now HEAPOVERRIDE flag that kicks out all Memory* definitions, allowing you to replace the whole allocator with something else than malloc.

Thanks a lot!

P.S. Documentation for MemoryLimitKb has a typo ...


Regards,
Novo
Re: Core 2019 [message #52049 is a reply to message #52048] Thu, 11 July 2019 20:14 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12096
Registered: November 2005
Ultimate Member
Novo wrote on Thu, 11 July 2019 19:55
mirek wrote on Thu, 27 June 2019 03:44
In addtion to USEMALLOC there is now HEAPOVERRIDE flag that kicks out all Memory* definitions, allowing you to replace the whole allocator with something else than malloc.

Thanks a lot!

P.S. Documentation for MemoryLimitKb has a typo ...


Must be blind, do not see it....
Re: Core 2019 [message #52050 is a reply to message #52049] Thu, 11 July 2019 20:53 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 890
Registered: December 2006
Experienced Contributor
mirek wrote on Thu, 11 July 2019 14:14
Novo wrote on Thu, 11 July 2019 19:55
mirek wrote on Thu, 27 June 2019 03:44
In addtion to USEMALLOC there is now HEAPOVERRIDE flag that kicks out all Memory* definitions, allowing you to replace the whole allocator with something else than malloc.

Thanks a lot!

P.S. Documentation for MemoryLimitKb has a typo ...


Must be blind, do not see it....

"defualt values"


Regards,
Novo
Re: Core 2019 [message #52060 is a reply to message #52048] Fri, 12 July 2019 10:09 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12096
Registered: November 2005
Ultimate Member
Novo wrote on Thu, 11 July 2019 19:55
mirek wrote on Thu, 27 June 2019 03:44
In addtion to USEMALLOC there is now HEAPOVERRIDE flag that kicks out all Memory* definitions, allowing you to replace the whole allocator with something else than malloc.

Thanks a lot!


Any chance to get allocator retested? If still unsatisfactory, send me allocation log?

Mirek
Re: Core 2019 [message #52063 is a reply to message #52060] Fri, 12 July 2019 16:50 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 890
Registered: December 2006
Experienced Contributor
mirek wrote on Fri, 12 July 2019 04:09
Novo wrote on Thu, 11 July 2019 19:55
mirek wrote on Thu, 27 June 2019 03:44
In addtion to USEMALLOC there is now HEAPOVERRIDE flag that kicks out all Memory* definitions, allowing you to replace the whole allocator with something else than malloc.

Thanks a lot!


Any chance to get allocator retested? If still unsatisfactory, send me allocation log?

Mirek

Yes, I can do that, but that will take me a few weeks. My apps are currently in broken state.

Could you please add info about HEAPOVERRIDE and MemoryOptions (it is just briefly mentioned in docs) to documentation?

Also linking to / embedding of Resolving memory leaks into Heap docs would be great because otherwise this info is distributed among multiple topics and is hard to find. I personally didn't know about the --memory-breakpoint__ trick.

And, I guess, it is possible to add new allocators to U++ via plugins now ...


Regards,
Novo
Re: Core 2019 [message #52197 is a reply to message #52063] Thu, 08 August 2019 21:47 Go to previous messageGo to next message
Novo is currently offline  Novo
Messages: 890
Registered: December 2006
Experienced Contributor
I checked "Heap implementation" article from "Help Topics" (which for some reason is not published on web), and it looks like it doesn't match current state of the allocator.

I'm trying to get minimum value of alignment of allocated memory. From a simple test below
	size_t sz = 0;
	void* m = nullptr;
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);

I got that min alignment is 32. Is this correct?
And min block size is 28. This is a little bit weird.
This message states that "the smallest allocation has size 32 and is 32 bytes aligned", which doesn't match the help topic.

Is it possible to expose allocator-related info via a public enum?
Knowing min alignment is critical. Info about block sizes is also important.

TIA


Regards,
Novo
Re: Core 2019 [message #52198 is a reply to message #52197] Thu, 08 August 2019 22:47 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12096
Registered: November 2005
Ultimate Member
Novo wrote on Thu, 08 August 2019 21:47
I checked "Heap implementation" article from "Help Topics" (which for some reason is not published on web), and it looks like it doesn't match current state of the allocator.

I'm trying to get minimum value of alignment of allocated memory. From a simple test below
	size_t sz = 0;
	void* m = nullptr;
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);
	m = MemoryAlloc(1);
	sz = GetMemoryBlockSize(m);

I got that min alignment is 32. Is this correct?
And min block size is 28. This is a little bit weird.
This message states that "the smallest allocation has size 32 and is 32 bytes aligned", which doesn't match the help topic.

Is it possible to expose allocator-related info via a public enum?
Knowing min alignment is critical. Info about block sizes is also important.

TIA


The problem is that the minimal block size is different in debug, as it adds fences everywhere.

In release, 32 holds true.

Mirek
Re: Core 2019 [message #52199 is a reply to message #52198] Thu, 08 August 2019 22:53 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12096
Registered: November 2005
Ultimate Member
PS.: Guaranteed alignment is 16.
Re: Core 2019 [message #52200 is a reply to message #52199] Thu, 08 August 2019 22:59 Go to previous messageGo to previous message
mirek is currently offline  mirek
Messages: 12096
Registered: November 2005
Ultimate Member
Commited:

enum {
	UPP_HEAP_ALIGNMENT = 16,
	UPP_HEAP_MINBLOCK = 32,
};
Previous Topic: ide: Assist / Display/apply patch
Next Topic: ide: pkg_config support
Goto Forum:
  


Current Time: Wed Nov 13 00:34:55 CET 2019

Total time taken to generate the page: 0.01686 seconds