U++ forum: Welcome to the forum

Status & Roadmap

Authors & License

Funding Ultimate++

Search on this site

Search in forums

Home » Community » U++ community news and announcements » Core 2019

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Return to the default flat view

Create a new topic

Submit Reply

Re: Core 2019 [message #51855 is a reply to message #51850]

Sun, 09 June 2019 21:39

Novo is currently offline

Novo
Messages: 1358
Registered: December 2006

Ultimate Contributor

mirek wrote on Sun, 09 June 2019 12:54

Looking at peak profile, it looks like there are very little "small" blocks and most of memory is in those 80 "huge" (that means >64KB) blocks.

Can that be correct?

Mirek

It is hard to tell. I'm not controlling that.
Another problem is that all allocations/deallocations happen in CoWork's threads. I cannot call RDUMP(*PeakMemoryProfile()) inside of CoWork because it will be called at least 181363 times ...

The app is parsing Wikipedia XML dump. It is decompressing a bz2 archive and parsing chunks of XML. After that my own parser is parsing Mediawiki text.
As a first pass my parser is building a list of tokens organized as a Vector<> (I'm not inserting in the middle Smile

Smile

)
My parser is avoiding memory allocation at all possible costs. I'm calling Vector::SetCountR and reusing these vectors. When I need to deal with String I'm using my own not owning data string class.
Unfortunately, I cannot control memory allocation with XmlParser. I have to relay on the default allocator.

Ideally, I'd love to see U++ allocator designed like this.
Related papers:
https://people.cs.umass.edu/~emery/pubs/berger-pldi2001.pdf
https://erdani.com/publications/cuj-2005-12.pdf
https://accu.org/content/conf2008/Alexandrescu-memory-alloca tion.screen.pdf

It doesn't have to be a complete implementation of everything. I just would like to be able plug into U++'s allocator in a similar fashion and extend/tune it.

mirek wrote on Sun, 09 June 2019 14:56

I would like to get a list of allocations your code is doing so that I can hopefully replicate it and investigate whether there can be anything done to reduce the fragmentation.... I will post temporary changes to get the log tomorrow, if you are willing to help.

Yes, I'm willing to help. I even willing to implement this policy-based allocator. I just need the ability to integrate it into U++. It doesn't have to be a part of U++.

Regards,
Novo

Report message to a moderator

Send a private message to this user

[Message index]

		Core 2019 By: mirek on Fri, 07 June 2019 13:56
		Re: Core 2019 By: Novo on Fri, 07 June 2019 17:51
		Re: Core 2019 By: mirek on Fri, 07 June 2019 18:01
		Re: Core 2019 By: Novo on Fri, 07 June 2019 23:00
		Re: Core 2019 By: Novo on Sat, 08 June 2019 18:30
		Re: Core 2019 By: mirek on Sat, 08 June 2019 18:40
		Re: Core 2019 By: Novo on Sat, 08 June 2019 19:44
		Re: Core 2019 By: mirek on Sat, 08 June 2019 20:38
		Re: Core 2019 By: Novo on Sat, 08 June 2019 21:45
		Re: Core 2019 By: Novo on Sat, 08 June 2019 21:42
		Re: Core 2019 By: mirek on Sat, 08 June 2019 18:31
		Re: Core 2019 By: Novo on Sat, 08 June 2019 21:54
		Re: Core 2019 By: Novo on Sat, 08 June 2019 22:06
		Re: Core 2019 By: mirek on Sun, 09 June 2019 10:03
		Re: Core 2019 By: Novo on Sun, 09 June 2019 15:20
		Re: Core 2019 By: mirek on Sun, 09 June 2019 16:33
		Re: Core 2019 By: mirek on Sun, 09 June 2019 16:43
		Re: Core 2019 By: Novo on Sun, 09 June 2019 17:34
		Re: Core 2019 By: mirek on Sun, 09 June 2019 18:54
		Re: Core 2019 By: Novo on Sun, 09 June 2019 21:39
		Re: Core 2019 By: mirek on Sun, 09 June 2019 23:11
		Re: Core 2019 By: mirek on Sun, 09 June 2019 23:25
		Re: Core 2019 By: mirek on Mon, 10 June 2019 17:27
		Re: Core 2019 By: Novo on Mon, 10 June 2019 18:01
		Re: Core 2019 By: mirek on Mon, 10 June 2019 18:17
		Re: Core 2019 By: Novo on Mon, 10 June 2019 18:21
		Re: Core 2019 By: Novo on Mon, 10 June 2019 18:45
		Re: Core 2019 By: Novo on Mon, 10 June 2019 18:18
		Re: Core 2019 By: Novo on Mon, 10 June 2019 18:34
		Re: Core 2019 By: Novo on Sun, 09 June 2019 17:02
		Re: Core 2019 By: Novo on Sun, 09 June 2019 17:15
		Re: Core 2019 By: mirek on Sun, 09 June 2019 20:56
		Re: Core 2019 By: mirek on Sun, 09 June 2019 18:51
		Re: Core 2019 By: Novo on Fri, 21 June 2019 05:43
		Re: Core 2019 By: Novo on Fri, 21 June 2019 06:14
		Re: Core 2019 By: mirek on Fri, 21 June 2019 09:16
		Re: Core 2019 By: mirek on Fri, 21 June 2019 09:23
		Re: Core 2019 By: Novo on Fri, 21 June 2019 19:58
		Re: Core 2019 By: Novo on Fri, 21 June 2019 20:28
		Re: Core 2019 By: Novo on Sat, 22 June 2019 02:51
		Re: Core 2019 By: Novo on Sat, 22 June 2019 02:53
		Re: Core 2019 By: mirek on Sat, 22 June 2019 10:23
		Re: Core 2019 By: Novo on Sat, 22 June 2019 17:49
		Re: Core 2019 By: mirek on Sat, 22 June 2019 22:22
		Re: Core 2019 By: Novo on Sat, 22 June 2019 23:44
		Re: Core 2019 By: mirek on Sun, 23 June 2019 08:21
		Re: Core 2019 By: Novo on Sun, 23 June 2019 21:26
		Re: Core 2019 By: mirek on Mon, 24 June 2019 09:22
		Re: Core 2019 By: Novo on Mon, 24 June 2019 17:36
		Re: Core 2019 By: mirek on Mon, 24 June 2019 18:42
		Re: Core 2019 By: Novo on Mon, 24 June 2019 19:36
		Re: Core 2019 By: mirek on Mon, 24 June 2019 20:15
		Re: Core 2019 By: mirek on Sun, 23 June 2019 09:55
		Re: Core 2019 By: mirek on Sun, 23 June 2019 10:19
		Re: Core 2019 By: Novo on Mon, 24 June 2019 06:13
		Re: Core 2019 By: mirek on Wed, 26 June 2019 08:52
		Re: Core 2019 By: mirek on Thu, 27 June 2019 09:44
		Re: Core 2019 By: mirek on Fri, 28 June 2019 09:09
		Re: Core 2019 By: Tom1 on Fri, 28 June 2019 16:56
		Re: Core 2019 By: mirek on Fri, 28 June 2019 17:30
		Re: Core 2019 By: mirek on Fri, 28 June 2019 17:31
		Re: Core 2019 By: mirek on Fri, 28 June 2019 17:40
		Re: Core 2019 By: Tom1 on Fri, 28 June 2019 18:38
		Re: Core 2019 By: Novo on Sun, 30 June 2019 19:02
		Re: Core 2019 By: mirek on Sun, 30 June 2019 19:11
		Re: Core 2019 By: Novo on Sun, 30 June 2019 20:12
		Re: Core 2019 By: mirek on Mon, 01 July 2019 00:03
		Re: Core 2019 By: Tom1 on Mon, 01 July 2019 09:14
		Re: Core 2019 By: Novo on Thu, 11 July 2019 19:55
		Re: Core 2019 By: mirek on Thu, 11 July 2019 20:14
		Re: Core 2019 By: Novo on Thu, 11 July 2019 20:53
		Re: Core 2019 By: mirek on Fri, 12 July 2019 10:09
		Re: Core 2019 By: Novo on Fri, 12 July 2019 16:50
		Re: Core 2019 By: Novo on Thu, 08 August 2019 21:47
		Re: Core 2019 By: mirek on Thu, 08 August 2019 22:47
		Re: Core 2019 By: mirek on Thu, 08 August 2019 22:53
		Re: Core 2019 By: mirek on Thu, 08 August 2019 22:59
		Re: Core 2019 By: Novo on Thu, 08 August 2019 23:30
		Re: Core 2019 By: mirek on Fri, 09 August 2019 00:05
		Re: Core 2019 By: Novo on Fri, 21 June 2019 17:59
		Re: Core 2019 By: Novo on Fri, 21 June 2019 18:01
		Re: Core 2019 By: Novo on Fri, 21 June 2019 18:10

Previous Topic:	ide: Assist / Display/apply patch
Next Topic:	ide: pkg_config support

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

PDF

]

Current Time: Fri Apr 26 16:35:56 CEST 2024

Total time taken to generate the page: 0.03719 seconds