Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search in forums












SourceForge.net Logo



Home » U++ Library support » U++ Core » Writing Bits object to disk
Writing Bits object to disk [message #47390] Wed, 11 January 2017 17:07 Go to next message
crydev is currently offline  crydev
Messages: 129
Registered: October 2012
Location: Netherlands
Experienced Member
Hello,

I was thinking about using Bits as an efficient data structure to write my data to disk. However, I noticed that this data structure is quite closed and does not allow callers to retrieve a pointer to the internal buffer of bits and neither does it allow itself being constructed from existing buffer and alloc variables.

Would it be a good idea to allow this, or have a similar data structure that allows retrieval of the data structure? If not, why do you think so? I now manually edited some support in, to see if my efficient idea works out well!

Thanks!

crydev
Re: Writing Bits object to disk [message #47396 is a reply to message #47390] Thu, 12 January 2017 01:20 Go to previous messageGo to next message
mr_ped is currently offline  mr_ped
Messages: 773
Registered: November 2005
Location: Czech Republic - Praha
Veteran

If your target is small disk space, you should rather allocate data in memory in reasonable common sizes, semantically grouped together, and run that through zlib compression, if those data resemble at least some patterns, this should save lot more, than packing them into bits and later having headache when you will want to extend them a bit.
Re: Writing Bits object to disk [message #47400 is a reply to message #47396] Thu, 12 January 2017 09:03 Go to previous messageGo to next message
crydev is currently offline  crydev
Messages: 129
Registered: October 2012
Location: Netherlands
Experienced Member
mr_ped wrote on Thu, 12 January 2017 01:20
If your target is small disk space, you should rather allocate data in memory in reasonable common sizes, semantically grouped together, and run that through zlib compression, if those data resemble at least some patterns, this should save lot more, than packing them into bits and later having headache when you will want to extend them a bit.


For random values this would be a good idea, but not for structured data that I want to write out. I have a sorted set of memory addresses, identified by the memory page they reside in. I can reduce space by at most 32 times if I save it in bits, and at most 8 times (in x64) if I use a Vector<bool>. Compression takes too much time, I tried. Smile
Re: Writing Bits object to disk [message #47411 is a reply to message #47390] Fri, 13 January 2017 08:39 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 10897
Registered: November 2005
Ultimate Member
crydev wrote on Wed, 11 January 2017 17:07
Hello,

I was thinking about using Bits as an efficient data structure to write my data to disk. However, I noticed that this data structure is quite closed and does not allow callers to retrieve a pointer to the internal buffer of bits and neither does it allow itself being constructed from existing buffer and alloc variables.

Would it be a good idea to allow this, or have a similar data structure that allows retrieval of the data structure? If not, why do you think so? I now manually edited some support in, to see if my efficient idea works out well!

Thanks!

crydev


I guess this is valid idea. Adding to RM.

Mirek
Re: Writing Bits object to disk [message #47457 is a reply to message #47411] Wed, 18 January 2017 08:51 Go to previous message
crydev is currently offline  crydev
Messages: 129
Registered: October 2012
Location: Netherlands
Experienced Member
mirek wrote on Fri, 13 January 2017 08:39
crydev wrote on Wed, 11 January 2017 17:07
Hello,

I was thinking about using Bits as an efficient data structure to write my data to disk. However, I noticed that this data structure is quite closed and does not allow callers to retrieve a pointer to the internal buffer of bits and neither does it allow itself being constructed from existing buffer and alloc variables.

Would it be a good idea to allow this, or have a similar data structure that allows retrieval of the data structure? If not, why do you think so? I now manually edited some support in, to see if my efficient idea works out well!

Thanks!

crydev


I guess this is valid idea. Adding to RM.

Mirek


Great, thanks!

crydev
Previous Topic: RegExp this'n that
Next Topic: Add compilable testcases for nontrivial problems!
Goto Forum:
  


Current Time: Wed Jan 25 00:17:25 CET 2017

Total time taken to generate the page: 0.00424 seconds