Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site











SourceForge.net Logo

Huge

 

class Huge

This class is intended for dealing of large raw binary data. The main use is supposed to be in serialization loading, where it is used to preload large binary data (e.g. String or Image) as means for protection against invalid input - Huge is storing data in 1MB chunks, so is likely to run to the end of stream without allocating invalid amounts of data.

To explain further, the problem we are trying to solve here is

 

void Foo::Serialize(Stream& s)

{

  int len;

  s % len;

  if(s.IsLoading()) {

      ptr = new byte[len];

      s.GetAll(ptr, len);

  }

  .....

}

 

now if input stream is invalid, the allocation is likely to fail. We could prevent this by examining the amount of remaining data in the stream, but for certain class of streams (e.g. compressed ones) we do not have that info. Huge solves the problem by serving as temporary buffer for loaded data:

 

void Foo::Serialize(Stream& s)

{

  int len;

  s % len;

  if(s.IsLoading()) {

           Huge h;

      s.GetAll(h, len);

      ptr = new byte[h.GetSize()];

      h.Get(ptr);

  }

  .....

}

 

Now if stream remaining length is less than len, Huge performs allocation just in 1MB chunks and the end of stream will be reached without allocating too much data. GetAll will then fail with LoadingError. The downside of this step is the fact that data get copied one more time and that for particular element, twice as much memory is required to load it, however, usually these are negligible.

 

Public Method List

 

enum CHUNK = 1024 * 1024 

This is the size of allocation chunk for Huge, currently at 1MB.

 


 

byte *AddChunk()

Adds a chunk and returns a pointer to it. Client code must respect the CHUNK size.

 


 

void Finish(int last_chunk_size)

After adding a last chunk, Huge is 'closed' with calling this method with the size of its last_chunk_size. Usually, Huge gets data from some source (like Stream) which returns full sized chunks until the last one.

 


 

size_t GetSize() const

Returns the number of bytes stored.

 


 

void Get(void *t_, size_t pos, size_t szconst

Reads data into target pointer.

 


 

void Get(void *t)

Same as Get(t, 0, GetSize()).

 


 

String Get() const

Returns data as String. There must be less than 2GB of data or operation fails with Panic.

 

Last edit by cxl on 06/07/2019. Do you want to contribute?. T++