U++ forum: Welcome to the forum

Re: How to read file into Memory [message #12124 is a reply to message #12113]

Fri, 12 October 2007 00:02

mirek
Messages: 14267
Registered: November 2005

Ultimate Member

spidertp wrote on Thu, 11 October 2007 08:05

Hello everybody,
I need to work with big files (> 100MB) and I like to load part of them to memory in a thread, process data in main app and read next part if needed.

I have 2 tables.
At first I load both of them with 10MB of binary data (so it gives me 20MB in memory). First I proccess data from table1 and when index goes to table2, I'm starting a thread to load new data to table1 and so on. Basically, in main app I want to process data without interrupts due to harddisk access.

The problem is that I have a lot of exceptions because of defining tables with new operator and deleting them.

I'm reading contents of file with code:

object * table1 = new object[10*1024*1024]; // 10MB
FileIn in(filename);
in.Get(table1, sizeof(table1));

After processing I make:

delete table1;

And... here I have a lot of problems with heap.

Well, using new / delete is not wise, anyway it should work, what problems exactly do you have?

Quote:

Can I load a file to memory in a different method than new/delete to some UPP container?

Well, I would use Buffer:

Buffer<object> table1(10*1024*1024);

BTW, are you aware that the allocation will most likely be sizeof(object) * 10MB (not only 10MB, if sizeof(object) > 1)?

Mirek

Report message to a moderator