Home » U++ Library support » U++ Core » How to read file into Memory
|
Re: How to read file into Memory [message #12124 is a reply to message #12113] |
Fri, 12 October 2007 00:02 |
|
mirek
Messages: 13975 Registered: November 2005
|
Ultimate Member |
|
|
spidertp wrote on Thu, 11 October 2007 08:05 | Hello everybody,
I need to work with big files (> 100MB) and I like to load part of them to memory in a thread, process data in main app and read next part if needed.
I have 2 tables.
At first I load both of them with 10MB of binary data (so it gives me 20MB in memory). First I proccess data from table1 and when index goes to table2, I'm starting a thread to load new data to table1 and so on. Basically, in main app I want to process data without interrupts due to harddisk access.
The problem is that I have a lot of exceptions because of defining tables with new operator and deleting them.
I'm reading contents of file with code:
object * table1 = new object[10*1024*1024]; // 10MB
FileIn in(filename);
in.Get(table1, sizeof(table1));
After processing I make:
delete table1;
And... here I have a lot of problems with heap.
|
Well, using new / delete is not wise, anyway it should work, what problems exactly do you have?
Quote: |
Can I load a file to memory in a different method than new/delete to some UPP container?
|
Well, I would use Buffer:
Buffer<object> table1(10*1024*1024);
BTW, are you aware that the allocation will most likely be sizeof(object) * 10MB (not only 10MB, if sizeof(object) > 1)?
Mirek
|
|
|
Re: How to read file into Memory [message #12129 is a reply to message #12124] |
Fri, 12 October 2007 08:08 |
spidertp
Messages: 16 Registered: October 2007 Location: Poland
|
Promising Member |
|
|
Thank you Mirek for your answer.
Quote: | Well, I would use Buffer:
Buffer<object> table1(10*1024*1024);
|
I read Container Tutorial but not "Overview of U++ containers - NTL" and I was not aware of Buffer container.
Quote: | BTW, are you aware that the allocation will most likely be sizeof(object) * 10MB (not only 10MB, if sizeof(object) > 1)?
|
Well, I'm dividing 10MB by sizeof(object) and then creating table with number of objects which can fit inside.
Quote: | Well, using new / delete is not wise, anyway it should work, what problems exactly do you have?
|
I think I had problems with delete operator because I used and not operator.
I had exceptions in FreeDebug function.
Thank you or your help and keep doing good work on UPP!
Best regards
Tomasz
Ah, I have one more question about debugging - how can I see specific object in a table or vector?
For example:
struct MyStruct
{
float table[100];
String text;
}
main
{
...
MyStruct myTable[100];
Vector<MyStruct> myVector[100];
myVector[99].table[1] = 15.5;
...
}
In debugging mode, in Explorer I write:
and he says
Only writing myVector is OK, but how can I see specific item in it? Only LOG or DUMP?
|
|
|
|
|
Goto Forum:
Current Time: Sat May 04 22:07:58 CEST 2024
Total time taken to generate the page: 0.02568 seconds
|