|
|
|
Home » U++ Library support » ArrayCtrl, HeaderCtrl & GridCtrl » Excessive memory use
| Excessive memory use [message #35636] |
Fri, 09 March 2012 05:10  |
nixnixnix
Messages: 415
Registered: February 2007
Location: Kelowna, British Columbia
|
Senior Member |
|
|
Hi Daniel,
I love GridCtrl and use it in many many places in my code. I am currently writing a data processing app in which I want the users to be able to look at multiple data sets and then process these datasets and be able to step back and forth through the steps being able to see each data set at each step. The data sets might take up 20MB each in ASCII format and I might have 12 or 20 loaded at once and then want to take them through 10 or more steps.
My concern is that when I load just three 20MB ASCII data files and display them on GridCtrl, my memory usage grows to 2.9GB of which I am pretty sure the majority is GridCtrl.
My question is: are there obvious things that I might be doing wrong or any tips that you have for conserving memory in GridCtrl when displaying large datasets?
[One of my 20MB datasets might be 4 years of 10 minute data with 30 fields per record. I need to write my software to be able to cope with much much more though.]
Cheers,
Nick
|
|
|
|
|
|
| Re: Excessive memory use [message #35643 is a reply to message #35638] |
Fri, 09 March 2012 14:22  |
|
|
Novo is right. GridCtrl doesn't support virtual rows. They are relatively easy to add but I have no time to do this. All I can do right now is to add it on my todo list 
Besides I think that displaying large data sets is useless. In most cases user is interested in a few first rows (instead of scrolling down user will sort or filter columns). Large sets of data have also another disadvantage. When you move a scroll thumb content of grid is scrolling very fast and you very often skip some data (I assume that scrolling is used to browse data) In my applications I simply use pagination. This is not built in feature of gridctrl but it's quite easy to implement.
It doesn't mean I'm against adding virtual rows/columns. Certainly there are cases when it fits in. Right now I have lot of other work to finish.
[Updated on: Fri, 09 March 2012 14:24] Report message to a moderator |
|
|
|
|
|
| Re: Excessive memory use [message #35660 is a reply to message #35657] |
Sun, 11 March 2012 15:27 |
Novo
Messages: 1430
Registered: December 2006
|
Ultimate Contributor |
|
|
IMHO, you need to call SetVirtualCount(). My app WordNet 3.0 browser is an example of using this technique. It is basically a database browser. I'm not sure that a table, I'm trying to scroll, has 150K rows, but it should be close to that. ArrayCtrl initialization code snippet is below.
warray.AddRowNumColumn("lemma", 80).SetConvert(*Number2Lexeme);
// Set count.
{
String sql;
Sql stmt(db_session);
sql = " SELECT max(wordid)";
sql << " FROM words";
stmt.SetStatement(sql);
stmt.RunX();
if (stmt.Fetch())
warray.SetVirtualCount(stmt[0]);
}
Regards,
Novo
|
|
|
|
|
|
| Re: Excessive memory use [message #35683 is a reply to message #35682] |
Mon, 12 March 2012 22:52 |
Novo
Messages: 1430
Registered: December 2006
|
Ultimate Contributor |
|
|
| nixnixnix wrote on Mon, 12 March 2012 17:44 |
Hi Novo,
SetVirtualCount doesn't appear to change anything. It still takes a minute or more to load up the data from my data structures into the ArrayCtrl one row at a time using ArrayCtrl::Set(int row, Vector<Value> vals); Is there a quicker way please? I can't see anything in the popup list of functions.
Cheers,
Nick
|
The secret of virtual array is that you do not need to load data into it.
You need to do two things:
1) Set number of virtual rows by calling SetVirtualCount().
2) Set converters for your columns, which will convert row number into data you want to display.
Take a closer look at
warray.AddRowNumColumn("lemma", 80).SetConvert(*Number2Lexeme);
There is an app VirtualArray in the "reference" assembly.
Regards,
Novo
[Updated on: Mon, 12 March 2012 22:53] Report message to a moderator |
|
|
|
|
|
| Re: Excessive memory use [message #35685 is a reply to message #35684] |
Tue, 13 March 2012 00:56 |
nixnixnix
Messages: 415
Registered: February 2007
Location: Kelowna, British Columbia
|
Senior Member |
|
|
 Hey Daniel,
My client is mostly interested in being able to examine flags in the data so I colour the flags and then use the panel at the right hand side to display the entire data set and whilst mousemove over that a popup window shows the data above and below the point that the mouse is over in the entire dataset along with the central section of the popup showing what will be on displayed on the main grid if you currently click in the right hand pane.
Just thought I'd share in case these ideas can be useful to others here.
Nick
[Updated on: Tue, 13 March 2012 00:59] Report message to a moderator |
|
|
|
| Re: Excessive memory use [message #35687 is a reply to message #35684] |
Tue, 13 March 2012 04:49 |
Novo
Messages: 1430
Registered: December 2006
|
Ultimate Contributor |
|
|
| nixnixnix wrote on Mon, 12 March 2012 18:08 |
However, when I go to edit a value, all the values in the row show as the row number and not the value that should be there. Is there a good way to deal with this? I noticed that your app doesn't do in place editing and of course the reference app doesn't either.
Nick
|
Well, I made a small experiment with EditField and Convert.
I tried to overwrite Format() and Scan() in Convert and set this new converter on EditField. Now EditField can display correct text instead of a number. Scan() seems to be a right place to store edited value.
Regards,
Novo
[Updated on: Tue, 13 March 2012 04:50] Report message to a moderator |
|
|
|
|
|
Goto Forum:
Current Time: Tue Apr 29 08:21:22 CEST 2025
Total time taken to generate the page: 0.01136 seconds
|
|
|
Members
Pages
Help
Register
Login
Home
