Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » U++ Library support » ArrayCtrl, HeaderCtrl & GridCtrl » GridCtrl large data set from DB
GridCtrl large data set from DB [message #25324] Wed, 17 February 2010 09:50 Go to next message
lokki is currently offline  lokki
Messages: 20
Registered: November 2005
Promising Member
Hello.

I'm using a GridCtrl in my app to display, search and edit data from the database and starting having problems as the amount of data grows. I believe it's caused by a bad design or bad usage of GridCtrl in my app.

In current implementation, the GridCtrl is filled by with all data from the DB table when window is displayed. I'm also using GridCtrl's search functionality to filter displayed data. The search however is getting really slow (search is immediate as I type into the default search input box in GridCtrl).

Do you have any suggestion on how to handle large data set in GridCtrl? How do you do that?

My idea is, that the GridCtrl won't be filled with all the data at the begining, but just with a subset, say limited to 100 records. The search will have to be overriden by a custom functionality, that it doesn't search the records in GridCtrl's memory, but that it calls a "like" query on the DB and display the output in GridCtrl.

Would that be a good approach or do you have a better idea? I just don't like storing all the data from the database table in the GridCtrl memory, while the amount of data is growing.

Thanks in advance
Re: GridCtrl large data set from DB [message #25328 is a reply to message #25324] Wed, 17 February 2010 12:06 Go to previous messageGo to next message
unodgs is currently offline  unodgs
Messages: 1354
Registered: November 2005
Location: Poland
Senior Contributor

Current GridCtrl's search algorithm is indeed slow. I can improve that, but it won't solve your problem anyway. If your database table size grows constantly no matter how searching implementation efficient will be some day you'll hit the wall. And filling grid with data will be taking more and more time as well as memory space. Another thing worth mentioning here is that users are mostly interested in new/fresh data, so loading everything to the memory is in most cases unnecessary.
In my application I use paging control (see the screen-shot). I wrote it myself and if you want I can publish it (but first I need to separate it from my project and write simple example for it). If any button or arrow is pressed I simply generate new query to database that returns a proper result range. Basically this is the same query used to load all data with offset/limit instructions added. Most databases support them. Also if you need sorting you must do it by changing the database query otherwise you will sort only visible part of data and consistency with paging will be lost.
So as you can see I'm doing the same what you were thinking of.
Other solution is to use virtual rows (but GridCtrl doesn't support them) and to load data on the fly as one moves the scrollbar. But that needs some smart cashing strategy, otherwise scrolling experience will be bad. It's very nice solved in xceed grid - http://xceed.com/Grid_WPF_Intro.html. Just run the demo. Maybe I'll add something similar to GridCtrl one day Smile
And of course grid searching will be working on loaded data only (for current page). You have to provide your own searching method to search all database data.
index.php?t=getfile&id=2295&private=0
  • Attachment: pager.png
    (Size: 4.05KB, Downloaded 587 times)

[Updated on: Wed, 17 February 2010 12:09]

Report message to a moderator

Re: GridCtrl large data set from DB [message #25330 is a reply to message #25328] Wed, 17 February 2010 13:05 Go to previous messageGo to next message
lokki is currently offline  lokki
Messages: 20
Registered: November 2005
Promising Member
Thanks Unodgs for quick reply. So I'm going to try the idea and will see how it will behave.

The search is important part of the application, because user is always interested only into records he searches for.

[Updated on: Wed, 17 February 2010 13:08]

Report message to a moderator

Re: GridCtrl large data set from DB [message #25334 is a reply to message #25330] Wed, 17 February 2010 14:42 Go to previous message
fudadmin is currently offline  fudadmin
Messages: 1298
Registered: November 2005
Location: London, UK
Senior Contributor
Administrator
to Speed up searches you can hide some columns. To avoid searching in all columns I made Skip column in my package:http://www.ultimatepp.org/forum/index.php?t=msg& &th=4970&goto=25191#msg_25191

You can try it. Maybe uno sees it useful and includes into his?

Btw, it would be nice to have virtual grid. I'm strugling myself with 2 mln. records.
Previous Topic: Cells appearance and text alignment
Next Topic: Calculate needed cell width to fit some text
Goto Forum:
  


Current Time: Wed Nov 13 10:35:34 CET 2019

Total time taken to generate the page: 0.01063 seconds