Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » U++ Library support » U++ MT-multithreading and servers » Ask for testing - Skylark with upload progress
Ask for testing - Skylark with upload progress [message #40407] Wed, 31 July 2013 00:11 Go to next message
mdelfede is currently offline  mdelfede
Messages: 1307
Registered: September 2007
Ultimate Contributor
Hi, I made some changes in skylark which will be committed on next days. I'd like if somebody tests it; here is the modified Skylark package, and the upload demo.

As you'll see in demo, a new macro has been added :

SKYLARK_PROGRESS(handler, path, progress)


on which progress is a pointer of a function of type :

int progress(int reason, Http& http, int size)


which is called during various phases of upload:

reason = PROGRESS_HEADERS when headers have been read
reason = PROGRESS_CONTENT during the (lengty) content reading
reason = PROGRESS_END when transfer is finished

The demo performs a multi-file upload with a progress bar.
If you test on localhost, I suggest to slow down the network with some tools, for example NETEM for Linux :

Add network delay on localhost :
sudo tc qdisc add dev lo root netem delay 20ms

Remove network delay on localhost:
sudo tc qdisc del dev lo root netem delay 20ms


I'll update documents before commit on SVN.

Ciao

Max
Re: Ask for testing - Skylark with upload progress [message #40415 is a reply to message #40407] Wed, 31 July 2013 11:19 Go to previous messageGo to next message
Zbych is currently offline  Zbych
Messages: 325
Registered: July 2009
Senior Member
Hi,

I had to modify path to static files in your example (static/SkylarkUpload/SkylarkUpload.css and static/SkylarkUpload/upload.js), but it works.

I think you should add some protection to this app:
- file size limit, now I can crush the app by uploading big file (>4GB on 32-bit system). Client side code (javascript) doesn't recognize this situation and after application restart shows "0%"
- upload directory total size limit

Is it possible to keep uploaded file parts on disk, not in RAM?
I can imagine situation when someone starts upload many times and doesn't finish it to make server run out of memory.
Re: Ask for testing - Skylark with upload progress [message #40416 is a reply to message #40415] Wed, 31 July 2013 11:30 Go to previous message
mdelfede is currently offline  mdelfede
Messages: 1307
Registered: September 2007
Ultimate Contributor
Zbych wrote on Wed, 31 July 2013 11:19

Hi,
I had to modify path to static files in your example (static/SkylarkUpload/SkylarkUpload.css and static/SkylarkUpload/upload.js), but it works.


Yep, I always have problems with static files... I still don't understand fully their behaviour
Quote:


I think you should add some protection to this app:
- file size limit, now I can crush the app by uploading big file (>4GB on 32-bit system). Client side code (javascript) doesn't recognize this situation and after application restart shows "0%"
- upload directory total size limit


Yes, of course.... that one is just a small quick demo, not intended to be used as it is.
I'm developing a cloud server based on Skylark, and this was just a step
Quote:


Is it possible to keep uploaded file parts on disk, not in RAM?
I can imagine situation when someone starts upload many times and doesn't finish it to make server run out of memory.



Not yet, and that depends on Skylark. By now it downloads tha whole http content in ram at once, and then starts processing it.
That's also the reason I passed the 'uploadid' variable inside URI (mixing post and get...), as the POsT part is processed only at the end of http transfer.
I'm convinced too that http content processing should be changed somehow, to allow to decode on the fly the parts already transferred and to allow bigger transfers.
By now it seems limited to 32 bit size, so 4 about 4 giga for the WHOLE content. So transferring 10 files of 500 MB each at once will overflow its capacity.

Btw, the big problem is NOT to keep parts on disk, that's done mostly automatically by OS, but the limit of 4 GB on whole transfer AND the inability of decode already transferred variables during process.

Ciao

Max
Previous Topic: Skylark and X11 name clash
Next Topic: Some additions to Skylark
Goto Forum:
  


Current Time: Fri Mar 29 11:05:50 CET 2024

Total time taken to generate the page: 0.01571 seconds