|
|
Home » Developing U++ » UppHub » FTP class and reference example for U++
Re: FTP class and reference example for U++ [message #47869 is a reply to message #47859] |
Fri, 14 April 2017 23:45   |
Oblivion
Messages: 1202 Registered: August 2007
|
Senior Contributor |
|
|
Hello,
As Klugier suggested above, I've re-written the convenience functions, using a more object-oriented approach.
As a result, Ftp convenience functions, both single-threaded and multithreaded variants, are changed, and much less cluttered and more extendible now.
// Single-threaded.
Ftp::Result FtpGet(const Ftp::Request& request, Gate<int64, int64> progress = false, Event<> whenwait = CNULL);
Ftp::Result FtpPut(const Ftp::Request& request, Gate<int64, int64> progress = false, Event<> whenwait = CNULL);
// Multi-threaded.
int FtpAsyncGet(Ftp::Request& request, Event<Ftp::Result> progress = CNULL);
int FtpAsyncPut(Ftp::Request& request, Event<Ftp::Result> progress = CNULL);
You can find the updated package in the first post of this topic.
FtpBrowser example and FTP package documentation are also updated to reflect the changes.
Regards.
Oblivion.
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
|
|
|
|
|
Re: FTP class and reference example for U++ [message #47882 is a reply to message #47877] |
Sun, 16 April 2017 18:28   |
Oblivion
Messages: 1202 Registered: August 2007
|
Senior Contributor |
|
|
Quote:Hello,
In my opinion handling multi-threading API using #ifdef in header is not good for the end user. Because, he/seh is obligated to use #ifdef in theire code. Personally, i would hide it inside the interface and use One container (Movable will not work for this class).
class Result // Interface - all methods are abstract
{
public:
// The same as previous
virtual bool IsAsync() = 0;
// Mt methods are presented without #ifdef guard
};
class RegularResult : public Result // The name could be different
{
public:
bool IsAsync() override { return false; }
// Mt methods returning falses, -1 etc.
};
class AsyncResult : public Result
{
public:
bool IsAsync() override { return true; }
// Return correct values for MT
};
One<Result> result(new AsyncResult());
What do you think?
Sincerely,
Klugier
Hello Klugier,
Thank you very much for your constructive criticism and suggestions.
I believe we might have a simpler solution. 
FtpAsyncGet() and FtpAsyncPut() functions call FtpAsyncIO().It is possible to return en error code if anything goes wrong.
So we can simply define a function, say, IsMultitihreaded(), to check the U++ multi-threading infrastructure automatically, in Ftp.cpp:
static inline bool IsMultithreaded()
{
#ifdef flagMT
return true;
#else
return false;
#endif
}
Then we can call it in FtpAsyncIO:
Ftp::Result FtpAsyncIO(Ftp::Request request, Event<Ftp::Result> progress, bool put)
{
// Check if U++ MT is enabled.
Ftp::Result r;
try {
if(!IsMultithreaded())
throw Exc("Multithreading is not enabled.");
// ...
}
catch(Exc& e) {
// Couldn'r create or run the worker thread.
r.info("state") = "failed";
r.info("rc") = -1;
r.info("msg") = e;
LLOG("-- FTP worker failed. Reason: " << e);
}
return r;
In this way we can remove other ifdefs, and user won't be confused.
(Multithreading as a requirement is already mentioned in API docs for each method and function. )
Note that I also changed the return value from int to Ftp::Result. (Now they are similar to single-threaded variants)
Calling an async function such as Result::InProgress() now won't do harm, it wil silently return false.
AS for your other suggestions:
ParseFtpDirEntry() actually modifies the ValueMap. It parses the string into key-value pairs. However, now I put the actual parser code into DirEntry::Parser() method, and got rid of the friend declaration.
Same thing goes for the Request class. So, the ugly forward declarations are removed.
I updated the package to reflect the changes...
Regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Sun, 16 April 2017 20:01] Report message to a moderator
|
|
|
Re: FTP class and reference example for U++ [message #47918 is a reply to message #43053] |
Sun, 23 April 2017 23:41   |
Oblivion
Messages: 1202 Registered: August 2007
|
Senior Contributor |
|
|
Hello,
Another round of updates.
This time FtpBrowser example has received updates too. But first things first:
FTP package:
multithreaded functions are further improved.
- Available worker slots are limited to 256 ftp worker thread per process.
- It is now possible to pre-set the thread priority of each worker.
- It is now possible to pass a user data (a U++ Value), using Request::UserData() and Result::GetUserData();
- Worker abort mechanism is both simplified and improved. Vector is ditched in favor of a static Bits instance.
This reduces both memory consumption and mutex lock/unlock times. Since, previously we needed to iterate over the vector to check if a worker was registered to be aborted.
This was time consuming. Now we simply set a bit on or off. Since the worker counter resets at 65536 and max. worker slots are limeted to 256, it should be pretty safe.
See the FtpAsyncIO(), Ftp::AbortWorker() and Ftp::IsWorkerAborted() for details. (I'm open to suggestions, if you have better ideas.)
- Made room for NetworkProxy support. I will add NetworkProxy support (HTTP/SOCKS4/SOCKS5) via a preprocessor directive (USEPROXY flag), since NetworkProxy package is currently not in bazaar.
FtpBrowser example:
- It is now possible to multiselect the files to be downloaded or uploaded.
- FtpBrowser no longer asks for download location for each download. A default download location can be selected. And it can be changed via environment settings.
- FtpBrowser gained file editing support. While it is still a barebone example with ability to edit only one file at a time, it will change in the near future.
(Mirek had suggested this when we were discussing SSH package last year on "Developer's Corner", which is on it's way, by the way.)
Of course this is far from what he suggested, but it is operational in it's current state anyway. :)
That's it for now. Two features (thread priority settings for ftp worker threads, and a ftp text editor for FtpBrowser example) are this weeks highlights.
You can find the updated FTP package in the first post of this topic.
Cheers!
Oblivion.
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Sun, 23 April 2017 23:54] Report message to a moderator
|
|
|
|
Re: FTP class and reference example for U++ [message #47929 is a reply to message #47925] |
Mon, 24 April 2017 22:12   |
 |
Klugier
Messages: 1099 Registered: September 2012 Location: Poland, Kraków
|
Senior Contributor |
|
|
Hello,
For me it is a great news that somebody published video related to U++. You could consider posting URL to our site in the movie description 
Backing to FTP - did you consider moving to enum class (c++11 feature) instead of old-c like enum. I think the whole API will gained on this change. For example:
class DirEntry {
// ...
enum Style { UNIX, DOS }; // In my opinion "Type" world is a better alternative in this situation...
enum class Type { UNIX, DOS };
Usage:
if (dirType == DirEntry::Type::UNIX) {
// .. do something
}
instead of
if (dirType == DirEntry::UNIX) {
// ...
}
Much more readable. Isn't it?
Let me ask you one question - do you have GitHub repository for that project?
Sincerely,
Klugier
U++ - one framework to rule them all.
[Updated on: Mon, 24 April 2017 22:15] Report message to a moderator
|
|
|
Re: FTP class and reference example for U++ [message #47932 is a reply to message #47929] |
Tue, 25 April 2017 08:37   |
Oblivion
Messages: 1202 Registered: August 2007
|
Senior Contributor |
|
|
Quote:For me it is a great news that somebody published video related to U++. You could consider posting URL to our site in the movie description Wink
Done. Credits go where it's due. 
Quote:Backing to FTP - did you consider moving to enum class (c++11 feature) instead of old-c like enum. I think the whole API will gained on this change. For example:
Ok, I'll change this, it won't hurt much, all it requires is some minor adjustments.
Quote:// In my opinion "Type" world is a better alternative in this situation...
Well this is more about semantics. According to Wiktionary and Oxford Dictionary, a style is a manner of doing, or presenting things, which is what directory listing in FTP is all about.
It does not necessarily say anything about the underlying file system or structure (FTP servers can and sometimes do have UNIX style dir listing, when they actually use Windows/DOS based file structure (and there are even newer and more detailed listing styles (See: https://tools.ietf.org/html/rfc3659) which I'm planning to support in next versions. That's why I refactored the Ftp::DirEntry class, using a ValueMap. It gives us a tremendous flexibility to support almost any dir listing syle used in ftp servers. Parsing of dir listings in ftp is a PITA, with no reliable or standard format around, by the way )
It is about the way of presenting directory entries (they are same type of objects in an ftp context: both are directory listings). So, I'd prefer it stay as it is.
Quote:Let me ask you one question - do you have GitHub repository for that project?
I do use and prefer git and github for my C/Python/Perl code, but I prefer using SVN for my U++ related projects, for it is supported by TheIDE. 
In 2015 I'd created a sourceforge SVN repo, named Ultimate++ Components (of course, with a disclaimer that the repository and the code it contains is not endorsed by
U++ in any way, and that it is simply an unofficial project to externally support the U++ framework.)
Initially it saw some activity, but I had to focus on my other projects and my job, so the repo lied dormant for a long time.
Not anymore I revived it some time ago, and restructuring it now.
I'm uploading the development versions of the components I develop and use, with BSD license.
https://sourceforge.net/projects/ultimatecomponents/
Regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Tue, 25 April 2017 09:23] Report message to a moderator
|
|
|
Re: FTP class and reference example for U++ [message #48274 is a reply to message #43053] |
Wed, 14 June 2017 00:06   |
Oblivion
Messages: 1202 Registered: August 2007
|
Senior Contributor |
|
|
Hello,
FTP package is updated.
With this new version, Ftp class finally gained transfer restart (resume) capability for both (binary) uploads and downloads. 
The update is also reflected in the ftp browser example.
FTP browser example can resume/restart failed or aborted transfers (if the restart feature is supported by the connected server, of course.)
Also the ftp browser now allows drag and drop (for uploads).
Also, you'll notice two nested packages within the FtpBrowser example: BrowserCtrl and TrackerCtrl.
These widgets, which I'm preparing to publish as seperate packages, are resuable ctrls.
BrowserCtrl is very similar to FileSel, but is embeddable and provides a simple api for a flexible, file system transparent file browser.
It is under heavy development.
TrackerCtrl is a transfer tracker. It has a generic api which can be used for tracking different type of file transfers (local file copy, SFTP, FTP, WEBDAV etc.) at once.
It is under heavy development too.
Which brings us to the next step: SSH (SFTP) package. Finally I have time to finish the technical preview version of SSH package.
I'll publish it within couple of weeks. And turn the FtpBrowser into a profile-based multibrowser capable of handling Local files, FTP, SFTP, WEBDAV (my next goal), etc.
Any suggestions, criticism, bug reports are always welcome.
Updated package can be found in the first post of this topic.

Tested on Arch Linux (kernel: 4 11.3) and Windows 7/10
Cheers!
Oblivion.
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Wed, 14 June 2017 00:13] Report message to a moderator
|
|
|
|
|
Re: FTP class and reference example for U++ [message #49680 is a reply to message #43053] |
Mon, 02 April 2018 00:17  |
Oblivion
Messages: 1202 Registered: August 2007
|
Senior Contributor |
|
|
Hello,
It's been a long while since the last update.
A new and major release of FTP class is finally here: Version 2.0.0
New version is far superior to the older (v 1.2) version of FTP package. Basically it is simpler yet more powerful.
API-wise, although I've tried hard to keep the old interface intact, and partially succeded in this, some breakages were necessary.
Yet I will also maintain the old code for some time.
I had to redesign the Ftp class to let it take advantage of the non-blocking interface I used in SSH pacakge. Now they are almost identical. 
Here are the major changes:
MAJOR RELEASE: Ftp package, version 2.0.0
Necessary API breakage.
Ftp is redesigned, using an effective, and simple queue-model similar to the SSH package's.
Old multithreading code completely ditched in favor of the new AsyncWork based ftp workers.
This resulted in noticable performance gain, and simplicity.
Non-blocking mode is finally implemented. Ftp class can do non-blocking calls easily.
FTPS mode improved. It is now possible to inspect the SSL information after handshake.
Support for user-defined consumer functions (for incoming data) is added.
A simple ftp URL scheme is implemented. This scheme replaces (partially) the old Ftp::Request class.
UTF-8 path name encoding support is added.
Optional Connect/Login() methods pair added. It is now possible to execute commands before login.
Ftp::DirEntry now uses pcre to parse listing strings.
Logging is improved.
Reference examples are added to the package.
Reference examples provided with the package:
- FtpGet: Demonstrates a basic FTP file download in blocking mode.
- FtpGetNB: Demonstrates a basic FTP file download in non-blocking mode.
- FtpGetMT: Demonstrates a basic Ftp file download, using worker threads.
- FtpMultiGetMT: Demonstrates FTP dir listing and concurrent file transfers, using worker threads.
- FtpGUI: Demonstrates a basic FTP browser with GUI (with upload, download, mkdir, rename, delete commands).
- FtpOverTLS: Demonstrates the secure connection capability of FTP package in blocking mode.
- FtpQueryFeatures: Demonstrates the feature query mechanism, as defined in RFC 2389
- FtpRawCommand: Demonstrates FTP raw command execution in blocking mode.
As usual, the updated package can be found in the first message of this topic.
And you can also grab it from:
Latest code is located at: https://github.com/ismail-yilmaz/upp-components/tree/master/ Core/FTP
Reference examples can be found at: https://github.com/ismail-yilmaz/upp-components/tree/master/ Examples
Older version (1.2) can be found at: https://github.com/ismail-yilmaz/upp-components/tree/master/ Attic/FTP
A screenshot of the simple FtpGUI example (GUI is inspired by the qt's example):

Suggestions, bug reports, patches, etc. are appreciated.
Best regards,
Oblivion
-
Attachment: FtpGUI.png
(Size: 138.14KB, Downloaded 631 times)
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Mon, 02 April 2018 13:38] Report message to a moderator
|
|
|
Goto Forum:
Current Time: Fri Apr 25 12:46:14 CEST 2025
Total time taken to generate the page: 0.01175 seconds
|
|
|