Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search in forums












SourceForge.net Logo



Home » Developing U++ » Bazaar » FTP class and reference example for U++
FTP class and reference example for U++ [message #43053] Sun, 27 April 2014 18:11 Go to next message
Oblivion is currently offline  Oblivion
Messages: 268
Registered: August 2007
Location: Turkey
Experienced Member
Hello,

Ftp class is a U++ client side implementation of the File Transfer Protocol as specified in RFC 959.



Features and highlights:
- No external C libraries. Written completely in U++, using U++ technologies, package comes with full public API document for Topic++, and has a typical BSD license.
- Uses U++ streams for data transfers.  
- IPV6 connections, using EPRT and EPSV commands. Ftp class will always try these extended commands first, and in case of failure it will fallback to IPv4 only PORT and PASV commands.
- Adjustable data chunk size for binary data transfers, using ChunkSize() method. Default chunk size is 64K.
- Nested Ftp::DirEntry class and related helper functions for easy parsing of DOS and UNIX style directory listings.
- Basic FTPS: FTP over TLS/SSL (per session) through "explicit" negotiation mode, as specified in RFC 2228. When FTPS mode is enabled, Ftp class will first request a TLS mode and try SSL if rejected.  


History:
(06-03-2017): GetReplyAsXml() method is added. This method simplifies the parsing of FTP protocol messages and internal messages.
(30-01-2017): From now on FTP package requires at least C++11.
(01-20-2015): Fixed Active mode IPV6 data connection. Improved FTPS. From now on, Ftp class supports also TLS. (As long as it is supported by U++ Core/SSL package)
(01-07-2015): Added basic FTPS (SSL) support. FTPS is limited to passive mode connections and can only be invoked per ftp session (latter is a security measure).    
(12-29-2014): Possible namspace clash fixed (Thanks Klugier!).
(12-25-2014): FtpGet() and FtpPut() convenience functions are added to the package. These helper functions simplfy ftp transfers while retaining flexibility and features of Ftp class.
(12-24-2014): Complete rewrite of Ftp class. (API breakage). Supports IPv6 and adjustable data chunk size. Abort() sequence is refined. Ftp command line example removed. 
(05-12-2014): Fixed DecodePath() method and download/upload start marker code 125 is now properly handled. Also, a simple Ftp browser example with non-blocking GUI is added.
(05-08-2014): Added Noop() method. Useful for keeping connections alive.
(05-07-2014): This is the first public beta. Package uploaded with a very simple command line ftp client. A non-blocking GUI example (an FTP Browser) will follow.  


ToDo:
- Network proxy (HTTP, SOCKS) support, using NetworkProxy class.


Suggestions, bug reports, patches, criticism are always welcome.

Regards,
Oblivion

[Updated on: Sun, 19 March 2017 00:25]

Report message to a moderator

Re: FTP class and reference example for U++ [message #43104 is a reply to message #43053] Wed, 07 May 2014 01:28 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 268
Registered: August 2007
Location: Turkey
Experienced Member
Hello guys,

Today I publish the first public beta of the FTP package (Link to the updated versions of the package can be found in the first post).

What is missing: Aside from the todo list in the first post, currently only GetReplyAsXml() method is missing,. but it will be added within this week. Also, I removed most of the internal error messages, I'll add them later.
I also wrote a FtpBrowser with a non-blocking GUI (not multithreaded), which will demonstrate how easy it is to set up a non-blcoking gui with Ftp and on U++.. It will be added as a reference example.
But not uploaded this time (code needs cleaning). Ftp class is also a demonstartion of TcpSocket based Server and Client sockets, since it actively uses both type of sockets.

And of course there will be bugs. I am sure because Ftp protocol is, by itself, a total mess. Smile

So, suggestions, bug reports, patches are always welcome.

Ftp class is tested under Linux 3.12+ and Windows (XP/7).

Cheers!

[Updated on: Wed, 07 May 2014 01:34]

Report message to a moderator

Re: FTP class and reference example for U++ [message #43115 is a reply to message #43053] Thu, 08 May 2014 11:52 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 268
Registered: August 2007
Location: Turkey
Experienced Member

Update: NOOP command is added to the Ftp class. (See the first post for package), .

Regards.
Re: FTP class and reference example for U++ [message #43127 is a reply to message #43053] Tue, 13 May 2014 02:03 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 268
Registered: August 2007
Location: Turkey
Experienced Member
Hello guys,

I updated the Ftp class and reference examples package (see the first post for package).
Reference examples package now contains a very simple (for demonstration only) Ftp Browser with non-blocking GUI.
This version of the example allows only one download at a time, but a multithreaded example which will allow multiple downloads, will follow. (It will be very similar to the U++ Reference/GuiMT example),
FtpBrowser utilizes two ftp objects (a browser and a worker). This makes possible to navigate while download. Also it demonstrates how to use FtpDirEntry class in conjunction with the FileList class.
Here is the screenshot of the ftp browser example.

index.php?t=getfile&id=4536&private=0

Bug reports, suggestions, requests, etc. are always welcome.


Regards.

[Updated on: Tue, 13 May 2014 02:06]

Report message to a moderator

Re: FTP class and reference example for U++ [message #43130 is a reply to message #43127] Wed, 14 May 2014 09:25 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 10953
Registered: November 2005
Ultimate Member
There already exists plugin/ftp. Could you please summarize advantages of your new implementation (before I will dwell into it myself...)?
Re: FTP class and reference example for U++ [message #43132 is a reply to message #43130] Wed, 14 May 2014 13:43 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 268
Registered: August 2007
Location: Turkey
Experienced Member
Hello Mirek,

While it is far from perfect, and I'm not sure if the below mentioned features count as advantages, aside from the basic operations, but here they are:

1) Ftp class is completely based on native Upp classes. It is derived from TcpSocket. Therefore supports its features (except asynchronous connections, due to FTP protocols synchronic nature).
This makes bug tracking and understanding the source code easy (also ftp class source code is around 600 lines, and meant to be clean and concise). No external lib, no C code, no need for external memory management, etc..
2) It comes with a directory entry and file information parser (FtpDirEntry class) which makes parsing (both DOS and UNIX based directory listing) as simple as it can be (of course getting the raw Listing strings is also possible (see CLI based ftp example).
3) It is designed the U++ GUI in mind; For example, FtpDirEntry class perfectly fits with FileList class (see the supplied ftp browser example). and achieving a non-blocking gui is very easy.
4) Ftp class does work on U++ streams (any kind), not simply strings (so it can take advantage of FileIn or FileOut streams, without any other effort, which means reduced code).
5) Ftp class api can be extended via SendCommand method (see CLI based ftp example) by the user/developer (there are many useful FTP extension commands out there, added later).
6) Ftp class is not just based on RFC 959. It also takes into account some practical deviations (D. J. Bernstein's recommendations: http://cr.yp.to/ftp.html) from the original draft.
7) Proper support for aborting transfers.
8 ) Ftp class will support IPV6 (actually It already does in development branch, and I will merge it soon).
9) Ftp class will support proxies.
10) I am also planning to add SSL/TLS support.
11) It is documented. (API document is complete, and I am currently writing a document on its mode of operations).
12) It is actively developed.


I believe the code is in good shape (after all, it is in beta state), but there is always plenty of room for improvement, and bugs that I don't know of yet.

Regards.

[Updated on: Wed, 14 May 2014 14:15]

Report message to a moderator

Re: FTP class and reference example for U++ [message #44089 is a reply to message #43053] Wed, 24 December 2014 12:06 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 268
Registered: August 2007
Location: Turkey
Experienced Member
Hello guys,

Ftp class is updated (rewritten, actually). It now takes advantage of EPRT and EPSV commands, which means it has from now on IPv6 support.

See first post for details and package.

Regards,
Oblivion.
Re: FTP class and reference example for U++ [message #44114 is a reply to message #43053] Thu, 25 December 2014 18:03 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 268
Registered: August 2007
Location: Turkey
Experienced Member
Hello,


Two convenience functions are added to the Ftp package: FtpGet and FtpPut:
int FtpGet(const String& path, Stream& out, const String& host, int port = 21, const String& user = Null, const String& pass = Null, 
	Gate2<int64, int64> progress = false, Callback whenwait = CNULL, int type = Ftp::BINARY, int mode = Ftp::PASSIVE);

int FtpPut(Stream& in, const String& path, const String& host, int port = 21, const String& user = Null, const String& pass = Null, 
	Gate2<int64, int64> progress = false, Callback whenwait = CNULL, int type = Ftp::BINARY, int mode = Ftp::PASSIVE);


These functions can simplify Ftp download and upload operations, while retaining the flexibility of the Ftp class. Ftp browser example is also updated to reflect this addition.

Regards,
Oblivion.

[Updated on: Thu, 25 December 2014 18:08]

Report message to a moderator

Re: FTP class and reference example for U++ [message #44146 is a reply to message #43053] Wed, 07 January 2015 20:52 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 268
Registered: August 2007
Location: Turkey
Experienced Member
Hello,

Ftp class gained basic FTPS (Ftp over SSL, to be specific) support. From now on Ftp class can connect to secure Ftp servers, using "explicit mode" negotiation. See RFC 2228 for details.

There are some limitations though:

1) Currently only SSL is supported. TLS is not yet supported.
2) FTPS can be only activated per ftp session. Namely, it must be enabled or disabled prior to an Ftp::Connect() call.
3) Active connection mode and FTPS are mutually exclusive. When FTPS is enabled, the ftp data retrieval methods (such as Ftp::ListDir(), Ftp::Put() or Ftp::Get(), or convenience funtions FtpPut() and FtpGet()) will fail in active connection (PORT, EPRT) mode.

Bu reports, criticism, patches, etc. are welcome.

Regards,
Oblivion.
Re: FTP class and reference example for U++ [message #44188 is a reply to message #43053] Tue, 20 January 2015 01:17 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 268
Registered: August 2007
Location: Turkey
Experienced Member
Hello,

It seems that U++ Core/SSL package already supports TLS connections (see: http://www.ultimatepp.org/forums/index.php?t=msg&th=9090 &goto=43822&#msg_43822 and https://www.openssl.org/docs/ssl/SSL_CTX_new.html).
So I added AUTH TLS support to the Ftp class.
I also tested this new feature with a secure ftp server which requires TLS and it apparently works well.

Ftp class will first explicity request a TLS mode and in case of rejection it will try SSL mode.
See first post for the details and the updated package.


Suggestions, bug reports, patches, criticism are always welcome.

Regards,
Oblivion.

[Updated on: Tue, 20 January 2015 01:25]

Report message to a moderator

Re: FTP class and reference example for U++ [message #44843 is a reply to message #44188] Tue, 07 July 2015 19:40 Go to previous messageGo to next message
unodgs is currently offline  unodgs
Messages: 1349
Registered: November 2005
Location: Poland
Senior Veteran

Hi Oblivion!

Great package. It works well on windows as well but do you plan to add SFTP (SSH File Transfer Protocol)?
PS: I would be grateful for adding upload in your example.

[Updated on: Tue, 07 July 2015 19:43]

Report message to a moderator

Re: FTP class and reference example for U++ [message #44846 is a reply to message #44843] Wed, 08 July 2015 08:50 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 268
Registered: August 2007
Location: Turkey
Experienced Member
Hello Daniel!

Glad to hear that someone found it useful.

Sure. The example is rudimentary.
I will add upload/delete/rename/create etc. options to the example. And turn it into a usable FTP browser in the following days.


SFTP, on the other hand, needs some think-through. I had thought of adding a SSH client library to U++ last year, but I did not have time to implement it then. (it is a somewhat complex protocol, and I don't want to end up creating something half-baked.)
IMO the best and easiest option is to create a U++ wrapper for libssh2 which is AFAIK cross-platform.
So, I will have spare time in the following days and I can start implementing a SFTP client utilizing all the U++ goodies and having usual U++ type API, based on libssh2.
In the meantime maybe I will also resurrect my U++ SSH client library project (But I'm not making any promises on this. Smile ).

What do you think?


Regards,

Oblivion

[Updated on: Wed, 08 July 2015 09:05]

Report message to a moderator

Re: FTP class and reference example for U++ [message #44849 is a reply to message #44846] Wed, 08 July 2015 10:20 Go to previous messageGo to next message
unodgs is currently offline  unodgs
Messages: 1349
Registered: November 2005
Location: Poland
Senior Veteran

For me sounds like a great news. Thank you Oblivion in advance!
PS: libssh2 looks like a good choice (pure C, easy to wrap up)
Re: FTP class and reference example for U++ [message #47752 is a reply to message #43053] Sat, 18 March 2017 23:28 Go to previous message
Oblivion is currently offline  Oblivion
Messages: 268
Registered: August 2007
Location: Turkey
Experienced Member
Hello guys,

It's time for some spring cleaning. Smile

Ftp class is updated.

From now on it requires at least C++11.

And GetReplyAsXml() method is added. This method simplifies the parsing of FTP protocol messages and internal messages, using simple XML tags.


FtpExample is also updated so that it can work. I know it is a crude example but I'm in the process of writing a better version of it with multithreading in mind.
It'll be replaced asap.

As usual, you can find the updated package in the first post of this topic.

Regards,

Oblivion.

[Updated on: Sat, 18 March 2017 23:31]

Report message to a moderator

Previous Topic: JobQueue: A simple and programmable job/queue model.
Goto Forum:
  


Current Time: Tue Mar 28 00:21:43 CEST 2017

Total time taken to generate the page: 0.15437 seconds