Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Developing U++ » Bazaar » FTP class and reference example for U++
Re: FTP class and reference example for U++ [message #47869 is a reply to message #47859] Fri, 14 April 2017 23:45 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 292
Registered: August 2007
Location: Turkey
Experienced Member
Hello,

As Klugier suggested above, I've re-written the convenience functions, using a more object-oriented approach.
As a result, Ftp convenience functions, both single-threaded and multithreaded variants, are changed, and much less cluttered and more extendible now.

// Single-threaded.
Ftp::Result FtpGet(const Ftp::Request& request, Gate<int64, int64> progress = false, Event<> whenwait = CNULL);
Ftp::Result FtpPut(const Ftp::Request& request, Gate<int64, int64> progress = false, Event<> whenwait = CNULL);

// Multi-threaded.
int FtpAsyncGet(Ftp::Request& request, Event<Ftp::Result> progress = CNULL);
int FtpAsyncPut(Ftp::Request& request, Event<Ftp::Result> progress = CNULL);        


You can find the updated package in the first post of this topic.

FtpBrowser example and FTP package documentation are also updated to reflect the changes.


Regards.

Oblivion.
Re: FTP class and reference example for U++ [message #47876 is a reply to message #47869] Sat, 15 April 2017 19:23 Go to previous messageGo to next message
Klugier is currently offline  Klugier
Messages: 442
Registered: September 2012
Location: Poland, Kraków
Senior Member
Hello,

It is good to hear you introduced my proposals Smile

_________________________________________

I see one minor things to improvement - you could change the parameter name from whenwait to whenWait it will be more readable.

DirEntry&       User(const String& u)                           { vm(USER) = u; return *this; }

Should it be SetUser(const String& u)? Moreover the u parameter is misleading for the API consumer it should be explicitly called user.

_________________________________________

You should also consider moving your enums to c++ style:
 enum Style      { UNIX, DOS }; 

enum class Type { UNIX, DOS }; // You could consider chaining the name to Type

// The user can refere:
DirEntry::Type::UNIX

instead of:
DirEntry::UNIX

_________________________________________

Do we rely need this friendship declaration here?
        ValueMap        vm;
        Bits            owner, group, other;
        friend bool     ParseFtpDirEntry(const String& in, Vector<Ftp::DirEntry>& out);


Why not just create
const ValueMap& GetVm(); // Read only access - i believe ParseFtpDirEntry doesn't set anything?

IMO the variable name "vm" is also misleading. I would call it dirInfo or something more verbose. The same improvment can be used in Result class - just call it resultInfo.

_________________________________________

(Cosmetic - tiny) The class beginning has got problem with readability. It could be:
class Ftp : private NoCopy {
public: // Just remove extra empty line.
	class Request;
	class Result; // Do we need forward declarations in class? I believe when we remove friendship we will not ;)

        class DirEntry final : Moveable<Ftp::DirEntry> {



Sincerely,
Klugier


Ultimate++ - one framework to rule them all.
Re: FTP class and reference example for U++ [message #47877 is a reply to message #47876] Sat, 15 April 2017 19:38 Go to previous messageGo to next message
Klugier is currently offline  Klugier
Messages: 442
Registered: September 2012
Location: Poland, Kraków
Senior Member
Hello,

In my opinion handling multi-threading API using #ifdef in header is not good for the end user. Because, he/seh is obligated to use #ifdef in theire code. Personally, i would hide it inside the interface and use One container (Movable will not work for this class).

class Result // Interface - all methods are abstract
{
public:
   // The same as previous

   virtual bool IsAsync() = 0;

   // Mt methods are presented without #ifdef guard
};

class RegularResult : public Result // The name could be different
{
public:
    bool IsAsync() override { return false; }

    // Mt methods returning falses, -1 etc.
};

class AsyncResult : public Result
{
public:
    bool IsAsync() override { return true; }
    
    // Return correct values for MT
};

One<Result> result(new AsyncResult());



What do you think?

Sincerely,
Klugier


Ultimate++ - one framework to rule them all.
Re: FTP class and reference example for U++ [message #47882 is a reply to message #47877] Sun, 16 April 2017 18:28 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 292
Registered: August 2007
Location: Turkey
Experienced Member
Quote:
Hello,

In my opinion handling multi-threading API using #ifdef in header is not good for the end user. Because, he/seh is obligated to use #ifdef in theire code. Personally, i would hide it inside the interface and use One container (Movable will not work for this class).

class Result // Interface - all methods are abstract
{
public:
// The same as previous

virtual bool IsAsync() = 0;

// Mt methods are presented without #ifdef guard
};

class RegularResult : public Result // The name could be different
{
public:
bool IsAsync() override { return false; }

// Mt methods returning falses, -1 etc.
};

class AsyncResult : public Result
{
public:
bool IsAsync() override { return true; }

// Return correct values for MT
};

One<Result> result(new AsyncResult());



What do you think?

Sincerely,
Klugier



Hello Klugier,

Thank you very much for your constructive criticism and suggestions.


I believe we might have a simpler solution. Smile

FtpAsyncGet() and FtpAsyncPut() functions call FtpAsyncIO().It is possible to return en error code if anything goes wrong.
So we can simply define a function, say, IsMultitihreaded(), to check the U++ multi-threading infrastructure automatically, in Ftp.cpp:


static inline bool IsMultithreaded() 
{
#ifdef flagMT
	return true;
#else
	return false;
#endif
}


Then we can call it in FtpAsyncIO:


Ftp::Result FtpAsyncIO(Ftp::Request request, Event<Ftp::Result> progress, bool put)
{
		// Check if U++ MT is enabled.
	Ftp::Result r;
	try {
		if(!IsMultithreaded())
			throw Exc("Multithreading is not enabled.");


// ...


	}
	catch(Exc& e) {
		// Couldn'r create or run the worker thread.
		r.info("state") = "failed";
		r.info("rc")    = -1;
		r.info("msg")   = e;
		LLOG("-- FTP worker failed. Reason: " << e);
	}
	return r;


In this way we can remove other ifdefs, and user won't be confused.
(Multithreading as a requirement is already mentioned in API docs for each method and function. )

Note that I also changed the return value from int to Ftp::Result. (Now they are similar to single-threaded variants)


Calling an async function such as Result::InProgress() now won't do harm, it wil silently return false.

AS for your other suggestions:

ParseFtpDirEntry() actually modifies the ValueMap. It parses the string into key-value pairs. However, now I put the actual parser code into DirEntry::Parser() method, and got rid of the friend declaration.

Same thing goes for the Request class. So, the ugly forward declarations are removed.

I updated the package to reflect the changes...

Regards,

Oblivion



[Updated on: Sun, 16 April 2017 20:01]

Report message to a moderator

Re: FTP class and reference example for U++ [message #47918 is a reply to message #43053] Sun, 23 April 2017 23:41 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 292
Registered: August 2007
Location: Turkey
Experienced Member
Hello,

Another round of updates.
This time FtpBrowser example has received updates too. But first things first:

FTP package:

 

multithreaded functions are further improved. 

- Available worker slots are limited to 256 ftp worker thread per process.

- It is now possible to pre-set the thread priority of each worker.

- It is now possible to pass a user data (a U++ Value), using Request::UserData() and Result::GetUserData();

- Worker abort mechanism is both simplified and improved.  Vector is ditched in favor of a static Bits instance.
  This reduces both memory consumption and mutex lock/unlock times. Since, previously we needed to iterate over the vector to check if a worker was registered to be aborted.
  This was time consuming. Now we simply set a bit on or off. Since the worker counter resets at 65536 and max. worker slots are limeted to 256, it should be pretty safe.
  See the FtpAsyncIO(), Ftp::AbortWorker() and Ftp::IsWorkerAborted() for details. (I'm open to suggestions, if you have better ideas.) 

- Made room for NetworkProxy support. I will add NetworkProxy support (HTTP/SOCKS4/SOCKS5) via a preprocessor directive (USEPROXY flag), since NetworkProxy package is currently not in bazaar.



FtpBrowser example:

- It is now possible to multiselect the files to be downloaded or uploaded.

- FtpBrowser no longer asks for download location for each download. A default download location can be selected. And it can be changed via environment settings.

- FtpBrowser gained file editing support. While it is still a barebone example with ability to edit only one file at a time, it will change in the near future.
  (Mirek had suggested this when we were discussing SSH package last year on "Developer's Corner", which is on it's way, by the way.)
  Of course this is far from what he suggested, but it is operational in it's current state anyway. :)



That's it for now. Two features (thread priority settings for ftp worker threads, and a ftp text editor for FtpBrowser example) are this weeks highlights.


You can find the updated FTP package in the first post of this topic.


Cheers!

Oblivion.

[Updated on: Sun, 23 April 2017 23:54]

Report message to a moderator

Re: FTP class and reference example for U++ [message #47925 is a reply to message #43053] Mon, 24 April 2017 19:01 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 292
Registered: August 2007
Location: Turkey
Experienced Member
Hello

I've made a video, demonstrating the basic capabilities (concurrent downloads of large files [three 4.5 GB files], remote file editing, browsing, etc.) of FTP package and browser.

Note that the example browser is still under development.)

Just select 1080p (HD)

https://vimeo.com/214530673

Cheers!

Oblivion.

[Updated on: Mon, 24 April 2017 19:04]

Report message to a moderator

Re: FTP class and reference example for U++ [message #47929 is a reply to message #47925] Mon, 24 April 2017 22:12 Go to previous messageGo to next message
Klugier is currently offline  Klugier
Messages: 442
Registered: September 2012
Location: Poland, Kraków
Senior Member
Hello,

For me it is a great news that somebody published video related to U++. You could consider posting URL to our site in the movie description Wink

Backing to FTP - did you consider moving to enum class (c++11 feature) instead of old-c like enum. I think the whole API will gained on this change. For example:
class DirEntry {
// ...
enum Style      { UNIX, DOS }; // In my opinion "Type" world is a better alternative in this situation...
enum class Type { UNIX, DOS };


Usage:
if (dirType == DirEntry::Type::UNIX) {
  // .. do something
}


instead of
if (dirType == DirEntry::UNIX) {
   // ...
}


Much more readable. Isn't it?

Let me ask you one question - do you have GitHub repository for that project?

Sincerely,
Klugier


Ultimate++ - one framework to rule them all.

[Updated on: Mon, 24 April 2017 22:15]

Report message to a moderator

Re: FTP class and reference example for U++ [message #47932 is a reply to message #47929] Tue, 25 April 2017 08:37 Go to previous message
Oblivion is currently offline  Oblivion
Messages: 292
Registered: August 2007
Location: Turkey
Experienced Member
Quote:
For me it is a great news that somebody published video related to U++. You could consider posting URL to our site in the movie description Wink


Done. Credits go where it's due. Smile




Quote:
Backing to FTP - did you consider moving to enum class (c++11 feature) instead of old-c like enum. I think the whole API will gained on this change. For example:


Ok, I'll change this, it won't hurt much, all it requires is some minor adjustments.


Quote:
// In my opinion "Type" world is a better alternative in this situation...


Well this is more about semantics. According to Wiktionary and Oxford Dictionary, a style is a manner of doing, or presenting things, which is what directory listing in FTP is all about.
It does not necessarily say anything about the underlying file system or structure (FTP servers can and sometimes do have UNIX style dir listing, when they actually use Windows/DOS based file structure (and there are even newer and more detailed listing styles (See: https://tools.ietf.org/html/rfc3659) which I'm planning to support in next versions. That's why I refactored the Ftp::DirEntry class, using a ValueMap. It gives us a tremendous flexibility to support almost any dir listing syle used in ftp servers. Parsing of dir listings in ftp is a PITA, with no reliable or standard format around, by the way Smile )
It is about the way of presenting directory entries (they are same type of objects in an ftp context: both are directory listings). So, I'd prefer it stay as it is.






Quote:
Let me ask you one question - do you have GitHub repository for that project?


I do use and prefer git and github for my C/Python/Perl code, but I prefer using SVN for my U++ related projects, for it is supported by TheIDE. Smile
In 2015 I'd created a sourceforge SVN repo, named Ultimate++ Components (of course, with a disclaimer that the repository and the code it contains is not endorsed by
U++ in any way, and that it is simply an unofficial project to externally support the U++ framework.)
Initially it saw some activity, but I had to focus on my other projects and my job, so the repo lied dormant for a long time.
Not anymore Smile I revived it some time ago, and restructuring it now.
I'm uploading the development versions of the components I develop and use, with BSD license.

https://sourceforge.net/projects/ultimatecomponents/


Regards,

Oblivion

[Updated on: Tue, 25 April 2017 09:23]

Report message to a moderator

Previous Topic: JobQueue: A simple and programmable job/queue model.
Goto Forum:
  


Current Time: Tue May 23 14:48:14 CEST 2017

Total time taken to generate the page: 0.03958 seconds