SSH package for U++
--------------------
SSH package is a feautre-rich, flexible yet very easy to use libssh2 wrapper for Ultimate++.
It supports both console and GUI-based applications on POSIX-compliant operating systems and
MS Windows (tm).

Currently it is in beta (version 1) stage.

Classes:
--------------------

- Base (core) class  -> Ssh
  - Ssh session        -----> SshSession
  - Sftp subsystem     -----> SFtp
  - Ssh channel        -----> SshChannel
      - Scp channel                 -----> Scp
      - Exec channel                -----> SshExec
      - Real-time interactive shell -----> SshShell
      - X11 forwarding              -----> SshShell (as operation mode)      
      - Tcp/IP and port forwarding  -----> SshTunnel

- Known hosts manager -> SshHosts

Features and Highlights:
--------------------

- Ssh-derived classes have pick semantics, based on RAII principle, support RTTI, and allow
  polymorphism (i.e. different classes can be stored in the same arrays, etc.) through a common
  interface. 
- Uses U++ memory allocators (Native allocators is also a compile-time option)
- Uses OpenSSL by default. 
- Supports time-constrained, blocking and non-blocking operation modes.
- Supoorts multithreaded file transfers and remote command execution (exec), using worker threads.
- Supports 3rd-party network proxies. (Such as NetProxy)
- Supports known hosts verification mechanism.
- Supports password, public key, host-based, and keyboard-interactive authentication methods.
- Supports ssh-agents.
- Supports real-time interactive command line (shell) interface with both console and GUI integration (works on Windows and Posix-compliant OS'es)
- Supports multiple X11 connection forwarding.
- Supports Tcp/IP and port forwarding.
- Supports detailed (full) debug logging.

Todo:
--------------------
- Add more high level methods.
- Refactor Ssh (core) class.
- Improve documentation.

Reference examples:
-------------------

- SFtpGet:           Demonstrates basic SFtp file download in blocking mode.
- SFtpGetNB:         Demonstrates basic SFtp file download in non-blocking mode.
- SFtpGetMT:         Demonstrates basic SFtp file download, using a worker thread.
- SFtpGUI:           Demonstrates a basic SFtp browser with GUI (with upload, download, mkdir, rename, delete commands).
- SFtpMultiGetMT:    Demonstrates SFtp dir listing and concurrent file transfers, using worker threads.
- SFtpConsumerGet:   Demonstrates the usage of a consumer function for SFtp download in blocking mode.
- SFtpConsumerGetMT: Demonstrates the usage of a consumer function for SFtp download, using a worker thread.
- SshExec:           Demonstrates basic Ssh remote command execution in blocking mode.
- SshExecNB:         Demonstrates basic Ssh remote command execution in non-blocking mode.
- SshExecMT:         Demonstrates basic Ssh remote command execution, using a worker thread.
- SshKeyboardAuth:   Demonstrates basic Ssh interactive (challenge-response) authentication method in blocking mode.
- SshLoggingExample: Demonstrates the logging capabilities of SSH package.
- SshOverTor:        Demonstrates a basic Ssh connection over TOR, using a third-party network proxy adapter.
- SshPolymorphismNB: Demonstrates the polymorphism capability of SSH channels in non-blocking mode.
- SshShell:          Demonstrates a basic, real-time SSH shell console in blocking mode.
- SshShellNB:        Demonstrates a basic, real-time SSH shell console in non-blocking mode.    
- SshShellX11:       Demonstrates a basic, real-time SSH shell console with multiple X11 forwarding.
- SshShellGUI:       Demonstrates the basic GUI integration of SshShell class with multiple X11 forwarding, in non-blocking mode.
- SshTunnelExample:  Demonstrates the basic SSH tunneling (as tunnel/server) in blocking mode.

#include <Core/Core.h>
#include <SSH/SSH.h>

using namespace Upp;

CONSOLE_APP_MAIN
{
    // This example demonstrates (in non-blocking mode):
    // 1) Reading the size of a file.
    // 2) Reading the content of a file (into a String).
    // 3) Reading the content of a directory as XML.

    int CMD_GET = 0, CMD_LIST = 0, CMD_SIZE = 0;
    const char *file = "/readme.txt";
    SFtp::DirList ls;

    Ssh::Trace();
    SshSession session;
    if(session.Timeout(30000).Connect("test.rebex.net", 22, "demo", "password")) {
        Array<SFtp> sftps;
        for(int i = 0; i < 3; i++) {
            auto& sftp = sftps.Add(new SFtp(session));
            sftp.NonBlocking();
            switch(i) {
                case 0: sftp.Get(file);  CMD_GET = sftp.GetId(); break;
                case 1: sftp.GetSize(file);  CMD_SIZE = sftp.GetId(); break;
                case 2: sftp.ListDir("/pub/example/", ls); CMD_LIST = sftp.GetId(); break;
                default: NEVER();
            }
        }
        while(!sftps.IsEmpty()) {
            for(int i = 0 ; i < sftps.GetCount(); i++) {
                SocketWaitEvent we;
                auto& sftp = sftps[i];
                sftp.AddTo(we);
                we.Wait(10);
                if(!sftp.Do()) {
                    if(sftp.IsError())
                        Cerr() << sftp.GetErrorDesc() << '\n';
                    else {
                        if(sftp.GetId() == CMD_GET) {
                            Cout() << sftp.GetResult();
                        }
                        else
                        if(sftp.GetId() == CMD_SIZE) {
                            Cout() << Format("Size of %s is %d bytes\n", file, sftp.GetResult());
                        }
                        else
                        if(sftp.GetId() == CMD_LIST) {
                            for(auto& e : ls)
                                Cout() << e.ToXml() << '\n';
                        }
                    }
                    sftps.Remove(i);
                    break;
                }
            }
        }
    }
    else
        Cerr() << session.GetErrorDesc();

}

- SFtpGet
- SFtpGetNB
- SFtpGetMT

- SshExec
- SshExecNB
- SshExecMT

- GetCurrentDir()
- GetParentDir()

SshKeyboardAuth: Demonstrates SSH keyboard interactive (challange/response) authentication method.

SFtpMultiGet:    Demonstrates download of multiple files simultaneously in SFTP non-blocking mode (non MT).

bool SshSession::Connect(const String& url)
--------------------------------------------
Connects to a SSH2 server specified by the url. Returns true on success. 
Syntax of the URL is as follows:  

[ssh|scp|sftp|exec]://[user:password@]host[:port]

#include <Core/Core.h>
#include <SSH/SSH.h>
#include <NetProxy/NetProxy.h>

using namespace Upp;

CONSOLE_APP_MAIN
{
	// This example requires a running TOR daemon.
	// Change below strings to your preferred values.
	
	const char* ssh_host = "dummysshhostname";
	const char* ssh_user = "dummysshusername";
	const char* ssh_pass = "dummysshpassword";
	int         ssh_port = 22;
	
	StdLogSetup(LOG_FILE|LOG_COUT);
	Ssh::Trace();
	NetProxy::Trace();
	
	SshSession session;
	session.WhenProxy = [=, &session] {
		return NetProxy(session.GetSocket(), "127.0.0.1", 9050)
		            .Timeout(30000)
		            .Socks5()
		            .Auth("none", "none")
		            .Connect(ssh_host, ssh_port);
	};
	
	if(session.Timeout(60000).Connect(ssh_host, ssh_port, ssh_user, ssh_pass)) {
		LOG("Successfully connected to " << ssh_host << " (over TOR)");
	}
	else
		LOG("Ssh connection via TOR failed. " << session.GetErrorDesc());
}

- SshSession: It is now possible to use encryption keys loaded into memory. (Load keys from mem.)

- SshSession: Host based authentication method is added.

- Ssh: TraceVerbose() method is added. This method allows full-level logging (redirection) of libsssh2 diagnostic messages.
   
- Documentation updated.

- SshOverTor:        Demonstrates a basic SSH connection over TOR (Requires NetProxy package and a TOR daemon)

- SshLoggingExample: Demostrates logging capabilities of SSH pakcage.

#include <Core/Core.h>
#include <SSH/SSH.h>

using namespace Upp;

// To activate verbose logging, set the LIBSSH2TRACE flag.
// (e.g. via TheIDE->main configuration settings)

CONSOLE_APP_MAIN
{
	StdLogSetup(LOG_COUT | LOG_FILE);
	
	// Ssh::Trace();
	
	Ssh::TraceVerbose(
//		LIBSSH2_TRACE_SOCKET    |
		LIBSSH2_TRACE_KEX       |
		LIBSSH2_TRACE_AUTH      |
		LIBSSH2_TRACE_CONN      |
//		LIBSSH2_TRACE_SCP       |
//		LIBSSH2_TRACE_SFTP      |
//		LIBSSH2_TRACE_PUBLICKEY |
		LIBSSH2_TRACE_ERROR
	);
	SshSession session;
	auto b = session.Timeout(30000).Connect("demo:password@test.rebex.net:22");
	LOG((b ? "Successfully connected to SSH2 server." : session.GetErrorDesc() << '\n'));
}

2018-01-19:  Alpha version 2.
             SshChannel reworked. It is now more flexible, and more analogous to a tcp socket. 

             Scp class gained a new Put method (and its corresponding operator overload).

             NEW: SShShell is added to the package. It allows GUI integration and  has a
                  "console mode"  that supports both POSIX consoles and Windows command prompt.

             NEW: SshTunnel class is added to the package. It allows TCP/IP and port forwarding
                  over SSH protocol.

             Various bug fixes, and improvements.  

- All SSH components have a very simple, easy to use, and uniform interface that supports time-constrained blocking, non-blocking operation modes and multithreading,

- They all work on Windows (tested on 7 & 10) and POSIX-compliant operating systems, and compile on both GCC/MingGW, and MSC.

- Shell component can work simultaneously with multiple X11 forwarding (per-shell), which is AFAIK a very rare  feature among the libssh2 wrappers out there. 

- And all this can be achieved writing very little code! (e.g. SshX11Shell has 10 LOCs for the actual code of X11-enabled full console, and SshShellGUI has 156 LOCs which are mostly usual U++ GUI setup)

- Blocking and non-blocking behaviour is now very similar to TcpSocket's.
- IsBlocking(), IsWorking() methods are added.
- WhenDo replaced with WhenWait, and WaitStep() method is added

  Remove progress gate parameters in the getters and putters, and replace them with a single WhenProgress gate.
- Add WhenContent:   Consumer function for incoming data transfers.

  Change the Async (multithreaded) getters and putters:
- They will have progress gates with three-parameteres (id, done, total), instead of two. This proved more useful.
- They will use a URL similar to the one I use in the new version of FTP package.
- Additional variants for async functions, which will use One<Stream> and picks input data (for outgoing transfers). 


- Update libssh2, as it has seen some activitiy, and gained new ciphers.

2018-04-15:  Consumer function support added to SFtp and SshChannel classes.
             GetWaitStep() method is added to Ssh class.
             Multithreaded methods rewritten.

#include <Core/Core.h>
#include <SSH/SSH.h>

using namespace Upp;

CONSOLE_APP_MAIN
{
	StdLogSetup(LOG_COUT|LOG_FILE);
//	Ssh::Trace();

	const char *file = "/pub/example/readme.txt";
	
	String data;

	SshSession session;
	if(session.Timeout(30000).Connect("demo:password@test.rebex.net:22")) {
		auto sftp = session.CreateSFtp();
		sftp.WhenContent = [&data](const void *buf, int len)
		{
			data.Cat(static_cast<const char*>(buf), len);
		};
		
		sftp.Get(file);
		LOG((!sftp.IsError() ? data : sftp.GetErrorDesc()));
	}
	else
		LOG(session.GetErrorDesc());
}

static AsyncWork<void> SFtp::AsyncConsumerGet(SshSession& session, const String& path, Event<int64, const void*, int> consumer)
static AsyncWork<void> Scp::AsyncConsumerGet(SshSession& session, const String& path, Event<int64, const void*, int> consumer)

- SFtpConsumerGetMT: Demonstrates the usage of a consumer function for SFtp download, using a worker thread.
- SshPolymorphismNB: Demonstrates the polymorphism capability of SSH channels in non-blocking mode.

2018-05-04:  Ssh::GetWaitEvents() fixed.
             SshTunnel::Validate() fixed.

SshTunnelExample:  Demonstrates the basic SSH tunneling (as tunnel/server) in blocking mode.

#include <Core/Core.h>
#include <SSH/SSH.h>

using namespace Upp;

// This example requires upp/reference/SocketServer and upp/reference/SocketClient examples.
// SocketClient: Set the port number to 3215.
//
// |SocketClient (client)|<---> |SshTunnelExample (tunnel/server)| <---> |SocketClient (server)|

bool SocketSendRecv(String& packet)
{
	TcpSocket s;
	if(!s.Connect("127.0.0.1", 3214)) {
		LOG("SocketSend(): " << s.GetErrorDesc());
		return false;
	}
	if(!s.PutAll(packet + '\n'))
		return false;
	packet = s.GetLine();
	return !packet.IsEmpty();
}
void StartTunnel(SshSession& session)
{
	SshTunnel listener(session);
	if(!listener.Listen(3215, 5)) {
		LOG("StartTunnel(): " << listener.GetErrorDesc());
		return;
	}
	LOG("SSH tunnel (server mode): Waiting for the requests to be tunneled...");
	for(;;) {
		SshTunnel tunnel(session);
		if(!tunnel.Accept(listener)) {
			LOG("StartTunnel(): " << tunnel.GetErrorDesc());
			return;
		}
		auto data = tunnel.GetLine();
		LOG("Tunneled Request: " << data);
		if(!data.IsEmpty() && SocketSendRecv(data)) {
			LOG("Tunneled Response: " << data);
			tunnel.Put(data + '\n');
		}
	}
}

CONSOLE_APP_MAIN
{
	StdLogSetup(LOG_FILE | LOG_COUT);
//	Ssh::Trace();

	SshSession session;
	if(session.Timeout(30000).Connect("username:password@localhost:22")) {
		StartTunnel(session.Timeout(Null));
		return;
	}
	LOG(session.GetErrorDesc());
}

2018-06-15:  Critical fix: SshChannel and SFtp data read and write methods fixed: The recently introduced
             socket wait mechanism was causing a constant I/O blocking. This is now fixed.   
             Cosmetics & cleanup.

2018-06-06:  SshShell: Console loop fixed.

1) Implementation document (giving an overview of the package and its implementation details).
2) MT server (listener) mode for SshTunnel.
3) Refactoring of Ssh (core) class.