Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » U++ Library support » U++ MT-multithreading and servers » SSH package for U++ (A feature-rich ilbssh2 wrapper for Ultimate++)
Re: SSH package for U++ [message #49108 is a reply to message #48973] Tue, 19 December 2017 14:44 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello,

SSH package (both core classes and examples) is updated. As usual, you can find the latest package in the first message of this topic, or via below git address:

SSH: https://github.com/ismail-yilmaz/upp-components/tree/master/ Core/SSH
Examples: https://github.com/ismail-yilmaz/upp-components/tree/master/ Examples

SSH package gained new features:

- SshSession: It is now possible to use encryption keys loaded into memory. (Load keys from mem.)

- SshSession: Host based authentication method is added.

- Ssh: TraceVerbose() method is added. This method allows full-level logging (redirection) of libsssh2 diagnostic messages.
   
- Documentation updated.


Examples are updated too. Two new reference example is added to the package:

- SshOverTor:        Demonstrates a basic SSH connection over TOR (Requires NetProxy package and a TOR daemon)

- SshLoggingExample: Demostrates logging capabilities of SSH pakcage.



SshLoggingExample demonstrates the powerful logging mechanism of SSH package:

#include <Core/Core.h>
#include <SSH/SSH.h>

using namespace Upp;

// To activate verbose logging, set the LIBSSH2TRACE flag.
// (e.g. via TheIDE->main configuration settings)

CONSOLE_APP_MAIN
{
	StdLogSetup(LOG_COUT | LOG_FILE);
	
	// Ssh::Trace();
	
	Ssh::TraceVerbose(
//		LIBSSH2_TRACE_SOCKET    |
		LIBSSH2_TRACE_KEX       |
		LIBSSH2_TRACE_AUTH      |
		LIBSSH2_TRACE_CONN      |
//		LIBSSH2_TRACE_SCP       |
//		LIBSSH2_TRACE_SFTP      |
//		LIBSSH2_TRACE_PUBLICKEY |
		LIBSSH2_TRACE_ERROR
	);
	SshSession session;
	auto b = session.Timeout(30000).Connect("demo:password@test.rebex.net:22");
	LOG((b ? "Successfully connected to SSH2 server." : session.GetErrorDesc() << '\n'));
}





Reviews, patches, bug fixes, criticism, and suggestions are always appreciated.

Best regards,

Oblivion



Re: SSH package for U++ [message #49211 is a reply to message #48973] Sun, 07 January 2018 22:09 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello,

In the following days, SSH package will receive a huge update. There are many bug fixes and improvements (hopefully with performance gain) here and there, but the most significant changes are made to SshChannel, which is now a full-fledge, and covering SSH channel class.

In the meantime I am putting here a short video of the one of the new members of ssh channel related classes: SshShell Smile

Update will contain SShShell, SShTunnel, and SshX11Tunnel classes. (Video link is down below the image)


index.php?t=getfile&id=5479&private=0



Below video demonstrates a real-time remote shell (simulated on a local test machine).

https://vimeo.com/250031042


Best regards,
Oblivion


Re: SSH package for U++ [message #49212 is a reply to message #49211] Mon, 08 January 2018 08:27 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3354
Registered: August 2008
Senior Veteran
It looks great.

I'm going to use it again soon, and I'm looking forward to it!


Best regards
Iñaki
Re: SSH package for U++ [message #49233 is a reply to message #48973] Tue, 09 January 2018 23:15 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello Koldo,

I hope you'll find it useful. Smile



Below screenshot (and short video) demonstrates the real-time non-blocking ssh shell with GUI integration (for the exapmle I used CodeEditor). (supports both Windows and *NIX)
"It just works". Smile


Video:
https://vimeo.com/250352882

Screenshot (obviously):

index.php?t=getfile&id=5481&private=0


Best regards,
Oblivion


[Updated on: Tue, 09 January 2018 23:18]

Report message to a moderator

Re: SSH package for U++ [message #49236 is a reply to message #49233] Wed, 10 January 2018 08:53 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3354
Registered: August 2008
Senior Veteran
Very didactic!

Best regards
Iñaki
Re: SSH package for U++ [message #49302 is a reply to message #48973] Fri, 19 January 2018 14:18 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello everyone,

SSH package is updated. This is a major update.

2018-01-19:  Alpha version 2.
             SshChannel reworked. It is now more flexible, and more analogous to a tcp socket. 

             Scp class gained a new Put method (and its corresponding operator overload).

             NEW: SShShell is added to the package. It allows GUI integration and  has a
                  "console mode"  that supports both POSIX consoles and Windows command prompt.

             NEW: SshTunnel class is added to the package. It allows TCP/IP and port forwarding
                  over SSH protocol.

             Various bug fixes, and improvements.  



Finally the code for SSH shell (console/interactive command line interface) has landed. I managed to get it work on Windows command prompt too (ver >= XP). Cool

Examples are updated too. SshShell, SshShellNB, and SshShellGUI examples are added to the package.

SShShellGUI demonstrates a very basic SSH terminal with a non-blocking GUI.


You can always find the updated package on the first message of this topic or you can grab the code from:

https://github.com/ismail-yilmaz/upp-components/tree/master/ Core/SSH

Short videos:

SSH Shell (Demonstrating console apps, such as nano, and top): https://vimeo.com/250031042
SSH Shell GUI (demonstrates a very basic SSH terminal with a non-blocking GUI): https://vimeo.com/250352882

I appreciate bug reports, reviews, criticism, patches etc.

Best regards,
Oblivion


Re: SSH package for U++ [message #49304 is a reply to message #48973] Sun, 21 January 2018 12:18 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello,

X11 forwarding is the last missing part of the SSH package, and it is shaping up good.

It is based on SshShell class and supports both console and GUI modes. There is even a good chance that it'll work on Windows, since the necessary foundation is alreay laid.

I'm looking into the possibility of compile-time (optional) integration with XMing or Cygwin/X.

Below short video can give you the idea. Smile

https://vimeo.com/252025261


Best regards,
Oblivion


[Updated on: Sun, 21 January 2018 12:19]

Report message to a moderator

Re: SSH package for U++ [message #49306 is a reply to message #49304] Sun, 21 January 2018 16:53 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3354
Registered: August 2008
Senior Veteran
Shocked Amazing!

Best regards
Iñaki
Re: SSH package for U++ [message #49334 is a reply to message #48973] Sun, 28 January 2018 12:13 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello all,

X11 forwarding support (the final missing piece of SSH package) code has finally landed.

X11 support is added as an operation mode for SshShell.

I believe that we have achieved a really remarkable result here (though this is a work-in-progress), given that:

- All SSH components have a very simple, easy to use, and uniform interface that supports time-constrained blocking, non-blocking operation modes and multithreading,

- They all work on Windows (tested on 7 & 10) and POSIX-compliant operating systems, and compile on both GCC/MingGW, and MSC.

- Shell component can work simultaneously with multiple X11 forwarding (per-shell), which is AFAIK a very rare  feature among the libssh2 wrappers out there. 

- And all this can be achieved writing very little code! (e.g. SshX11Shell has 10 LOCs for the actual code of X11-enabled full console, and SshShellGUI has 156 LOCs which are mostly usual U++ GUI setup)



Examples directory contains SshX11Shell and re-written SshShellGUI example with X11 and multiple shell support.

Here is a screenshot:

index.php?t=getfile&id=5493&private=0

As usual, you can find the code and examples in the first message of this topic,
or you can grab them from: https://github.com/ismail-yilmaz/upp-components/tree/master/ Core/SSH

I appreciate bug reports, reviews, criticism, patches etc.

Best regards,
Oblivion


[Updated on: Sun, 28 January 2018 12:43]

Report message to a moderator

Re: SSH package for U++ [message #49683 is a reply to message #48973] Mon, 02 April 2018 23:27 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello,

The new FTP package contains a simple FTP GUI example called FtpGUI. And now, here is its SFTP counterpart.
It has the same functionality. More importantly, their code are almost identical. Cool

You can compare them from the below links:

FtpGUI:
https://github.com/ismail-yilmaz/upp-components/tree/master/ Examples/FtpGUI


SFtpGUI:

https://github.com/ismail-yilmaz/upp-components/tree/master/ Examples/SFtpGUI

Screenshot:

index.php?t=getfile&id=5540&private=0

I'm considering writing multithreaded and multiprotocol (Local, FTP/SFTP/HTTP) global file get and put functions (similar to a lightweight GIO/KIO) for U++, based on AsyncWork, and using a url scheme.
If you have any thougts about this, I'd like to hear it.

Best regards,
Oblivion
  • Attachment: SFtpGUI.png
    (Size: 49.24KB, Downloaded 805 times)


[Updated on: Mon, 02 April 2018 23:38]

Report message to a moderator

Re: SSH package for U++ [message #49703 is a reply to message #48973] Sat, 07 April 2018 16:25 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello all,

SSH package and examples are updated.
There are some small improvements and a small change:


- Blocking and non-blocking behaviour is now very similar to TcpSocket's.
- IsBlocking(), IsWorking() methods are added.
- WhenDo replaced with WhenWait, and WaitStep() method is added


Upcoming version will break the interface slighlty, I'm afraid. This will be a final breakage, as I am also going to declare the package a beta stage library.
Almost all of the issues are resolved, and last bits are going to be resovelved -hopefully- with the next version.
SSH package is currently very stable though.

Yet, as with the Ftp, I will also move this version of the SSH into the Attic foder, and maintain it for some time.

I am going to make the following changes:

  Remove progress gate parameters in the getters and putters, and replace them with a single WhenProgress gate.
- Add WhenContent:   Consumer function for incoming data transfers.

  Change the Async (multithreaded) getters and putters:
- They will have progress gates with three-parameteres (id, done, total), instead of two. This proved more useful.
- They will use a URL similar to the one I use in the new version of FTP package.
- Additional variants for async functions, which will use One<Stream> and picks input data (for outgoing transfers). 


- Update libssh2, as it has seen some activitiy, and gained new ciphers.



GIT repo: https://github.com/ismail-yilmaz/upp-components/tree/master/ Core/SSH
Examples: https://github.com/ismail-yilmaz/upp-components/tree/master/ Examples

Best regards,
Oblivion.


[Updated on: Sun, 08 April 2018 11:21]

Report message to a moderator

Re: SSH package for U++ [message #49736 is a reply to message #48973] Sun, 15 April 2018 00:39 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello,

SSH package updated. It has finally reached beta version.


2018-04-15:  Consumer function support added to SFtp and SshChannel classes.
             GetWaitStep() method is added to Ssh class.
             Multithreaded methods rewritten.


It is now possible to use consumer functions. A reference example demonstrating this behavious is added, accordingly.
Also, multithreaded functions are rewritten. They now use a three-parameter progress gate.

Consumer function example (SFtpConsumerGet):
#include <Core/Core.h>
#include <SSH/SSH.h>

using namespace Upp;

CONSOLE_APP_MAIN
{
	StdLogSetup(LOG_COUT|LOG_FILE);
//	Ssh::Trace();

	const char *file = "/pub/example/readme.txt";
	
	String data;

	SshSession session;
	if(session.Timeout(30000).Connect("demo:password@test.rebex.net:22")) {
		auto sftp = session.CreateSFtp();
		sftp.WhenContent = [&data](const void *buf, int len)
		{
			data.Cat(static_cast<const char*>(buf), len);
		};
		
		sftp.Get(file);
		LOG((!sftp.IsError() ? data : sftp.GetErrorDesc()));
	}
	else
		LOG(session.GetErrorDesc());
}



Below you can find the latest package and the GIT address where you can always get the latest version.

GIT repo: https://github.com/ismail-yilmaz/upp-components/tree/master/ Core/SSH
Examples: https://github.com/ismail-yilmaz/upp-components/tree/master/ Examples
Older Version: https://github.com/ismail-yilmaz/upp-components/tree/master/ Attic/SSH

Please feel free to comment on it. Bug reports, reviews, criticism, etc. are appreciated.

Best regards,
Oblivion


[Updated on: Sun, 15 April 2018 00:42]

Report message to a moderator

Re: SSH package for U++ [message #49750 is a reply to message #48973] Sat, 21 April 2018 08:12 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello,

SSH package and its reference examples are updated.

Two new multithreaded transfer methods are added to the package:

static AsyncWork<void> SFtp::AsyncConsumerGet(SshSession& session, const String& path, Event<int64, const void*, int> consumer)
static AsyncWork<void> Scp::AsyncConsumerGet(SshSession& session, const String& path, Event<int64, const void*, int> consumer)


These asynchronous methods use a consumer function to download data.

Also, two new reference examples are added to the package:

- SFtpConsumerGetMT: Demonstrates the usage of a consumer function for SFtp download, using a worker thread.
- SshPolymorphismNB: Demonstrates the polymorphism capability of SSH channels in non-blocking mode.



You can always get the latest version from the below git adresses.

GIT repo: https://github.com/ismail-yilmaz/upp-components/tree/master/ Core/SSH
Examples: https://github.com/ismail-yilmaz/upp-components/tree/master/ Examples
Older Version: https://github.com/ismail-yilmaz/upp-components/tree/master/ Attic/SSH


Best regards,
Oblivion


[Updated on: Sat, 21 April 2018 08:23]

Report message to a moderator

Re: SSH package for U++ [message #49784 is a reply to message #48973] Fri, 04 May 2018 22:08 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello

SSH package is updated.

SSH:
2018-05-04:  Ssh::GetWaitEvents() fixed.
             SshTunnel::Validate() fixed.


Reference examples added to the package:

SshTunnelExample:  Demonstrates the basic SSH tunneling (as tunnel/server) in blocking mode.



Note that the SSH tunnel example requires upp/reference/SocketClient and upp/reference/SocketServer examples.
it acts as a SSH server between the socket client and server. Although the example demonstrates one of the basic tunneling capabilities of the SshTunnel class,
very complex SSH tunnels can be built using it.

Code:
#include <Core/Core.h>
#include <SSH/SSH.h>

using namespace Upp;

// This example requires upp/reference/SocketServer and upp/reference/SocketClient examples.
// SocketClient: Set the port number to 3215.
//
// |SocketClient (client)|<---> |SshTunnelExample (tunnel/server)| <---> |SocketClient (server)|

bool SocketSendRecv(String& packet)
{
	TcpSocket s;
	if(!s.Connect("127.0.0.1", 3214)) {
		LOG("SocketSend(): " << s.GetErrorDesc());
		return false;
	}
	if(!s.PutAll(packet + '\n'))
		return false;
	packet = s.GetLine();
	return !packet.IsEmpty();
}
void StartTunnel(SshSession& session)
{
	SshTunnel listener(session);
	if(!listener.Listen(3215, 5)) {
		LOG("StartTunnel(): " << listener.GetErrorDesc());
		return;
	}
	LOG("SSH tunnel (server mode): Waiting for the requests to be tunneled...");
	for(;;) {
		SshTunnel tunnel(session);
		if(!tunnel.Accept(listener)) {
			LOG("StartTunnel(): " << tunnel.GetErrorDesc());
			return;
		}
		auto data = tunnel.GetLine();
		LOG("Tunneled Request: " << data);
		if(!data.IsEmpty() && SocketSendRecv(data)) {
			LOG("Tunneled Response: " << data);
			tunnel.Put(data + '\n');
		}
	}
}

CONSOLE_APP_MAIN
{
	StdLogSetup(LOG_FILE | LOG_COUT);
//	Ssh::Trace();

	SshSession session;
	if(session.Timeout(30000).Connect("username:password@localhost:22")) {
		StartTunnel(session.Timeout(Null));
		return;
	}
	LOG(session.GetErrorDesc());
}


Below is the GIT address where you can always get the latest version.

GIT repo: https://github.com/ismail-yilmaz/upp-components/tree/master/ Core/SSH
Examples: https://github.com/ismail-yilmaz/upp-components/tree/master/ Examples

Best regards,
Oblivion


[Updated on: Fri, 04 May 2018 22:44]

Report message to a moderator

Re: SSH package for U++ [message #50008 is a reply to message #48973] Sat, 16 June 2018 19:09 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello,

SSH package and its examples are updated.
I highly recommentd updating the package, since it contains a critical fix.

2018-06-15:  Critical fix: SshChannel and SFtp data read and write methods fixed: The recently introduced
             socket wait mechanism was causing a constant I/O blocking. This is now fixed.   
             Cosmetics & cleanup.

2018-06-06:  SshShell: Console loop fixed.



You can find the updated packages in the first message of this topic, or at the below git address:

SSH:
https://github.com/ismail-yilmaz/upp-components/tree/master/ Core/SSH

Examples:
https://github.com/ismail-yilmaz/upp-components/tree/master/ Examples

Best regards,
Oblivion


Re: SSH package for U++ [message #50065 is a reply to message #50008] Tue, 10 July 2018 14:37 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
https://www.ultimatepp.org/forums/index.php?t=msg&goto=5 0064&#msg_50064

libeay32MTd ssleay32MTd are now unavailable for U++ setup....

(you have these configured in SSH).
Re: SSH package for U++ [message #50066 is a reply to message #50065] Tue, 10 July 2018 15:38 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Also, it looks like with new SSL, you need to add crypt32.lib for win32...

(Sending ssh.upp that seems to work now).

  • Attachment: SSH.upp
    (Size: 0.79KB, Downloaded 208 times)
Re: SSH package for U++ [message #50067 is a reply to message #50066] Tue, 10 July 2018 15:42 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
SSH looks pretty reasonable. Can I move to Core/SSH? Smile

That said, I find myself sort of cofusing the order of arguments sometimes:

Put(Stream& in, const String& path);

I would expect path to be the first argument here... Do you insist on this order? Smile

Mirek

[Updated on: Tue, 10 July 2018 15:49]

Report message to a moderator

Re: SSH package for U++ [message #50070 is a reply to message #50067] Wed, 11 July 2018 08:35 Go to previous messageGo to next message
Oblivion is currently offline  Oblivion
Messages: 1091
Registered: August 2007
Senior Contributor
Hello Mirek,

Quote:

SSH looks pretty reasonable. Can I move to Core/SSH?


Of course! Smile

Quote:
Put(Stream& in, const String& path);

I would expect path to be the first argument here... Do you insist on this order?


That's because I use "[source], [destination]..." order. I'd prefer it stay that way but I don't insist on it. It can be changed.

What's left (none of them are critical, and can be added later via patches):
1) Implementation document (giving an overview of the package and its implementation details).
2) MT server (listener) mode for SshTunnel.
3) Refactoring of Ssh (core) class.


I am going to test the package with the latest build of the U++ on Windows, tonight.

Best regards,
Oblivion






Re: SSH package for U++ [message #50119 is a reply to message #50070] Tue, 31 July 2018 11:07 Go to previous messageGo to previous message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Oblivion wrote on Wed, 11 July 2018 08:35
Hello Mirek,

Quote:

SSH looks pretty reasonable. Can I move to Core/SSH?


Of course! Smile

Quote:
Put(Stream& in, const String& path);

I would expect path to be the first argument here... Do you insist on this order?


That's because I use "[source], [destination]..." order. I'd prefer it stay that way but I don't insist on it. It can be changed.


Whether the "source, target" or "target, source" is more natural is certainly debatable, however please notice that since the dawn of computer languages the custom is to write target = source (also see strcpy etc... Smile That said, where handle or path is involved, I would probably like to have it as first parameter always.

Besides, it does not seem to be consistent anyway:
bool Get(SFtpHandle* handle, Stream& out);
bool Put(SFtpHandle* handle, Stream& in);

Now I am only scrathich the surface, mostly being interested in SFTP for now. Anyway, seeing that interface, I would probably like to change following things:

SFtpHandle* -> SFtpHandle - if something is "HANDLE", it should not be a pointer to handle.

Instead of:

auto sftp = session.CreateSFtp();

I would like to have

SFtp sftp(session);

Besides, I know that there is some movement to use 'auto' everywhere, but I strongly disagree with that, ESPECIALLY in reference examples.

More important: I thing I would remove whole bunch of Gets and Puts, leave only handle variant in sftp and then add SFtpStream instead. With methods provided in SFtp, it should be easy to do, and it would mostly remove "source/destination controversy".

Interestingly, it looks like the most important Gets / Puts are missing there

int Get(SFtpHandle h, void *ptr, int size);
bool Put(SFtpHandle h, const void *ptr, int size);

Anyway, it is now moved to Core/SSH and single example is now in reference - that is another issue I have not solved for now, there is a lot of examples and they really make sense, so maybe I will put them gradually to SSH subfolder.

I am giving you write access to Core/SSH and reference.

Mirek
Previous Topic: WebSockets client in javascript connected to an U++ server sending binary messages
Next Topic: TURTLE high cpu usage, potential security flaw, and client handling problem
Goto Forum:
  


Current Time: Thu Mar 28 12:57:27 CET 2024

Total time taken to generate the page: 0.01545 seconds