Home » Developing U++ » UppHub » A terminal emulator widget for U++
Re: A terminal emulator widget for U++ [message #53731 is a reply to message #51415] |
Sat, 25 April 2020 00:36 |
Oblivion
Messages: 1112 Registered: August 2007
|
Senior Contributor |
|
|
Hello,
I have pushed the the critical performance fix that I mentioned in my last message.
The main culprit was using a Vector where a BiVector was a much better option. In fact, it seems to be the optimal solution to the scrollback/hsitory buffer problem. Removing elements from the beginning of a Vector, especially when its length > 64K is -not surprisingly- very slow. BiVector solves exactly this problem because a scrollback buffer is mutated only from the either end.
There was also a constant display refresh happening when a page was scrolled, even when a single line scrolled, degrading performance. This is fixed too.
Now Terminal ctrl's scroll speed / throughput can be very fast. (It is Even faster than the above given numbers: ~3.5 secoonds on the same test while acting as a both SSH (using Upp/SSH package) and local terminal. ("time find /usr/share" command was outputting > 300.000 file paths in these tests, with the size of scrollback buffer of each terminal was set to 64 k)
However, you can't see the real difference if you use the provided reference example codes, because they have a very rudimentary event loop.
If you are curious and you want to test the scrolling/throughput performance of Terminal ctrl, you can use the below code: While still rudimentary, it is closer to real world scenarios.
#include <Terminal/Terminal.h>
#include <Terminal/PtyProcess.h>
using namespace Upp;
const char *nixshell = "/bin/bash";
struct TerminalExample : TopWindow {
Terminal term;
PtyProcess pty;
TerminalExample()
{
SetRect(term.GetStdSize());
Sizeable().Zoomable().CenterScreen().Add(term.SizePos());
term.WhenBell = [=]() { BeepExclamation(); };
term.WhenTitle = [=](String s) { Title(s); };
term.WhenOutput = [=](String s) { pty.Write(s); };
term.WhenResize = [=]() { pty.SetSize(term.GetPageSize()); };
term.WhenLink = [=](const String& s) { PromptOK(s); };
term.InlineImages().Hyperlinks().WindowOps().DynamicColors();
term.SetHistorySize(65536);
}
void Run()
{
pty.Start(nixshell, Environment(), GetHomeDirectory());
OpenMain();
while(IsOpen() && pty.IsRunning()) {
int ms = 16;
String in = pty.Get();
if(!IsNull(in)) {
term.WriteUtf8(in);
int len = in.GetLength();
if(len >= 1024)
ms = 1024 * 16 / len; // Scale to workload...
}
ProcessEvents();
Sleep(ms);
}
}
};
GUI_APP_MAIN
{
TerminalExample().Run();
}
There are other improvements and bug fixes too (mostly on serialization/xmlization/jsonization fronts).
If you have any questions, criticism, bug reports, patches, ideas, let me know.
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Sat, 25 April 2020 01:06] Report message to a moderator
|
|
|
|
Re: A terminal emulator widget for U++ [message #53956 is a reply to message #51415] |
Sun, 17 May 2020 22:20 |
Oblivion
Messages: 1112 Registered: August 2007
|
Senior Contributor |
|
|
Hello,
Ranger, a terminal based file manager, can now show image previews via iterm2's inline image protocol, on terminal ctrl.
By the way, I am planning to implement optional, range-based font substitution for Terminal ctrl v0.4 (2020.2, synced with U++).
My initial plan is to give the client the ability to compose a font list (using a simple structure called Terminal::RangeFont), and pass it to terminal ctrl instances so the client will be responsible for taking care of the list. (Similar to how Upp::Display objects are handled.)
If you have any other suggestions I have opened a ticket for it. Feel free to share your ideas.
Best regards,
Oblivion.
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
|
|
|
|
|
Re: A terminal emulator widget for U++ [message #54061 is a reply to message #54059] |
Thu, 28 May 2020 11:36 |
Oblivion
Messages: 1112 Registered: August 2007
|
Senior Contributor |
|
|
Hello slashupp,
Quote:I want to test if Terminal fits the needs of my app, but fail to find
a HowTo on installing it in my upp-dev-environment (which is standard)
Yes, this is missing from the docs. I will update it next week. In the meantime,
How to get it:
Terminal is a regular ctrl. It does not require anything other than U++ (ver >= 2020.1, because I use some new color methods which simplified the color management code.)
If you want to try it out, you can:
a) Clone the upp-components git repository (This is the recommended way to get updates,etc.) and set up an assembly in TheIDE for upp-components/CtrlLib (It is no different than adding another "bazaar")
b) Simply download the repo, unzip it, and move the CtrlLib/Terminal folder to your preferred folder.
Now that you have the source code, It should compile out of the box. There should be no need for tweaking, etc.
How to use it:
Well, the upp-components/Examples section already contains example code for different, basic use cases, I suggest you check there and play with them first (if you haven't already).
They are pretty simple. A minimal example is here upp-components/Examples/TerminalExample
(That minimal example, by the way, can run almost anything, out of the box)
Steps to write your own code:
1) Check the API docs first. Terminal ctrl contains regular topic++ files for TheIDE. It should be available through TheIDE's help system.
2) Create a CtrlLib application (basic or with layout, it's up to you. Terminal ctrl contains a layout file, so you can use it in TheIDE's layout editor.)
3) Add Terminal ctrl to you app. As it is a regular ctrl, it can be added to your window or any other "parent" ctrl via Ctrl::Add() method.
4) If you are going to use it as a local terminal, using a pty (currently on POSIX only), then Terminal package contains a PtyProcess class.
Add #include <Terminal/PtyProcess.h> to your app. Then all you need to do is set up the events and loop.
Provided examples use a rudimentary loop which is sufficient for most cases but you can write much efficient loops that'll suit your personal need and use-case.
5) If you are going to use it as a GUI front end for, say, SSH shells, add uppsrc/Core/SSH package to you app, and connect the loop. See the rudimentary upp-components/Examples/SshTerminalExample
Again, this is just generic information on how to install and use it.
If you have any specific questions in the meanitme, I'll be happy to address them.
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Thu, 28 May 2020 11:57] Report message to a moderator
|
|
|
Re: A terminal emulator widget for U++ [message #54067 is a reply to message #54059] |
Fri, 29 May 2020 12:40 |
Oblivion
Messages: 1112 Registered: August 2007
|
Senior Contributor |
|
|
An example demonstrating the absolute minimum setup for a standalone terminal app, and two different loops:
#include <Terminal/Terminal.h>
#include <Terminal/PtyProcess.h>
using namespace Upp;
struct MinimalTerminalExample : TopWindow {
Terminal term; // A terminal ctrl instance.
PtyProcess pty; // A pseudoconsole process instance.
MinimalTerminalExample()
{
SetRect(0, 0, 800, 600);
Sizeable().Zoomable().CenterScreen().Add(term.SizePos());
term.WhenOutput = [=](String out) { pty.Write(out); }; // Terminal output
term.WhenResize = [=] { pty.SetSize(term.GetPageSize()); }; // Notifies the pty about page size changes.
pty.Start("/bin/bash", Environment(), GetHomeDirectory()); // Start a shell process on pty.
// For a reasonable throughput, a callback-based loop is sufficient.
SetTimeCallback(-1, [=] { term.WriteUtf8(pty.Get()); if(!pty.IsRunning()) Break(); });
}
// For very high throughput, below loop (or something better) can be used.
//
// void Run()
// {
// pty.Start("/bin/bash", Environment(), GetHomeDirectory());
// OpenMain();
// while(IsOpen() && pty.IsRunning()) {
// ProcessEvents();
// String s = pty.Get(); // PtyProcess reads are non-blocking.
// if(!IsNull(s))
// term.WriteUtf8(s);
// int l = s.GetLength();
// Sleep(l >= 1024 ? 1024 * 10 / l : 10); // Scale...
// }
// }
};
GUI_APP_MAIN
{
MinimalTerminalExample().Run();
}
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Fri, 29 May 2020 13:05] Report message to a moderator
|
|
|
Re: A terminal emulator widget for U++ [message #54194 is a reply to message #51415] |
Sat, 06 June 2020 13:57 |
Oblivion
Messages: 1112 Registered: August 2007
|
Senior Contributor |
|
|
Hello,
Terminal ctrl is updated. It has a number of under-the-hood improvements that lead to higher performance/throughput.
To give you an idea, here are some benchmark results I've been compiling (both synthetic and real-usage) for some time:
CPU: AMD FX 6100 - Black Edition (Six-core processor)
Memory: 8 GiB, DDR-3 1800 Mhz
Storage: 128 Gib SSD, 1Tib HDD
Graphics: AMD Radeon R7 240, 2GiB
Monitor: LCD, HD (1080p)
OS/Kernel: Arch/Linux 5.6 (x86_64)
Desktop: GNOME 3.36 (X11)
Framework: Ultimate++ 2020.1 (alpha)
Flags: GUI
Compilers: GCC 10.0.1 CLANG 10.0.0
Application: TerminalExample (with a high throughput loop).
Font: The same IBM Plex Mono, monospace font used in all tests.
Disclaimer: The benchmarks below do not represent any competition.
They don't represent any weak spots of the mentioned terminals. They are all high quality stuff.
The sole purpose of these benchmarks is to give an idea as to where the Terminal ctrl's "engine" performance stands ATM.
Note: All benchmarks are repeated 50 times, automatically.
The performance difference between GCC/CLANG builds is negligable.
The latest stable versions of the mentioned terminals are used in these tests (as of May-June 2020).
Results of command: "time find /usr", file count: 415897, page size: 237 x 55, Monospace Font)
Terminal Emulator real user sys Notes
----------------- ---------- ---------- ---------- ----------
Alacritty 0m3,776s 0m1,134s 0m2,637s Scrollback buffer size: 64k
Terminal ctrl 0m4,126s 0m1,202s 0m2,466s Scrollback buffer size: 64k
Kitty 0m5,608s 0m1,224s 0m2,763s Scrollback buffer size: 64k
Gnome Terminal 0m6,785s 0m1,243s 0m3,015s Scrollback buffer size: 64k
xterm 0m24,368s 0m1,661s 0m3,766s Scrollback buffer size: 64k
Terminal Emulator real user sys Notes
----------------- ---------- ---------- ---------- ----------
Simple terminal (0.8) 0m3,791s 0m1,162s 0m2,533s Has no scrollback buffer.
Terminal ctrl (0.3) 0m3,971s 0m1,120s 0m2,472s Scrollback disabled.
Terminal Emulator real user sys Notes
----------------- ---------- ---------- ---------- ----------
Terminal ctrl (0.3) 0m4,380s 0m1,479s 0m2,536s localhost, Scrollback buffer size: 2000 lines, no compression, (SshTerminalExample is used)
PuTTY (0.73) 0m40,828s 0m1,354s 0m3,071s localhost, Scrollback buffer size: 2000 lines, no compression
// Vte-bench, benchmark 1: scrolling. page size: 237 x 55, Monospace Font.
Terminal Emulator real user sys Notes
----------------- ---------- ---------- ---------- ----------
Alacritty 0m1,936s 0m0,001s 0m0,946s
Kitty 0m2,985s 0m0,000s 0m0,567s
Gnome Terminal 0m3,758s 0m0,001s 0m0,940s
Terminal ctrl 0m4,064s 0m0,000s 0m0,859s
Simple terminal (0.8) 0m4,623s 0m0,000s 0m1,105s
xterm 0m47,249s 0m0,000s 0m0,567s
// Vte-bench, benchmark 2: alt-screen-random-write, page size: 237 x 55, Monospace Font.
Terminal Emulator real user sys Notes
----------------- ---------- ---------- ---------- ----------
Alacritty 0m1,520s 0m0,000s 0m0,554s
Kitty 0m2,750s 0m0,000s 0m0,408s
Gnome Terminal 0m3,153s 0m0,004s 0m0,726s
Terminal ctrl 0m3,790s 0m0,000s 0m0,474s
Simple terminal (0.8) 0m7,097s 0m0,001s 0m0,825s
xterm 0m19,797s 0m0,001s 0m0,993s
// Vte-bench, benchmark 3: scrolling-in-region, page size: 237 x 55, Monospace Font.
Terminal Emulator real user sys Notes
----------------- ---------- ---------- ---------- ----------
Alacritty 0m3,337s 0m0,001s 0m3,297s
Gnome Terminal 0m5,280s 0m0,001s 0m3,254s
Kitty 0m6,326s 0m0,001s 0m6,034s
Terminal ctrl 0m6,739s 0m0,001s 0m4,033s
Simple terminal (0.8) 0m7,844s 0m0,001s 0m7,348s
xterm 3m36,599s 0m0,001s 0m11,011s
(Eventually I will add the final benchmark results to the Terminal package as a markdown file. Consider the above mentioned results only as a preview.)
If you have any questions, feature requests, bug reports, suggestions, criticism, etc. let me know.
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Sat, 06 June 2020 14:25] Report message to a moderator
|
|
|
Re: A terminal emulator widget for U++ [message #54207 is a reply to message #51415] |
Tue, 09 June 2020 23:48 |
Oblivion
Messages: 1112 Registered: August 2007
|
Senior Contributor |
|
|
Hello,
Terminal ctrl's TURTLE backend support was broken for some time.
The reason: Apparently there is no Upp::Append() method for image drag and drop (Or I am missing something...).
I've committed a patch to workaround this problem by disabling image drag-and-drops in Turtle backend.
A good news is that it is now possible to compile Terminal Ctrl for linux framebuffer, using the LinuxFrameBuffer package.
There are some rough edges (e.g. the mouse wheel does not work ATM), but it is basically working on linux framebuffer too.
However, currently the support for linux framebuffer is "unofficial". If you are going to try it at all, TRY IT AT YOUR OWN RISK.
I will focus on these backends after the V0.4 release...
Best regards,
Oblivion.
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Tue, 09 June 2020 23:50] Report message to a moderator
|
|
|
|
Re: A terminal emulator widget for U++ [message #54214 is a reply to message #54210] |
Thu, 11 June 2020 11:08 |
Oblivion
Messages: 1112 Registered: August 2007
|
Senior Contributor |
|
|
Hello Zbych,
Thank you for your efforts!
Indeed, it is working now.
Since you are here, allow me to point out two problems I've encountered while testing both terminal package and others (uword, for example).
- Text mode is not disabled on linuxframebuffer based apps when they are run from the linux (tested with v5.6) console. (The framebuffer display is overwritten by text and messages). The "fix" seems to be setting the STDINPUT to KD_GRAPHICS and resetting it to KD_TEXT before the app exits (using ioctl). But since I don't know the specifics of LinuxFrameBuffer package, I can't give more info as to where the problem lies.
- Both on UWord and Terminal package, the return/enter keys are sending the keycode twice (or so it appears: text cursor is not moved to next line but two lines below.)
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Thu, 11 June 2020 11:08] Report message to a moderator
|
|
|
|
|
|
Re: A terminal emulator widget for U++ [message #54293 is a reply to message #54292] |
Sun, 21 June 2020 22:36 |
Oblivion
Messages: 1112 Registered: August 2007
|
Senior Contributor |
|
|
Hello Zbych,
Quote:
I did some research and it appears that turning keyboard off in virtual terminal is correct solution.
Unfortunately neither ioctl(tty_handle, KDSKBMUTE, 1) nor ioctl(tty_handle, KDSKBMODE, K_RAW) solves the problem with keys leaking to the console.
They simply return error "Operation not permitted".
Again, thank you very much for your efforts!
Also thanks for the links.
I am going to look into LinuxFrameBuffer code in the following days and if I can come up with a workaround or solution, I'll send a patch to review.
By the way, did you try to set STDIN_FILENO to KD_GRAPHICS too? because IIRC, that (Setting both input and output to graphics mode) was the trick used by an app whose name I forgot now, some time ago.)
I'd really love to see Terminal ctrl run on linux frame buffer, because then I can get rid of unnecessary cruft on my personal servers.
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
|
|
|
Re: A terminal emulator widget for U++ [message #54294 is a reply to message #54293] |
Sun, 21 June 2020 22:47 |
Zbych
Messages: 327 Registered: July 2009
|
Senior Member |
|
|
Oblivion wrote on Sun, 21 June 2020 22:36
By the way, did you try to set STDIN_FILENO to KD_GRAPHICS too?
Do you mean using STDIN_FILENO in ioctl(STDIN_FILENO, KDSETMODE, KD_GRAPHICS)?
it returns error: Inappropriate ioctl for device (25)
In my tests I pass path to virtual terminal manually (for example /dev/tty3), I decided to leave tty number autodetection as next step.
[Updated on: Sun, 21 June 2020 22:55] Report message to a moderator
|
|
|
|
|
|
Re: A terminal emulator widget for U++ [message #54345 is a reply to message #54338] |
Sun, 28 June 2020 11:57 |
Oblivion
Messages: 1112 Registered: August 2007
|
Senior Contributor |
|
|
Hello Zbych,
I have attached the patched LinuxFrameBuffer package for you to review. Feel free to comment on it.
Package is compiled with the latest stable GCC and CLANG on Linux v5.4 and v5.6 machines.
For testing, I used the supplied LinuxFrameBufferExample (UWord) code, Terminal ctrl, and some other U++ examples made to run on FB backend.
Changes/additions:
- Automatic VT allocation mechanism is implemented.
Since U++ suppports function keys from F1 to F12, I have limited the valid VT numbers within tty1-tty12 range (This can be easily changed if required).
- Accordingly, VT switching is implemented.
Since we mute the vt keyboard input, this required manual handling of the linux signals, and as a result the vt handle had to be made a global static integeer (atomic).
- To prevent any screen buffer damage, U++ linuxframebuffer backend now does not copy the drawn image onto framebuffer when the
vt is switched away (i,e., whem it loses focus).
- Accordingly, CTRL + ALT + [F1-f12] key combinations are reserved for VT switching.
Remaining issues:
- SHIFT keys are not working properly. (Can't get any capital letters.)
- No double click.
- No key repeat.
I am not sure about the security implications though. If you find any problem, let me know.
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Sun, 28 June 2020 12:04] Report message to a moderator
|
|
|
Goto Forum:
Current Time: Fri Sep 20 05:38:03 CEST 2024
Total time taken to generate the page: 0.03203 seconds
|