|
|
Home » Developing U++ » UppHub » Coroutines package for U++
Coroutines package for U++ [message #59118] |
Sat, 05 November 2022 12:19 ![Go to next message Go to next message](theme/default/images/down.png) |
Oblivion
Messages: 1106 Registered: August 2007
|
Senior Contributor |
|
|
Hi,
I have added an experimental CoRoutines package to both UppHub and upp-components repo.
It provides a simple and basic interface for building coroutines.
Main github repo (for pull request etc.):
There are two types of CoRoutine in the package:
CoRoutine -> A regular coroutine that can return any value (or void)
CoGenerator -> A regular generator type function.
- The classes provided with the package expose "traditional" Upp-like interface (Do(), Get(), Pick() and Next() -methods, depending on their type.)
- Allows exception handling and propagation.
- Also an example code is included.
Example:
#include <Core/Core.h>
#include "CoRoutine.h"
using namespace Upp;
CoRoutine<Vector<int>> CoMakeIota(int begin, int end, int step)
{
Vector<int> v;
for(int i = begin; i < end; i += step) {
v.Add(i);
RLOG(__func__);
co_await CoSuspend();
}
co_return v;
}
CoGenerator<int> CoGenerateNumbers()
{
int i = 0;
for(;;) {
RLOG(__func__);
co_yield i++;
}
}
CONSOLE_APP_MAIN
{
StdLogSetup(LOG_COUT|LOG_FILE);
auto co1 = CoMakeIota(0, 10, 2);
auto co2 = CoGenerateNumbers();
try {
while(co1.Do())
RLOG(__func__); // NOP;
auto v = co1.Pick();
RDUMP(v);
for(int i = 0; i < 10; i = co2.Next())
RDUMP(i);
}
catch(...) {
RLOG("Unhandled exception");
}
}
Suggestion, pull requests, reviews, etc. are welcome.
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Sat, 05 November 2022 14:44] Report message to a moderator
|
|
|
|
|
Re: Coroutines package for U++ [message #59125 is a reply to message #59124] |
Sun, 06 November 2022 21:31 ![Go to previous message Go to previous message](theme/default/images/up.png) ![Go to next message Go to next message](theme/default/images/down.png) |
Oblivion
Messages: 1106 Registered: August 2007
|
Senior Contributor |
|
|
Hello Klugier,
Quote:Hello Oblvion,
Why you decieded to create separate package in concuret to Core? Is there Co* family functions like CoWork, AsyncWork, CoPartition and CoDo not enough for your needs?
On the contary, I use them daily. ![Smile](images/smiley_icons/icon_smile.gif)
However, It seems that there is a misunderstading here.
C++20 coroutines are not "concurrency" functions. They are not threads. They are suspendable functions, meaning that you can return from the function in the middle of its operation and you can resume it later. This makes async programming without threads impressively easy!
Excerpt:
A coroutine is a function that can suspend execution to be resumed later. Coroutines are stackless: they suspend execution by returning to the caller and the data that is required to resume execution is stored separately from the stack. This allows for sequential code that executes asynchronously (e.g. to handle non-blocking I/O without explicit callbacks), and also supports algorithms on lazy-computed infinite sequences and other uses.
While it is one of the coolest features of C++20, the original design is, well, let's say it isn't very elegant... (no wonder even seasoned developers are having a hard time understanding how to use them - watch the cppcon's ever-increasing videos about them. )
So, I haven't created a "yet another" thread-based concurrency tool, in the traditional sense. I have implemented and made available something completely new to U++. (hence the upload.)
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Sun, 06 November 2022 21:56] Report message to a moderator
|
|
|
|
|
Re: Coroutines package for U++ [message #59132 is a reply to message #59118] |
Mon, 07 November 2022 18:38 ![Go to previous message Go to previous message](theme/default/images/up.png) ![Go to next message Go to next message](theme/default/images/down.png) |
![](https://www.ultimatepp.org/forums/images/custom_avatars/34402.jpg) |
peterh
Messages: 108 Registered: November 2018 Location: Germany
|
Experienced Member |
|
|
Hi,
Out of interest I tried the example from UppHub.
(I have used cooperative coroutines 40 years ago on DOS in Borland Pascal (and in Modula II but the latter only experimental).
We used a commercial library. The program was rather big and consisted out of a base routine and overlays which where swapped.
We used this for a program that had a complex user interface and that must communicate with a running machine, store the process data in realtime, visualize data and control machine parameters interactively, all at the same time, using a single computer)
So far I have read, C++20 coroutines are different, these are "stackless" coroutines, which implies some limitations. (cannot suspend or yield in a subroutine)
I do not fully understand it yet and the purpose of it.
I noticed, if I set a breakpoint inside the body of a coroutine, then it is not hit.
Edit: A call to DebugBreak() is hit.
If I set a breakpoint at the entry point of a coroutine, it is hit.
If I call a subroutine from a coroutine and set a breakpoint inside this subroutine, then it is hit.
[Updated on: Mon, 07 November 2022 20:02] Report message to a moderator
|
|
|
Re: Coroutines package for U++ [message #59133 is a reply to message #59130] |
Mon, 07 November 2022 20:07 ![Go to previous message Go to previous message](theme/default/images/up.png) ![Go to next message Go to next message](theme/default/images/down.png) |
Oblivion
Messages: 1106 Registered: August 2007
|
Senior Contributor |
|
|
Hello koldo,
Quote:why does a programmer need to use coroutines?
Simple: To write non-blocking APIs or programs without spaghetti code: Recall the initial version of our very own SSH package. It was designed to be fully NB. This was achieved using a blend of an event queue, storing callbacks in a bivector (in a predefined order) and tracking their state. It did work, but the underlying code was very complex. (Fun fact: The same mechanism (but a stripped down version) is still used in NetProxy package.)
Now, the coroutines eliminates BOTH. You don't need to track the state of the internals of the NB object externally or worry about a state machine. Basically it is done for you by the compiler.
You write a local code block, and mark some points (usually where the code WOULDBLOCK) to suspend the coroutine and the compiler simply suspends it and returns the control back to its caller. The suspended coroutine can be resumed later from where it is left off.
This simplifites writing NB/async apps without using threads. Not to mention that you you don't need to worry about on how to terminate it. (CoRoutines are scope bound objects.)
Hello Peter:
Quote:I do not fully understand it yet and the purpose of it.
Well I don't think this debate (stackfull/stackless, i mean) will ever end --theoretically, at least. In practice, however, stackless coroutines are selected for C++20 (as in AFAIK C# and Python).
They can't be nested directly, yes.Some see this as a disadvantage, some don't. (I'm on the latter camp, since it forces simpler coroutines. (as no nesting is possible.)
As for the GDB: Yes it still can't handle coroutines properly..
A side note: I will add more complex examples to the package along with docs...
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Mon, 07 November 2022 20:27] Report message to a moderator
|
|
|
Re: Coroutines package for U++ [message #59134 is a reply to message #59133] |
Mon, 07 November 2022 20:30 ![Go to previous message Go to previous message](theme/default/images/up.png) ![Go to next message Go to next message](theme/default/images/down.png) |
![](https://www.ultimatepp.org/forums/images/custom_avatars/34402.jpg) |
peterh
Messages: 108 Registered: November 2018 Location: Germany
|
Experienced Member |
|
|
I understand it this way, C++20 coroutines are meant to write fast state machines in a simpler way.
In earlier times in plain old C longjump() could be (ab)used for this, but more complicated.
I have implemented sort of a coroutine in assembler for a device driver for an embedded device which had to handle XON/XOFF handshake and timeout properly without deadlock and I implemented the interrupt routine as a coroutine which was called by UART interrupt. This simplified the job a lot.
I do yet not understand, where does a C++ coroutine store its local variables?
On yield the stack is destroyed, so it must store the local variables elsewhere because these are persistent. (Local variables are still alive at the next invocation)
[Updated on: Mon, 07 November 2022 20:32] Report message to a moderator
|
|
|
|
|
|
Re: Coroutines package for U++ [message #59151 is a reply to message #59126] |
Wed, 09 November 2022 22:52 ![Go to previous message Go to previous message](theme/default/images/up.png) ![Go to next message Go to next message](theme/default/images/down.png) |
Oblivion
Messages: 1106 Registered: August 2007
|
Senior Contributor |
|
|
Hello,
As per Lance's request, I am adding some "non-trivial" coroutines examples to the package.
To demonstrate its usage, I have started with adapting the uppsrc/reference/GuiWebDownload example, so that it can be compared:
#include <CtrlLib/CtrlLib.h>
#include <CoRoutines/CoRoutines.h>
// GuiWebDownloader example, adapted to coroutines.
using namespace Upp;
CoRoutine<void> HttpDownload(const String& url)
{
Progress pi;
String path = AppendFileName(Nvl(GetDownloadFolder(), GetHomeDirFile("downloads")), GetFileName(url));
HttpRequest http(url);
http.Timeout(0);
int loaded = 0;
{
FileOut out(path);
http.WhenContent = [&out, &loaded](const void *ptr, int size) { out.Put(ptr, size); loaded += size; };
while(http.Do()) {
if(http.GetContentLength() >= 0) {
pi.SetText("Downloading " + GetFileName(url));
pi.Set((int)loaded, (int)http.GetContentLength());
}
else {
pi.Set(0, 0);
pi.SetText(http.GetPhaseName());
}
if(pi.Canceled())
http.Abort();
else
co_await CoSuspend();
}
}
if(http.IsFailure()) {
DeleteFile(path);
Exclamation("Download has failed.&\1" +
(http.IsError()
? http.GetErrorDesc()
: AsString(http.GetStatusCode()) + ' ' + http.GetReasonPhrase()));
}
}
GUI_APP_MAIN
{
StdLogSetup(LOG_COUT|LOG_FILE);
String url = "http://downloads.sourceforge.net/project/upp/upp/4179/upp-x11-src-4179.tar.gz";
for(;;) {
if(!EditText(url, "Download", "URL"))
return;
auto downloader = HttpDownload(url); // Multiple downloaders can be created/run at once.
try
{
while(downloader.Do())
Sleep(1); // Simulate work.
}
catch(const Exc& e)
{
}
catch(...)
{
}
}
}
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Wed, 09 November 2022 22:56] Report message to a moderator
|
|
|
|
|
|
|
|
Re: Coroutines package for U++ [message #59171 is a reply to message #59170] |
Mon, 14 November 2022 05:56 ![Go to previous message Go to previous message](theme/default/images/up.png) ![Go to next message Go to next message](theme/default/images/down.png) |
Oblivion
Messages: 1106 Registered: August 2007
|
Senior Contributor |
|
|
Hello Lance,
Quote:This article seems to be quite interesting:
C++20 Coroutines and io_uring
Yes, a really nice article that sums up what coroutines/awaitable is useful for. Thank you for this article!
This is the reason why I made my package a "light-weight" implementation of this nice cpp20 feature, and this is the reason why I haven't (yet) implemented a CoAwaiter (using awaitables).
IMHO, what the author achieves by using awaitables + threads, we already have it: Upp::CoDo & Upp::CoFor. And they work on also earlier versions of C++.
The main use of CoRoutines package is to simplify writing async/single threaded code and easily manageable generators.
Best regards,
Oblivion
Github page: https://github.com/ismail-yilmaz
upp-components: https://github.com/ismail-yilmaz/upp-components
Bobcat the terminal emulator: https://github.com/ismail-yilmaz/Bobcat
[Updated on: Mon, 14 November 2022 06:03] Report message to a moderator
|
|
|
|
Goto Forum:
Current Time: Sun Sep 01 07:42:05 CEST 2024
Total time taken to generate the page: 0.01976 seconds
|
|
|