Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Developing U++ » U++ Developers corner » Ideas on U++ as library
Ideas on U++ as library [message #45645] Fri, 18 December 2015 14:40 Go to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

Moving this discussion here, so I don't hijack original discussion about PR.

mirek wrote on Fri, 18 December 2015 11:01
dolik.rce wrote on Thu, 17 December 2015 21:08

I remember some experiments with converting upp files to Visual Studio projects, waf based builds and of course the universal makefile, but frankly, all of those are quite difficult to use without previous knowledge of U++. Making a library that could be just linked against would help a lot, but I know this has already been tried several times before, with mixed results.


Well, I am afraid this is still mostly the question of INITBLOCK.

Should we drop INITBLOCK and require manual registration of modules used?

Mirek


Do you mean that INITBLOCKs cause trouble with building icpp files or that they are not called in library? I believe the first issue is more or less solved, so I'll assume the second is what you meant. Correct me if I'm mistaken.

I haven't really think any of this through, but here are some ideas:

1) Some script during library build goes through all code and gathers INITBLOCKs, then generates code that executes them in init function/DllMain. It might require some changes to how INITBLOCK macro is defined when building as library, but that should be simple.

2) Ignore INITBLOCKs when building library and allow manual registration of everything the user needs. This would require good documentation of what can registered When using static linking, everything could still work as of now.

3) Putting all INITBLOCKs content into separate header files. It would then be users responsibility to include necessary headers exactly once into an application. This would make the manual initialization quite simple (one include per package) and it would also simplify compilation - you'd just need to special handle one file that includes all the initialization headers, instead of all icpp files.

Any other ideas are welcome Smile If I have some spare time during holidays, I'll try to experiment with this a bit.

Having a dynamic library would lower the barrier for newbies significantly. I believe many leave U++ before trying it because a) it seems to be slow, when building everything for the first time or b) it requires static linking.

Honza
Re: Ideas on U++ as library [message #45655 is a reply to message #45645] Sat, 19 December 2015 08:55 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13996
Registered: November 2005
Ultimate Member
Well, there is one equally pressing issue:

What should be in the library?
Re: Ideas on U++ as library [message #45656 is a reply to message #45655] Sat, 19 December 2015 13:26 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

mirek wrote on Sat, 19 December 2015 08:55
Well, there is one equally pressing issue:

What should be in the library?

Well, I was thinking along the lines of one library per package. For simple experiments, linking to "libUppCore" would be enough, for GUI, you'd have to link "libUppCtrlLib". The dependencies between the libs would follow the same pattern as U++ packages, that would make the most sense to me.

It doesn't really feel natural to take away all the modularity of U++ by compiling it all into single huge library, so I'd try to avoid that if possible. In any case, I think "libuppcore" would be a good place to start experimenting. We could figure out the rest when we get there Smile

Honza
Re: Ideas on U++ as library [message #45667 is a reply to message #45656] Sun, 20 December 2015 11:40 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13996
Registered: November 2005
Ultimate Member
Well, I suggest starting with allocator....

It is the least controversial thing and relatively simple to 'extract'.

Technically, there is one problem though to be resolved: When thread exits, heap cleanup function has to be called.

That was easy to do in U++, where we have our own Thread class, but will need to be resolved for standalone allocator.

It looks like pthread_key_create resolves issue in Posix (maybe...), but I do not see equivalent in Win32 Sad Worst scenario - it has to be documented.... Smile

Other than that, the endproduct that I would like to produce here is single 'amalgamated' .c file (like sqlite3). You include it in .cpp or add to project and you get your application running faster, using less memory. Or include in .c and get 'super_malloc', 'super_free' functions. With something #define JUST_DECLARATIONS, you can also include it in header.

So the task would be mostly about

- convert current code in C++ to be .c compilable. Tedious, but possible
- in the process, add more comments, including algorithm overview
- create 'amalgamating export' utility that will take current Core and export amalgamated allocator code as single .c file (or maybe some other extension)
- create website section for it, publish Smile

Mirek
Re: Ideas on U++ as library [message #45668 is a reply to message #45667] Sun, 20 December 2015 17:43 Go to previous messageGo to next message
cbpporter is currently offline  cbpporter
Messages: 1406
Registered: September 2007
Ultimate Contributor
mirek wrote on Sun, 20 December 2015 12:40
Well, I suggest starting with allocator....

It is the least controversial thing and relatively simple to 'extract'.

Technically, there is one problem though to be resolved: When thread exits, heap cleanup function has to be called.

That was easy to do in U++, where we have our own Thread class, but will need to be resolved for standalone allocator.

It looks like pthread_key_create resolves issue in Posix (maybe...), but I do not see equivalent in Win32 Sad Worst scenario - it has to be documented.... Smile

Other than that, the endproduct that I would like to produce here is single 'amalgamated' .c file (like sqlite3). You include it in .cpp or add to project and you get your application running faster, using less memory. Or include in .c and get 'super_malloc', 'super_free' functions. With something #define JUST_DECLARATIONS, you can also include it in header.

So the task would be mostly about

- convert current code in C++ to be .c compilable. Tedious, but possible
- in the process, add more comments, including algorithm overview
- create 'amalgamating export' utility that will take current Core and export amalgamated allocator code as single .c file (or maybe some other extension)
- create website section for it, publish Smile

Mirek

Maybe it should be clearly set in stone what we are trying to achieve with this. There where multiple reasons proposed in the larger thread for making libs. One goal was to cut out small useful parts of U++ and offer them as libs to people who don't have this functionality. If this is the goal, then anything goes, like C, but this is mostly useful for non U++ users. Another proposed reason was to increase the modularity and accessibility of parts of U++ for random environments, and that is all about having the maximum power done in the most U++ way without forcing people to use a full package and TheIDE. This is useful for all people, maybe even more to U++ users. If this is the case, the allocator that will be created from this attempt should be C++, with the best and leanest OOP design and be well documented. And created with the expectation that while not all people will use Thread, most will, since outside of boost there are no threads in C++.

So which approach shall we take?

And I also think that before we get started we answer both the previous question and decide on what libs to create and how? While U++ is not popular, it is a good product and it would be a shame to ruin it or increase its complexity with some ad-hoc decisions meant to attract new people.
Re: Ideas on U++ as library [message #45669 is a reply to message #45668] Sun, 20 December 2015 20:28 Go to previous messageGo to next message
dolik.rce is currently offline  dolik.rce
Messages: 1789
Registered: August 2008
Location: Czech Republic
Ultimate Contributor

cbpporter wrote on Sun, 20 December 2015 17:43
Maybe it should be clearly set in stone what we are trying to achieve with this. There where multiple reasons proposed in the larger thread for making libs.

This is very good point. When I started this thread, I was thinking mainly about the second case you mention, that is to increase the accessibility, while keeping things U++ way - fast, simple and modular.

Concerning the allocator, I believe it could simply provide both C and C++ API. Both is easily possible from C++ library. My original idea was something like jemalloc, which can be simply preloaded before loading any other program. It overrides the calls to malloc/free, new/delete and other memmory handling functions and provides better performance than the standard version without even recompiling the application. It is super easy to test (for jemalloc it is just calling "LD_PRELOAD=libjemalloc.so.1 MyApplication") so people can see the performance boost quickly and may even start digging into the underlaying U++ technology.

cbpporter wrote on Sun, 20 December 2015 17:43
And created with the expectation that while not all people will use Thread, most will, since outside of boost there are no threads in C++.
Actually there is std::thread in C++11 Wink

cbpporter wrote on Sun, 20 December 2015 17:43
So which approach shall we take?

And I also think that before we get started we answer both the previous question and decide on what libs to create and how? While U++ is not popular, it is a good product and it would be a shame to ruin it or increase its complexity with some ad-hoc decisions meant to attract new people.

I think those two approaches might go in parallel. Mirek is probably the right person to separate the allocator (I quickly tried today and it seems to be quite tagled with other Core code), while I, or anyone else can try to turn the U++ packages into libraries. Both task are IMHO important.

Honza
Re: Ideas on U++ as library [message #45671 is a reply to message #45669] Sun, 20 December 2015 21:39 Go to previous messageGo to next message
cbpporter is currently offline  cbpporter
Messages: 1406
Registered: September 2007
Ultimate Contributor
dolik.rce wrote on Sun, 20 December 2015 21:28

cbpporter wrote on Sun, 20 December 2015 17:43
And created with the expectation that while not all people will use Thread, most will, since outside of boost there are no threads in C++.
Actually there is std::thread in C++11 Wink

There is? Sorry, I'm really behind with my C++11. The thing is I have pretty much given up hope on C++. By the time the comity gets the balls to turn C++ into an elegant productivity language, I've moved on hopefully to something better.
Re: Ideas on U++ as library [message #45674 is a reply to message #45668] Mon, 21 December 2015 10:05 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13996
Registered: November 2005
Ultimate Member
cbpporter wrote on Sun, 20 December 2015 17:43

And I also think that before we get started we answer both the previous question and decide on what libs to create and how? While U++ is not popular, it is a good product and it would be a shame to ruin it or increase its complexity with some ad-hoc decisions meant to attract new people.


If there is something I can guarantee, any efforts like this will NOT affect U++ usability.

The only thing I plan here is to remove C++ constructs from heap allocator, which will lead to slight increase in code complixity (I estimate 10-20%), but nothing serious. Mostly, it is just about removing nested structs and replacing methods with functions (with _this parameter Smile. This will make possible to reuse the code for C easily. About the worth thing I might do is to add some comments meant as markers for 'export' utility.
Re: Ideas on U++ as library [message #45677 is a reply to message #45674] Mon, 21 December 2015 17:51 Go to previous messageGo to next message
cbpporter is currently offline  cbpporter
Messages: 1406
Registered: September 2007
Ultimate Contributor
Well I did spend the better half of my Sunday night googling allocators, including jemalloc and watching this: https://www.youtube.com/watch?v=RcWp5vwGlYU

I'm afraid if the first step is the allocator, it needs to be accompanied by a benchmark too Smile.
Re: Ideas on U++ as library [message #45680 is a reply to message #45677] Mon, 21 December 2015 20:52 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13996
Registered: November 2005
Ultimate Member
cbpporter wrote on Mon, 21 December 2015 17:51
Well I did spend the better half of my Sunday night googling allocators, including jemalloc and watching this: https://www.youtube.com/watch?v=RcWp5vwGlYU

I'm afraid if the first step is the allocator, it needs to be accompanied by a benchmark too Smile.


No problem. But you might help, it is fun.

Starting point might be benchmarks/SimpleAlloc.

I yet have to invent some MT benchmarks.
Re: Ideas on U++ as library [message #45687 is a reply to message #45680] Tue, 22 December 2015 03:39 Go to previous message
Novo is currently offline  Novo
Messages: 1371
Registered: December 2006
Ultimate Contributor
mirek wrote on Mon, 21 December 2015 14:52

I yet have to invent some MT benchmarks.

This code can be useful, IMHO.


Regards,
Novo
Previous Topic: Considering different approach to Win32 release
Next Topic: Parallel sort
Goto Forum:
  


Current Time: Thu Jul 25 00:40:13 CEST 2024

Total time taken to generate the page: 0.28434 seconds