Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Community » U++ community news and announcements » 32 bit wchar - for now in branch
32 bit wchar - for now in branch [message #57636] Tue, 19 October 2021 20:22 Go to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Recently I am transitioning wchar to 32 bit for improved unicode support. This is the first for the full Unicode support - at this phase, I do not yet expect advanced text shaping to be used, just to allow codepoints outside of basic plane (codepoints > 65536).

Today I have reached the milestone when I have finally displayed some emoji characters in UWord in Win32, so maybe this is time to announce the existence of 'wchar' branch on github - that is the one with sizeof(wchar) == 4 so that maybe you can start testing this too.

So far I am only supporting Win32.

There was quite a lot things to fix in uppsrc (unexpectly so). The fact that WCHAR is not the same as wchar caused some probles as well as '2' instead of 'sizeof(wchar)' in some places (yep, thats bad coding 10 years ago, shame on me). So I recommend to do "Find all" "WCHAR" and "wchar" in your code just to check all is OK...

(Note that I am regularly rebasing 'wchar' branch, which unfortunately means force pushes and rewriting 'wchar' history...)

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57642 is a reply to message #57636] Sat, 30 October 2021 17:41 Go to previous messageGo to next message
zsolt is currently offline  zsolt
Messages: 697
Registered: December 2005
Location: Budapest, Hungary
Contributor
Thanks!

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57648 is a reply to message #57642] Sat, 06 November 2021 16:38 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
It now works on all platforms, the merge is imminent...

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57663 is a reply to message #57648] Mon, 15 November 2021 11:59 Go to previous messageGo to next message
Tom1
Messages: 1212
Registered: March 2007
Senior Contributor
Hi Mirek,

Before merge... It seems there are issues with MSBT19x64 compilation here:

C:\upp-16111\ultimatepp-wchar\uppsrc\Draw\FontWin32.cpp(157): error C2446: ':': no conversion from 'int' to 'Upp::Vector<Upp::char16>'
C:\upp-16111\ultimatepp-wchar\uppsrc\Draw\FontWin32.cpp(157): note: Constructor for class 'Upp::Vector<Upp::char16>' is declared 'explicit'
C:\upp-16111\ultimatepp-wchar\uppsrc\Draw\FontWin32.cpp(157): error C2660: 'EnumFontFamiliesW': function does not take 3 arguments

And here:
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\LocalProcess.cpp(85): error C2446: ':': no conversion from 'int' to 'Upp::Vector<Upp::char16>'
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\LocalProcess.cpp(85): note: Constructor for class 'Upp::Vector<Upp::char16>' is declared 'explicit'
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\LocalProcess.cpp(85): error C2660: 'CreateProcessW': function does not take 9 arguments

Additionally, there are a whole bunch of warnings on MSBT19x64. I did not try MSBT19x32 yet...

Best regards,

Tom

EDIT: PS. I updated to current git source today before testing.

[Updated on: Mon, 15 November 2021 12:01]

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57664 is a reply to message #57663] Mon, 15 November 2021 15:28 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Tom1 wrote on Mon, 15 November 2021 11:59
Hi Mirek,

Before merge... It seems there are issues with MSBT19x64 compilation here:

C:\upp-16111\ultimatepp-wchar\uppsrc\Draw\FontWin32.cpp(157): error C2446: ':': no conversion from 'int' to 'Upp::Vector<Upp::char16>'
C:\upp-16111\ultimatepp-wchar\uppsrc\Draw\FontWin32.cpp(157): note: Constructor for class 'Upp::Vector<Upp::char16>' is declared 'explicit'
C:\upp-16111\ultimatepp-wchar\uppsrc\Draw\FontWin32.cpp(157): error C2660: 'EnumFontFamiliesW': function does not take 3 arguments

And here:
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\LocalProcess.cpp(85): error C2446: ':': no conversion from 'int' to 'Upp::Vector<Upp::char16>'
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\LocalProcess.cpp(85): note: Constructor for class 'Upp::Vector<Upp::char16>' is declared 'explicit'
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\LocalProcess.cpp(85): error C2660: 'CreateProcessW': function does not take 9 arguments

Additionally, there are a whole bunch of warnings on MSBT19x64. I did not try MSBT19x32 yet...

Best regards,

Tom

EDIT: PS. I updated to current git source today before testing.


Thank you! Fixes pushed. Let me know....

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57665 is a reply to message #57664] Mon, 15 November 2021 15:55 Go to previous messageGo to next message
Tom1
Messages: 1212
Registered: March 2007
Senior Contributor
Hi Mirek,

Thanks! Now it compiles without any errors. Just some warnings left with MSBT19x64 and MSBT19:
Utf.cpp
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\Utf.cpp(136): warning C4927: illegal conversion; more than one user-defined conversion has been implicitly applied
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\Utf.cpp(136): note: while calling the constructor 'Upp::WString::WString(const Upp::wchar *)'
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\String.h(831): note: see declaration of 'Upp::WString::WString'
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\Utf.cpp(156): warning C4927: illegal conversion; more than one user-defined conversion has been implicitly applied
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\Utf.cpp(156): note: while calling the constructor 'Upp::WString::WString(const Upp::wchar *)'
C:\upp-16111\ultimatepp-wchar\uppsrc\Core\String.h(831): note: see declaration of 'Upp::WString::WString'

I will run some more tests tomorrow.

Best regards,

Tom

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57666 is a reply to message #57665] Mon, 15 November 2021 19:27 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Tried to fix that as well (although I am not getting warnings there).

Mirek

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57668 is a reply to message #57666] Tue, 16 November 2021 08:28 Go to previous messageGo to next message
Tom1
Messages: 1212
Registered: March 2007
Senior Contributor
Hi Mirek,

Success there: Now MSBT19x64 and MSBT19 compile without errors and warnings. (The difference between our compiler environments may be that I just updated my MSBT yesterday to its latest version.)

Is MSBT17x64 and MSBT17 still relevant? If so, there are errors.

The 32-bit CLANG compilation on Windows fails with this error:
----- ide/Java ( GUI CLANG BLITZ WIN32 ) (17 / 42)
BLITZ: JavaVersion.cpp Java.cpp Jdk.cpp
In file included from C:/upp-16111/out/uppsrc/ide/Debuggers/CLANG.Blitz.Gui\ide/Debuggers$blitz.cpp:84:
c:\upp-16111\ultimatepp-wchar\uppsrc\ide\Debuggers\Sym.cpp:446:3: error: no matching function for call to 'SymEnumTypes'
                SymEnumTypes(hProcess, current_modbase, [](PSYMBOL_INFO pSym, ULONG SymbolSize, PVOID UserContext)->int {
                ^~~~~~~~~~~~
C:\upp-16111\bin\clang\include\psdk_inc/_dbg_common.h:1005:20: note: candidate function not viable: no known conversion from '(lambda at c:\upp-16111\ultimatepp-wchar\uppsrc\ide\Debuggers\Sym.cpp:446:43)' to 'PSYM_ENUMERATESYMBOLS_CALLBACK' (
    aka 'int (*)(_SYMBOL_INFO *, unsigned long, void *) __attribute__((stdcall))') for 3rd argument
  WINBOOL IMAGEAPI SymEnumTypes(HANDLE hProcess,ULONG64 BaseOfDll,PSYM_ENUMERATESYMBOLS_CALLBACK EnumSymbolsCallback,PVOID UserContext);
                   ^


Best regards,

Tom

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57669 is a reply to message #57668] Tue, 16 November 2021 08:35 Go to previous messageGo to next message
Tom1
Messages: 1212
Registered: March 2007
Senior Contributor
In more detail, MSBT17 builds OK but MSBT17x64 fails on:
c:\upp-16111\ultimatepp-wchar\uppsrc\core\SIMD_SSE2.h(8): error C2668: 'Upp::_mm_storeu_ps': ambiguous call to overloaded function
c:/program files (x86)/microsoft visual studio/2017/buildtools/vc/tools/msvc/14.16.27023/include\intrin.h(853): note: could be 'void Upp::_mm_storeu_ps(float *,__m128)'
c:/program files (x86)/microsoft visual studio/2017/buildtools/vc/tools/msvc/14.16.27023/include\xmmintrin.h(345): note: or       'void _mm_storeu_ps(float *,__m128)' [found using argument-dependent lookup]
c:\upp-16111\ultimatepp-wchar\uppsrc\core\SIMD_SSE2.h(8): note: while trying to match the argument list '(float *, __m128)'
c:\upp-16111\ultimatepp-wchar\uppsrc\core\SIMD_SSE2.h(9): error C2668: 'Upp::_mm_store_sd': ambiguous call to overloaded function
c:/program files (x86)/microsoft visual studio/2017/buildtools/vc/tools/msvc/14.16.27023/include\intrin.h(842): note: could be 'void Upp::_mm_store_sd(double *,__m128d)'
c:/program files (x86)/microsoft visual studio/2017/buildtools/vc/tools/msvc/14.16.27023/include\emmintrin.h(203): note: or       'void _mm_store_sd(double *,__m128d)' [found using argument-dependent lookup]
c:\upp-16111\ultimatepp-wchar\uppsrc\core\SIMD_SSE2.h(9): note: while trying to match the argument list '(double *, __m128d)'
...

And there are quite a few of these places in SIMD_SSE2.h with MSBT17x64. The same issue was present with MSBT19x64 until I updated the compiler, so this must be a compiler bug that needs bypassing.

Best regards,

Tom

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57670 is a reply to message #57669] Tue, 16 November 2021 09:05 Go to previous messageGo to next message
Tom1
Messages: 1212
Registered: March 2007
Senior Contributor
Another strange thing happened: OpenGL has stopped working. E.g. reference/OpenGL no longer works with this, while at 16111 it was still fine.

Best regards,

Tom

EDIT: The current git version works too. It's just this wchar version that does not show OpenGL content.

[Updated on: Tue, 16 November 2021 09:25]

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57677 is a reply to message #57670] Wed, 17 November 2021 10:22 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Tom1 wrote on Tue, 16 November 2021 09:05
Another strange thing happened: OpenGL has stopped working. E.g. reference/OpenGL no longer works with this, while at 16111 it was still fine.

Best regards,

Tom

EDIT: The current git version works too. It's just this wchar version that does not show OpenGL content.


I am a bit confused by EDIT....

What do you mean by "current git version"? wchar head or master head?

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57678 is a reply to message #57677] Wed, 17 November 2021 11:09 Go to previous messageGo to next message
Tom1
Messages: 1212
Registered: March 2007
Senior Contributor
Hi Mirek,

I mean the master head. So master works, but wchar does not. Sorry about the unclear edit...

Best regards,

Tom

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57679 is a reply to message #57678] Wed, 17 November 2021 11:12 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Hopefully fixed. Thanks for spotting this.

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57680 is a reply to message #57679] Wed, 17 November 2021 12:48 Go to previous messageGo to next message
Tom1
Messages: 1212
Registered: March 2007
Senior Contributor
Mirek,

Thanks! It's all good now: CLANG, CLANGx64, MSBT17, MSBT17x64, MSBT19 and MSBT19x64.

Also, GLCtrl works properly again.

Best regards,

Tom

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57682 is a reply to message #57680] Wed, 17 November 2021 13:27 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Please keep testing, this is most valuable!

Mirek

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57683 is a reply to message #57682] Wed, 17 November 2021 14:37 Go to previous messageGo to next message
Tom1
Messages: 1212
Registered: March 2007
Senior Contributor
I'm working on it... Smile

Best regards,

Tom

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57687 is a reply to message #57683] Fri, 19 November 2021 09:26 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Tom1 wrote on Wed, 17 November 2021 14:37
I'm working on it... Smile

Best regards,

Tom


I would like to merge asap, probably today. Do you think it is OK?

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57688 is a reply to message #57687] Fri, 19 November 2021 10:48 Go to previous messageGo to next message
Tom1
Messages: 1212
Registered: March 2007
Senior Contributor
Hi Mirek,

I have (mostly) tested my programs here. Everything compiles and everything at least seems to work just like they did before on Windows side.

However, on Linux Mint I just tried to compile reference/OpenGL and it failed with the following error:
1 error generated.
In file included from /home/tom/ultimatepp-wchar/uppsrc/CtrlCore/CtrlCoreInit.cpp:1:
In file included from /home/tom/ultimatepp-wchar/uppsrc/CtrlCore/CtrlCore.h:4:
In file included from /home/tom/ultimatepp-wchar/uppsrc/RichText/RichText.h:4:
In file included from /home/tom/ultimatepp-wchar/uppsrc/Draw/Draw.h:6:
In file included from /home/tom/ultimatepp-wchar/uppsrc/Core/Core.h:66:
/usr/lib/llvm-10/lib/clang/10.0.0/include/intrin.h:12:15: fatal error: 'intrin.h' file not found
#include_next <intrin.h>
              ^~~~~~~~~~


Best regards,

Tom

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57689 is a reply to message #57688] Sat, 20 November 2021 10:17 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
Tom1 wrote on Fri, 19 November 2021 10:48
Hi Mirek,

I have (mostly) tested my programs here. Everything compiles and everything at least seems to work just like they did before on Windows side.

However, on Linux Mint I just tried to compile reference/OpenGL and it failed with the following error:
1 error generated.
In file included from /home/tom/ultimatepp-wchar/uppsrc/CtrlCore/CtrlCoreInit.cpp:1:
In file included from /home/tom/ultimatepp-wchar/uppsrc/CtrlCore/CtrlCore.h:4:
In file included from /home/tom/ultimatepp-wchar/uppsrc/RichText/RichText.h:4:
In file included from /home/tom/ultimatepp-wchar/uppsrc/Draw/Draw.h:6:
In file included from /home/tom/ultimatepp-wchar/uppsrc/Core/Core.h:66:
/usr/lib/llvm-10/lib/clang/10.0.0/include/intrin.h:12:15: fatal error: 'intrin.h' file not found
#include_next <intrin.h>
              ^~~~~~~~~~


Best regards,

Tom


Hopefully fixed.

Report message to a moderator

Re: 32 bit wchar - for now in branch [message #57690 is a reply to message #57689] Sat, 20 November 2021 11:31 Go to previous messageGo to previous message
mirek is currently offline  mirek
Messages: 13975
Registered: November 2005
Ultimate Member
I am now experiencing two minor issues with wchar32 theide which I am not sure whether are releated to wchar32 or perhaps some changes in windows:

- sometimes dropping the droplist (e.g. switching the build method) seems delayed or does not happen at all.
- After building, there is "Gathering files" progress; this should basically just check for changed files (by file stamps) and as far as I remember, was quick. Sometimes this now takes longer time than I would like, which does not really make sense at all (because those timestamps should be read from OS cache).

If you are running wchar32 theide, have seen these? (If not, I would appreciate you compiled and used it.)

Report message to a moderator

Previous Topic: text<->double conversions refactored
Next Topic: ide: Ctrl+D - duplicate - now works on selection too
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ] [ PDF ]

Current Time: Thu Apr 25 00:59:11 CEST 2024

Total time taken to generate the page: 0.05381 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.5.
Copyright ©2001-2013 FUDforum Bulletin Board Software