U++ forum: Welcome to the forum

Status & Roadmap

Authors & License

Funding Ultimate++

Search on this site

Search in forums

Home » Developing U++ » U++ Developers corner » Choosing the best way to go full UNICODE

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Return to the default flat view

Create a new topic

Submit Reply

Re: Choosing the best way to go full UNICODE [message #48160 is a reply to message #48153]

Mon, 29 May 2017 14:36

cbpporter is currently offline

cbpporter
Messages: 1401
Registered: September 2007

Ultimate Contributor

Hi Mirek!

Welcome to my problems, circa 1-2 years in of using U++ Laughing

Laughing

.

I didn't know if you remember, but I was pressuring you left and right to fix these issues, with occasional fixes and what not.

Some fixes managed to get in, but there was no real interest in it, so my solution was to eventually deprecate whole String support in U++, using it as a pure storage. String + custom conversions methods + some custom things, like ToUpper, fixes the problems.

U++ is at least a decade behind on Unicode, with absolutely no good reason. It is not like the issue can't all be fixed in like 3 weeks. I'm sure it would fail 100% of the more difficult compatibility tests.

The good news is that these issues are so rare that you are extraordinarily unlikely to encounter them as an American or European.

As anecdotal proof, this is the first time you encountered issues.

I'm using a very dense data representation method, but even like this, full support is almost 200 KiB of data in every single executable. This is without more advanced stuff, like scripts. I believe 180 KiB is the minimum data that can be stored for the first 3 Unicode planes, this vs. your 2048 entry table in U++, but this gives you lower, upper, title case and Unicode category. I have unit-tests covering these, making sure they are always correct.

String and WString are used for storage. Mostly String.

Contrary to popular belief, Unicode code points are not indexable, so DString does not help. Unicode is glyph based. This + DString being the least beneficent method of storage makes DString not really needed.

In most cases, you can treat the String as opaque.

When not, the ideal solution is an amortized string walker and light algorithm rewrites. You can create an indexed string walker, that seeks, but if you ask it for indices in order, the seek overhead is basically zero. This is only needed for glyph based algorithms, like case conversion and rendering. Or a traditional string walker without indices, a bit faster, but a lit uglier to use, with Get and ++/-- methods.

Report message to a moderator

Send a private message to this user

[Message index]

		Choosing the best way to go full UNICODE By: mirek on Sat, 27 May 2017 16:58
		Re: Choosing the best way to go full UNICODE By: Zbych on Sat, 27 May 2017 20:02
		Re: Choosing the best way to go full UNICODE By: mirek on Sat, 27 May 2017 20:48
		Re: Choosing the best way to go full UNICODE By: mirek on Sat, 27 May 2017 20:51
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 29 May 2017 14:36
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 29 May 2017 19:40
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 30 May 2017 10:31
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 30 May 2017 11:03
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 30 May 2017 11:23
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 30 May 2017 11:45
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 10:30
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 11:00
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 11:30
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 12:07
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 12:26
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 12:40
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 13:12
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 13:20
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 13:43
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 14:41
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 15:06
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 15:25
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 15:34
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 15:38
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 15:50
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 05 June 2017 17:51
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 09:28
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 06 June 2017 10:41
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 11:18
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 06 June 2017 13:21
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 13:39
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 13:58
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 10:00
		Re: Choosing the best way to go full UNICODE By: mirek on Thu, 08 June 2017 10:26
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 10:43
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 11:22
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 13:00
		Re: Choosing the best way to go full UNICODE By: mirek on Sun, 11 June 2017 13:57
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 09:39
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:13
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 10:21
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:28
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:31
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 10:53
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:57
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 11:37
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 11:41
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 12:50
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 13:06
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 14:20
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 13 June 2017 16:31
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 11:07
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 12:07
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 14 June 2017 12:30
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 12:42
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 14 June 2017 19:09
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 23:19
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 23:31
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 19 June 2017 10:03
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 10:22
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 19 June 2017 10:40
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 10:51
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 19 June 2017 10:58
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 11:07
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 10:23
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 14 June 2017 12:17

Previous Topic:	Some addition proposals
Next Topic:	Help needed with link errors (serversocket)

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

PDF

]

Current Time: Mon May 06 20:31:34 CEST 2024

Total time taken to generate the page: 0.02284 seconds