U++ forum: Welcome to the forum

Status & Roadmap

Authors & License

Funding Ultimate++

Search on this site

Search in forums

Home » Developing U++ » U++ Developers corner » Choosing the best way to go full UNICODE

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Return to the default flat view

Create a new topic

Submit Reply

Re: Choosing the best way to go full UNICODE [message #48237 is a reply to message #48236]

Thu, 08 June 2017 11:22

cbpporter is currently offline

cbpporter
Messages: 1427
Registered: September 2007

Ultimate Contributor

Here are the primary stats:
- there are only 5721 characters that have decomposition.
- 2060 out of them have normal decomposition. All of these are two characters. So representing them is easy. Unfortunately the highest CP is 2FA1D. But it is CJK compatibility. I think it should be ignored. Without those the highest codepoint is 1D1C0. But if you ignore a bunch of stuff, like hebrew, hiragana, musical notations I think one could stop even as low as the aptly named character: http://www.fileformat.info/info/unicode/char/2adc/index.htm

Going lower than 0x2adc will soon cut of stuff like Greek. You need 2000 to not cut off Greek.

- the rest of decomposition are 2 to 4, but there are some weird exceptions, like a 18 character one.
- if you stop ar 0x2000/Greek the max CP a decomposition is 8190. If yous top at 0x2adc, it is 12297.

So a dump scheme for the first 0x2adc or a bit higher would be 43.888 bytes. In my lib I already have (256 * 256 * PLANES / BS) * 2 + count2 * 8 = 131456 bytes of data used by uppercase/lower case, so even 43k more is pushing it. I'll investigate how to represent sparsely both the first 0x2adc characters with exact 2 character long decomposition the entire Unicode range.

Report message to a moderator

Send a private message to this user

[Message index]

		Choosing the best way to go full UNICODE By: mirek on Sat, 27 May 2017 16:58
		Re: Choosing the best way to go full UNICODE By: Zbych on Sat, 27 May 2017 20:02
		Re: Choosing the best way to go full UNICODE By: mirek on Sat, 27 May 2017 20:48
		Re: Choosing the best way to go full UNICODE By: mirek on Sat, 27 May 2017 20:51
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 29 May 2017 14:36
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 29 May 2017 19:40
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 30 May 2017 10:31
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 30 May 2017 11:03
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 30 May 2017 11:23
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 30 May 2017 11:45
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 10:30
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 11:00
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 11:30
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 12:07
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 12:26
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 12:40
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 13:12
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 13:20
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 13:43
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 14:41
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 15:06
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 15:25
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 15:34
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 15:38
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 15:50
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 05 June 2017 17:51
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 09:28
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 06 June 2017 10:41
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 11:18
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 06 June 2017 13:21
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 13:39
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 13:58
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 10:00
		Re: Choosing the best way to go full UNICODE By: mirek on Thu, 08 June 2017 10:26
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 10:43
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 11:22
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 13:00
		Re: Choosing the best way to go full UNICODE By: mirek on Sun, 11 June 2017 13:57
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 09:39
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:13
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 10:21
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:28
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:31
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 10:53
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:57
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 11:37
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 11:41
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 12:50
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 13:06
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 14:20
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 13 June 2017 16:31
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 11:07
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 12:07
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 14 June 2017 12:30
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 12:42
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 14 June 2017 19:09
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 23:19
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 23:31
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 19 June 2017 10:03
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 10:22
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 19 June 2017 10:40
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 10:51
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 19 June 2017 10:58
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 11:07
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 10:23
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 14 June 2017 12:17

Previous Topic:	Some addition proposals
Next Topic:	Help needed with link errors (serversocket)

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

PDF

]

Current Time: Tue Jul 01 09:59:18 CEST 2025

Total time taken to generate the page: 0.04637 seconds