|
|
Home » Developing U++ » U++ Developers corner » Choosing the best way to go full UNICODE
Re: Choosing the best way to go full UNICODE [message #48243 is a reply to message #48238] |
Sun, 11 June 2017 13:57 |
|
mirek
Messages: 13980 Registered: November 2005
|
Ultimate Member |
|
|
cbpporter wrote on Thu, 08 June 2017 13:00Not quite 100% done analyzing the data, but here is what I think I'll do:
- respect the 3 plane convention. Unicode has 17 planes, with the first 3 in active use. Plane 14 is used, but it is specific and only has 368 allocated code points. It is so specific that I'll add exclude it, the same as I do all planes except planes 0-2. All excluded planes have the property that any function f(cp) = cp.
- I'll ignore all special substitutions: sub and superscript, font, circle, square, fractions and of course compatibility substitutions. I won't be using a flag for now, just exclude them.
- I'll ignore all CJK COMPATIBILITY IDEOGRAPHs. There is no way a general purpose library can provide satisfactory use case for these. If you really needs such substitution, you will probably use a more competent third party library. f(CJK COMPATIBILITY IDEOGRAPH) = CJK COMPATIBILITY IDEOGRAPH
All these combined with my two table solution, with a chunk size of 256 to 1024 will leave me with around 8000-9000 bytes of data in each executable that does decomposition. Final numbers will be determined once implementation is done and round trip testing is complete.
I think this is a reasonable subset that can handle NFD, at the small price of a flat 9K in exe size, + the size of the actual methods.
I have managed to squeeze complete composition to 3.8KB table...
Interesting observation: With UnicodeCompose / Decompose, with first 2048 codepoints covered by "fast table", there is no need for further tables for ToUpper, ToLower, ToAscii.
|
|
|
|
|
Choosing the best way to go full UNICODE
By: mirek on Sat, 27 May 2017 16:58
|
|
|
Re: Choosing the best way to go full UNICODE
By: Zbych on Sat, 27 May 2017 20:02
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Sat, 27 May 2017 20:48
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Sat, 27 May 2017 20:51
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Mon, 29 May 2017 19:40
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Tue, 30 May 2017 11:03
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Tue, 30 May 2017 11:45
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Wed, 31 May 2017 11:00
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Wed, 31 May 2017 12:07
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Wed, 31 May 2017 12:40
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Wed, 31 May 2017 13:20
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Wed, 31 May 2017 14:41
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Wed, 31 May 2017 15:25
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Wed, 31 May 2017 15:34
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Wed, 31 May 2017 15:50
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Mon, 05 June 2017 17:51
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Tue, 06 June 2017 10:41
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Tue, 06 June 2017 13:21
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Thu, 08 June 2017 10:26
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Sun, 11 June 2017 13:57
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Mon, 12 June 2017 10:13
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Mon, 12 June 2017 10:28
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Mon, 12 June 2017 10:31
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Mon, 12 June 2017 10:57
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Mon, 12 June 2017 11:41
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Wed, 14 June 2017 12:30
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Wed, 14 June 2017 19:09
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Mon, 19 June 2017 10:03
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Mon, 19 June 2017 10:40
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Mon, 19 June 2017 10:58
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
|
|
|
Re: Choosing the best way to go full UNICODE
By: mirek on Wed, 14 June 2017 12:17
|
Goto Forum:
Current Time: Wed May 29 06:36:02 CEST 2024
Total time taken to generate the page: 0.01803 seconds
|
|
|