U++ forum: Welcome to the forum

Status & Roadmap

Authors & License

Funding Ultimate++

Search on this site

Search in forums

Home » Developing U++ » U++ Developers corner » Choosing the best way to go full UNICODE

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Return to the default flat view

Create a new topic

Submit Reply

Re: Choosing the best way to go full UNICODE [message #48188 is a reply to message #48187]

Wed, 31 May 2017 13:43

cbpporter is currently offline

cbpporter
Messages: 1401
Registered: September 2007

Ultimate Contributor

Without RLE, I think you can get around this by positions not representing characters, but sequence starts.

As a mandatory condition, every time the position is updated, it is guaranteed to be at the start of a sequence.

As a more complicated example, let's say you have a selection of text, with a "begin" and "end" pos. You handle a key press. Taking your start pos as a sequence start, you determine the sequence end. This means looking to see how many code units it is, seeking over combination marks and ligatures. Basically on the fly glyph analysis. With sequence start end end you know for a fact that everything between these two values must go. You do the same for the end position. As an optimization, you can mark everything for deletion between the start sequence begin and end sequence end. The text marked to be replaced will replaced with multiple code units.

The real challenge is to standardize these operations so you don't have to repeat them.

Maybe we need some GlyphInfoExtractor class or something. Something when given a random sequence of code units and a valid code point start, it can handle such common operations?

Here is a sample from unciode.org:

index.php?t=getfile&id=5305&private=0

This is 14 code units, 5 code units, 4 glyphs. The user will see and recognize 4 items as more or less "atomic", so we should focus on this.

We need and API that can locate each glyph start and allow us to replace glyphs 2 and 3 with an already properly encoded glyph sequence, like ſʒ, as an example.

This can be done on the fly with something high level like:
s = s.GlyphMid(0, 1) + "ſʒ" + s.GlyphMid(3, 1)
or
s.GlyphReplace(2, 3, "ſʒ")

or we can go lower level. Or we can go into multi-byte String territory.

PS: the high level stuff still is StringWalker territory.

Attachment: char_combmark_ex1.png
(Size: 1.47KB, Downloaded 544 times)

Report message to a moderator

Send a private message to this user

[Message index]

		Choosing the best way to go full UNICODE By: mirek on Sat, 27 May 2017 16:58
		Re: Choosing the best way to go full UNICODE By: Zbych on Sat, 27 May 2017 20:02
		Re: Choosing the best way to go full UNICODE By: mirek on Sat, 27 May 2017 20:48
		Re: Choosing the best way to go full UNICODE By: mirek on Sat, 27 May 2017 20:51
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 29 May 2017 14:36
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 29 May 2017 19:40
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 30 May 2017 10:31
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 30 May 2017 11:03
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 30 May 2017 11:23
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 30 May 2017 11:45
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 10:30
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 11:00
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 11:30
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 12:07
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 12:26
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 12:40
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 13:12
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 13:20
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 13:43
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 14:41
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 15:06
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 15:25
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 15:34
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 31 May 2017 15:38
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 31 May 2017 15:50
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 05 June 2017 17:51
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 09:28
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 06 June 2017 10:41
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 11:18
		Re: Choosing the best way to go full UNICODE By: mirek on Tue, 06 June 2017 13:21
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 13:39
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 06 June 2017 13:58
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 10:00
		Re: Choosing the best way to go full UNICODE By: mirek on Thu, 08 June 2017 10:26
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 10:43
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 11:22
		Re: Choosing the best way to go full UNICODE By: cbpporter on Thu, 08 June 2017 13:00
		Re: Choosing the best way to go full UNICODE By: mirek on Sun, 11 June 2017 13:57
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 09:39
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:13
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 10:21
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:28
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:31
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 10:53
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 10:57
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 11:37
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 12 June 2017 11:41
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 12:50
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 13:06
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 12 June 2017 14:20
		Re: Choosing the best way to go full UNICODE By: cbpporter on Tue, 13 June 2017 16:31
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 11:07
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 12:07
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 14 June 2017 12:30
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 12:42
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 14 June 2017 19:09
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 23:19
		Re: Choosing the best way to go full UNICODE By: cbpporter on Wed, 14 June 2017 23:31
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 19 June 2017 10:03
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 10:22
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 19 June 2017 10:40
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 10:51
		Re: Choosing the best way to go full UNICODE By: mirek on Mon, 19 June 2017 10:58
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 11:07
		Re: Choosing the best way to go full UNICODE By: cbpporter on Mon, 19 June 2017 10:23
		Re: Choosing the best way to go full UNICODE By: mirek on Wed, 14 June 2017 12:17

Previous Topic:	Some addition proposals
Next Topic:	Help needed with link errors (serversocket)

Goto Forum:

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

PDF

]

Current Time: Wed May 15 19:07:15 CEST 2024

Total time taken to generate the page: 0.02483 seconds