Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » U++ Library support » U++ Libraries and TheIDE: i18n, Unicode and Internationalization » It's suspected to be an issue with Font.
It's suspected to be an issue with Font. [message #31658] Fri, 18 March 2011 23:50 Go to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
The programs I used to compare are UWord from the UPP, and MS Word. The platform is Windows 7.

The text I used to test is:

ultimate++是一个性能优良的C++GUI库


The problem with U++ drawed text is that some characters are notably larger than others and some have incorrect horizontal displacement.

Please see attached picture for a visual effect.


I also encountered issue where chinese characters are displayed correctly displayed on Windows but are blank on Ubuntu. And when I copies the same text that was displayed as blank to, say gedit, the text displayed correctly as in Windows. That part I will attach picture in future.
Re: It's suspected to be an issue with Font. [message #31942 is a reply to message #31658] Sun, 10 April 2011 14:42 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12497
Registered: November 2005
Ultimate Member
Lance wrote on Fri, 18 March 2011 18:50

The programs I used to compare are UWord from the UPP, and MS Word. The platform is Windows 7.

The text I used to test is:

ultimate++是一个性能优良的C++GUI库


The problem with U++ drawed text is that some characters are notably larger than others and some have incorrect horizontal displacement.



It works fine on my Windows 7. However, I believe that the problem is caused by font substitution mechanism and perhaps on your system, you have some font that takes precendence for some glyphs, but does not contain other characters.

However, I have noticed that OpenOffice or Wordpad are preferring SimSun font, so I have given it higher priority now. Please check.

The core table for font subsitutions is in Draw/FontCR, table sFontReplacements. If U++ does not find required glyph in the font, it tries to get the glyph by going through fonts in this table and uses first glyph available... Maybe you could play with it a bit to get better results.

Quote:


I also encountered issue where chinese characters are displayed correctly displayed on Windows but are blank on Ubuntu. And when I copies the same text that was displayed as blank to, say gedit, the text displayed correctly as in Windows. That part I will attach picture in future.



I believe this is basically the same issue - gedit has better knowledge about fonts, so is able to find a better replacement.

Maybe, if openoffice behaves similary in Ubuntu as in Windows, you can try to paste characters into Openoffice and then check what font it has actually used for it. Then we can add this font to replacement table...

Alternatively please post some problematic text here so that I can do this myself.
Re: It's suspected to be an issue with Font. [message #31967 is a reply to message #31942] Tue, 12 April 2011 14:03 Go to previous messageGo to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
Great Mirek.

Thank you very much for your effort. I didn't know you were working on it. Thanks a million.

I will download the most recent version and give it a try.

Lance
Re: It's suspected to be an issue with Font. [message #31968 is a reply to message #31967] Tue, 12 April 2011 15:19 Go to previous messageGo to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
Sorry but the problem doesn't seemed to be fixed.

I tested only on Windows 7 with version 3336, didn't notice any improvement. Even font substitution is the old way (SimHei I believe instead of SimSun).

Has be fix been committed yet?
Re: It's suspected to be an issue with Font. [message #32005 is a reply to message #31968] Sat, 16 April 2011 20:54 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12497
Registered: November 2005
Ultimate Member
Lance wrote on Tue, 12 April 2011 09:19

Sorry but the problem doesn't seemed to be fixed.

I tested only on Windows 7 with version 3336, didn't notice any improvement. Even font substitution is the old way (SimHei I believe instead of SimSun).

Has be fix been committed yet?


Well, it was.

Maybe you could try yourself, I believe it is easy to do so.

Maybe create just simple app that only draws the text in window, then put a DUMP to Replace(Font fnt, int chr, Font& rfnt) in CRFont.cpp to find out what is going on...

Re: It's suspected to be an issue with Font. [message #32006 is a reply to message #32005] Sat, 16 April 2011 21:02 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12497
Registered: November 2005
Ultimate Member
#include <CtrlLib/CtrlLib.h>

using namespace Upp;

struct MyApp : TopWindow {
	virtual void Paint(Draw& w) {
		w.DrawRect(GetSize(), White);
		w.DrawText(10, 10, "ultimate++是一个性能优良的C++GUI库");
	}
};

GUI_APP_MAIN
{
	MyApp().Run();
}



And this DDUMP

bool Replace(Font fnt, int chr, Font& rfnt)
{
	static Vector<int> rface;
	static Vector<dword> l, h;
	ONCELOCK {
		for(int i = 0; i < __countof(sFontReplacements) && rface.GetCount() < 20; i++) {
			int q = Font::FindFaceNameIndex(sFontReplacements[i].name);
			if(q > 0) {
				rface.Add(q);
				l.Add(sFontReplacements[i].l);
				h.Add(sFontReplacements[i].h);
			}
		}
	}

	Font f = fnt;
	dword tl = chr < 4096 ? 0x80000000 >> (chr >> 7) : 0;
	dword th = 0x8000000 >> ((dword)chr >> 11);
//	DDUMP(FormatIntHex(chr));
//	DDUMP(FormatIntHex(th));
	for(int i = 0; i < rface.GetCount(); i++) {
//		DDUMP(Font(rface[i], 10));
//		DDUMP(FormatIntHex(h[i]));
//		DDUMP(FormatIntHex(h[i] & th));
		if(((l[i] & tl) || (h[i] & th)) && IsNormal(f.Face(rface[i]), chr)) {
			int a = fnt.GetAscent();
			int d = fnt.GetDescent();
			if(f.GetAscent() > a || f.GetDescent() > d) {
				static sFontMetricsReplacement cache[256];
				int q = CombineHash(fnt, f) & 255;
				if(cache[q].src != fnt || cache[q].dst != f) {
					cache[q].src = fnt;
					cache[q].dst = f;
					while((f.GetAscent() > a || f.GetDescent() > d) && f.GetHeight() > 1) {
						f.Height(max(1, min(f.GetHeight() - 1, f.GetHeight() * 9 / 10)));
					}
					cache[q].mdst = f;
				}
				else
					f = cache[q].mdst;
			}
			rfnt = f;
			DDUMP(rfnt.GetFaceName()); // <<- HERE
			return true;
		}
	}
	return false;
}

[Updated on: Sat, 16 April 2011 21:03]

Report message to a moderator

Re: It's suspected to be an issue with Font. [message #32007 is a reply to message #32006] Sat, 16 April 2011 21:21 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12497
Registered: November 2005
Ultimate Member
BTW, I have tried with debian, default installation, and it does not display the text even in the browser Smile
Re: It's suspected to be an issue with Font. [message #32083 is a reply to message #32007] Wed, 20 April 2011 22:07 Go to previous messageGo to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
Just saw it. Thank for the effort! Font is too complicated a topic for me Sad but I'll definitely give it a try.

Re: It's suspected to be an issue with Font. [message #32084 is a reply to message #32083] Wed, 20 April 2011 22:30 Go to previous messageGo to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
Here is the output. The test was done on a Windows machine.


rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = MS UI Gothic
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = MS UI Gothic
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS

Re: It's suspected to be an issue with Font. [message #32199 is a reply to message #32084] Sat, 30 April 2011 10:37 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12497
Registered: November 2005
Ultimate Member
Lance wrote on Wed, 20 April 2011 16:30

Here is the output. The test was done on a Windows machine.


rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = MS UI Gothic
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = MS UI Gothic
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS




Well, for me, it is SimSun all the way...

Could you please try to paste the text into WordPad and tell me what font it reports?

Mirek
Re: It's suspected to be an issue with Font. [message #32214 is a reply to message #32199] Sun, 01 May 2011 17:23 Go to previous messageGo to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
Hi Mirek:

Thank you for your continued attention to this issue.

I don't have immediate access to a Windows U++ development environment. I did the same test on Ubuntu and here is a screen shot:

index.php?t=getfile&id=3208&private=0


See how different characters are missing in TheIDE editors and in the running test program? That is the problem I mentioned earlier regarding missing characters on Linux. BTW: it's Ubuntu 11.04.

And the LOG file:

rfnt.GetFaceName() = UnDotum
rfnt.GetFaceName() = UnDotum
rfnt.GetFaceName() = UnDotum
rfnt.GetFaceName() = UnDotum
rfnt.GetFaceName() = UnDotum
rfnt.GetFaceName() = UnDotum


I will report the result as soon as I can put my hands on a Windows version of TheIDE.
Re: It's suspected to be an issue with Font. [message #32215 is a reply to message #32214] Sun, 01 May 2011 17:35 Go to previous messageGo to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
As a comparison, in Ubuntu gedit, no matter what font I use, the Chinese characters either displayed as default font if these Characters are not in the Font's character set, or displayed with the font.

index.php?t=getfile&id=3209&private=0
Re: It's suspected to be an issue with Font. [message #32218 is a reply to message #32215] Sun, 01 May 2011 19:41 Go to previous messageGo to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
On Windows XP, here is the LOG file content:

rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = MS UI Gothic
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = MS UI Gothic
rfnt.GetFaceName() = Arial Unicode MS
rfnt.GetFaceName() = Arial Unicode MS
Re: It's suspected to be an issue with Font. [message #32223 is a reply to message #32218] Mon, 02 May 2011 07:07 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12497
Registered: November 2005
Ultimate Member
Well, but I am still not getting information about what font should be used on your systems.

I need to you to paste the text into editor that would display replacement font. I know for sure that OpenOffice is capable of doing that. WordPad in Win7 as well.

Simply paste text there, move cursor over CJK characters, it would show what font is used.
Re: It's suspected to be an issue with Font. [message #32234 is a reply to message #32223] Mon, 02 May 2011 15:58 Go to previous messageGo to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
It seems OpenOffice hide the font substitution from end user. Here is the result of my experiment in Ubuntu + Open Office. See the hilighted character is related with nominal font "FreeSerif"
index.php?t=getfile&id=3212&private=0


Let me know if I can somehow find the actual font used.

Regardless what font (serif, sans, etc), gedit/open office will render Chinese characters at uniform size and space, so the Upp way of interpret non-western font may have flaw. Interesting enough, on Windows, the characters almost always get displayed albeit with size/placement issues, while on Ubuntu, some characters cannot be displayed and become blanks.
Re: It's suspected to be an issue with Font. [message #32235 is a reply to message #32234] Mon, 02 May 2011 16:11 Go to previous messageGo to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
By the way, SimSun should be a safe bet. Any Chinese characters should be representable in SimSun. Say there is a fancy font FancyFont. It only implemented 2000 most common chinese characters, than a character that's not in this set should be displayed in SimSun
Re: It's suspected to be an issue with Font. [message #32240 is a reply to message #32235] Mon, 02 May 2011 20:07 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 12497
Registered: November 2005
Ultimate Member
Lance wrote on Mon, 02 May 2011 10:11

By the way, SimSun should be a safe bet. Any Chinese characters should be representable in SimSun. Say there is a fancy font FancyFont. It only implemented 2000 most common chinese characters, than a character that's not in this set should be displayed in SimSun


Well, but problem is that SimSun is already tested and required...

It is all really weird. In Windows, it works for me just fine. In your machine, it seems to choose "Arial Unicode MS" instead.

Hm, one possible explanation: Do not you have the same Arial Unicode MS installed on ubuntu?

It actually seems there is something wrong with that font, the replacement algorithm seems to have detected required glyphs in it...

OK, one more try: try to move SimSun in the list BEFORE Arial Unicode MS.

struct sRFace {
	const char *name;
	dword l, h;
} sFontReplacements[] = {
	{ "sans-serif", 0xffee0008, 0xdc000801 },
	{ "Arial", 0xfffe0000, 0x09c00080 },
	{ "SimSun", 0xfd800000, 0x09ffff00 },
	{ "Arial Unicode MS", 0xfffc3fef, 0xfa7ff7e7 },
	{ "MS UI Gothic", 0xffc01008, 0x0fffff00 },
	{ "MS Mincho", 0xffc01008, 0x0fffff00 },
.....


Let us see, maybe it could help in windows.

Another thing to consider is to list all fonts to find out whether there is something bad with names:

GUI_APP_MAIN
{
	for(int i = 0; i < Font::GetFaceCount(); i++)
		LOG(Font::GetFaceName(i));
}

[Updated on: Mon, 02 May 2011 20:07]

Report message to a moderator

Re: It's suspected to be an issue with Font. [message #32248 is a reply to message #32240] Tue, 03 May 2011 14:48 Go to previous messageGo to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
DDUMP font result.

rfnt.GetFaceName() = UnDotum
rfnt.GetFaceName() = UnDotum
rfnt.GetFaceName() = UnDotum
rfnt.GetFaceName() = UnDotum
rfnt.GetFaceName() = UnDotum
rfnt.GetFaceName() = UnDotum



List Fonts Result:
STDFONT
serif
sans-serif
monospace
新宋体
UnDotum
LMMonoLt10
Century Schoolbook L
OpenSymbol
Khmer OS System
LMSansQuot8
LMMathSymbols10
LMRomanSlant9
LMRomanSlant8
LMSans9
LMSans8
Mukti Narrow
Meera
Vemana2000
LMMonoSlant10
Umpush
Purisa
Pothana2000
DejaVu Sans Mono
Norasi
Loma
AR PL UKai TW MBE
URW Palladio L
Phetsarath OT
Sawasdee
Tlwg Typist
Lucida Bright
URW Gothic L
Dingbats
URW Chancery L
Ubuntu
FreeSerif
ori1Uni
文泉驿等宽微米黑
Kedage
DejaVu Sans
Kinnari
LMSans17
LMSans12
LMSans10
Lohit Punjabi
LMRoman17
LMRoman12
LMRoman10
TlwgMono
Symbol
楷体
LMRomanDunh10
LMRoman7
LMRoman6
LMRoman5
LMRoman9
LMRoman8
Bitstream Charter
KacstOne
AR PL UKai TW
Khmer OS
AR PL UKai HK
Liberation Mono
Courier 10 Pitch
AR PL UKai CN
文泉驿等宽正黑
Nimbus Sans L
TlwgTypewriter
TakaoPGothic
AR PL UMing TW MBE
LMRomanDemi10
Rachana
文泉驿微米黑
AR PL UMing HK
LMMonoCaps10
AR PL UMing CN
LMMonoLtCond10
AR PL UMing TW
Standard Symbols L
Lohit Gujarati
Nimbus Mono L
Nimbus Mono L
Lucida Sans
Liberation Serif
Mallige
LMMathItalic10
Nimbus Roman No9 L
LMMathItalic12
LMRomanUnsl10
Ubuntu
文泉驿点阵正黑
宋体
Lucida Sans Typewriter
Liberation Sans
LMMono10
Lucida Sans Typewriter
LMMono12
LMMathItalic7
LMMathItalic6
LMMathItalic5
LMMathItalic9
LMMathItalic8
Mukti Narrow
LMMathSymbols6
LMMathSymbols7
文泉驿正黑
LMMathSymbols5
FreeSans
LMMathSymbols8
LMMathSymbols9
LMMono8
LMMono9
LMMathExtension10
Lohit Tamil
Tlwg Typo
LMRomanCaps10
UnBatang
Lohit Bengali
LMSansDemiCond10
LMRomanSlant10
LMRomanSlant12
LMRomanSlant17
Waree
gargi
Lohit Hindi
DejaVu Serif
Saab
LMMonoProp10
SimSun-ExtB
Garuda
Rekha
WenQuanYi Bitmap Song
URW Bookman L
LMMonoPropLt10
FreeMono
Re: It's suspected to be an issue with Font. [message #32249 is a reply to message #32248] Tue, 03 May 2011 14:50 Go to previous messageGo to next message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
Above test was done on Ubuntu 11.04.

Will do a similar test on Windows as soon as I can. Thanks!
Re: It's suspected to be an issue with Font. [message #32250 is a reply to message #32249] Tue, 03 May 2011 14:52 Go to previous messageGo to previous message
Lance is currently offline  Lance
Messages: 351
Registered: March 2007
Senior Member
STDFONT
serif
sans-serif
monospace
新宋体         <<-----This is the SimSun I suppose, but somehow it
              <<      was skipped and the font below it was chosen.
Previous Topic: There are too short lines after synchronization. t files.
Next Topic: A proposal: Adding a Translation & Localization GUI to TheIDE
Goto Forum:
  


Current Time: Wed May 27 18:37:09 CEST 2020

Total time taken to generate the page: 0.01931 seconds