Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » U++ Library support » RichText,QTF,RTF... » Qtf and non UTF8 strings
Qtf and non UTF8 strings [message #45241] Fri, 16 October 2015 09:07 Go to next message
koldo is currently offline  koldo
Messages: 3432
Registered: August 2008
Senior Veteran
Hello all

Sometimes for example handling file names, it is necessary to show in QTF strings that have some characters that are not UTF8. Unfortunately the QTF output gets broken with an ugly bold non UTF8 error message.

Is it possible to transform a dubious string into a right UTF8 string by, for example, substituting wrong chars with a symbol (a square or other)?

Best regards
Iñaki

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45243 is a reply to message #45241] Fri, 16 October 2015 10:50 Go to previous messageGo to next message
forlano is currently offline  forlano
Messages: 1207
Registered: March 2006
Location: Italy
Senior Contributor
koldo wrote on Fri, 16 October 2015 09:07


Is it possible to transform a dubious string into a right UTF8 string by, for example, substituting wrong chars with a symbol (a square or other)?


Hello,

this is an annoying problem even with the file content. The proposed solution would be very nice and easily recover the error in the file.

Luigi

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45246 is a reply to message #45243] Fri, 16 October 2015 23:23 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3432
Registered: August 2008
Senior Veteran
index.php?t=getfile&id=4850&private=0
  • Attachment: window.jpg
    (Size: 15.57KB, Downloaded 762 times)


Best regards
Iñaki

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45249 is a reply to message #45246] Sun, 18 October 2015 11:20 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3432
Registered: August 2008
Senior Veteran
I mean that instead of output this ugly error message and truncate string, it could be preferable to output all string replacing strange characters with a symbol (a square or other). This could be done by an U++ function called ConvertToUf8() (or a better name Smile )

Best regards
Iñaki

[Updated on: Sun, 18 October 2015 11:21]

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45250 is a reply to message #45241] Sun, 18 October 2015 16:14 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 14255
Registered: November 2005
Ultimate Member
koldo wrote on Fri, 16 October 2015 09:07
Hello all

Sometimes for example handling file names, it is necessary to show in QTF strings that have some characters that are not UTF8. Unfortunately the QTF output gets broken with an ugly bold non UTF8 error message.


I really doubt the "necessary" part here...

Quote:

Is it possible to transform a dubious string into a right UTF8 string by, for example, substituting wrong chars with a symbol (a square or other)?


Anyway, this is perhaps possible. Still, I would say error should be displayed....

Mirek

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45251 is a reply to message #45250] Mon, 19 October 2015 08:31 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3432
Registered: August 2008
Senior Veteran
Quote:
Anyway, this is perhaps possible. Still, I would say error should be displayed....
Probably the final user wants to read a string as clear as possible, and s/he does not know about UTF8 or internal character codifications that are more devoted to programmer and debugging.
Anyway if it is necessary to have this UTF8 message, I would ask for a function to convert a string to a fully readable one so it would be as complete as possible for an end user.

Best regards
Iñaki

[Updated on: Mon, 19 October 2015 08:32]

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45252 is a reply to message #45251] Mon, 19 October 2015 08:44 Go to previous messageGo to next message
forlano is currently offline  forlano
Messages: 1207
Registered: March 2006
Location: Italy
Senior Contributor
koldo wrote on Mon, 19 October 2015 08:31
Probably the final user wants to read a string as clear as possible, and s/he does not know about UTF8 or internal character codifications that are more devoted to programmer and debugging.


I can support this point of view. Most of the users know nothing about UTF8. If the error appear in the content of a qtf file the visual effect is terrible (part of the file is shown and a part is missing or with table bad formatted). He simply go in panic and is not able to recover the problem (just change the bad character).

Luigi

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45253 is a reply to message #45251] Mon, 19 October 2015 13:08 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 14255
Registered: November 2005
Ultimate Member
koldo wrote on Mon, 19 October 2015 08:31
Quote:
Anyway, this is perhaps possible. Still, I would say error should be displayed....
Probably the final user wants to read a string as clear as possible, and s/he does not know about UTF8 or internal character codifications that are more devoted to programmer and debugging.
Anyway if it is necessary to have this UTF8 message, I would ask for a function to convert a string to a fully readable one so it would be as complete as possible for an end user.


User does not now about UTF8, but programmer does. IMO is it programmer's error to put non-UTF8 into QTF, something that should be fixed. Hence the error to alert the programmer.

Still, maybe I am missing context. Where that non-UTF8 string came from?

Mirek

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45254 is a reply to message #45253] Mon, 19 October 2015 15:48 Go to previous messageGo to next message
forlano is currently offline  forlano
Messages: 1207
Registered: March 2006
Location: Italy
Senior Contributor
mirek wrote on Mon, 19 October 2015 13:08

Still, maybe I am missing context. Where that non-UTF8 string came from?
Mirek


Good question. Even I do not understand how the problem is originated Smile .

Anyway, for my program, the user import data from his national database or from the file he got somewhere (for example the German database has some special characters that do not appear in the standard latin alphabet). It seems that the import went OK because I can see the character in an ArrayCtrl. However when the output is sent to a QTF file something goes wrong. Usually are accented characters.
Perhaps the problem is the internal conversion of the file where the data is taken. The problem do not occur if the special character is typed within the application and sent to qtf file.

If exists the possibility to intercept the error and substitute the unknown code with something else I would do it myself.

Luigi

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45255 is a reply to message #45254] Tue, 20 October 2015 10:42 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3432
Registered: August 2008
Senior Veteran
In my case the problem comes when reading file names in Windows. These files have accented names, and were copied first from Windows to Linux and then copied back to Windows. Windows does not complain and can open or copy these files without problems, but I cannot show its names using QTF.

Best regards
Iñaki

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45256 is a reply to message #45255] Tue, 20 October 2015 14:48 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 14255
Registered: November 2005
Ultimate Member
koldo wrote on Tue, 20 October 2015 10:42
In my case the problem comes when reading file names in Windows. These files have accented names, and were copied first from Windows to Linux and then copied back to Windows. Windows does not complain and can open or copy these files without problems, but I cannot show its names using QTF.


Now, there might be error in U++... If you are using U++ to read those file names, e.g. FindFile, and you have default charset (UTF-8), then you should already get them in UTF-8 (from FindFile).

Is this the case?

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45302 is a reply to message #45256] Thu, 29 October 2015 11:01 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3432
Registered: August 2008
Senior Veteran
Hello Mirek

Problem found. Simplest code:
Exclamation("00.-Qu├® es.pdf");
Exclamation(ToSystemCharset("00.-Qu├® es.pdf"));     // "Invalid UTF-8 sequence"
ToSystemCharset() is used as the program takes the file to compress it with 7z using Sys("7z.exe " + ToSystemCharset(fileName)).

Now in case of error the program would show in Exclamation() fileName instead of ToSystemCharset(fileName).

Thank you.

Best regards
Iñaki

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45343 is a reply to message #45302] Sat, 31 October 2015 15:06 Go to previous messageGo to next message
sergeynikitin is currently offline  sergeynikitin
Messages: 748
Registered: January 2008
Location: Moscow, Russia
Contributor

Try DeQtf() function before call.
for ex.:
instead
Exclamation("00.-Qu├® es.pdf");
put
Exclamation(DeQtf("00.-Qu├® es.pdf"));

SergeyNikitin<U++>( linux, wine )
{
    under( Ubuntu || Debian || Raspbian );
}

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45344 is a reply to message #45343] Sat, 31 October 2015 15:41 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 14255
Registered: November 2005
Ultimate Member
sergeynikitin wrote on Sat, 31 October 2015 15:06
Try DeQtf() function before call.
for ex.:
instead
Exclamation("00.-Qu├® es.pdf");
put
Exclamation(DeQtf("00.-Qu├® es.pdf"));


Well, that was not a cause here. However, speaking of DeQtf, in most cases you can use "\1" escape intead.

Exclamation("\1" + filename + "\1");

(In most case -> as long as you can be reasonably sure there is no \1 in the string... Smile

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45346 is a reply to message #45343] Sat, 31 October 2015 19:07 Go to previous messageGo to next message
koldo is currently offline  koldo
Messages: 3432
Registered: August 2008
Senior Veteran
sergeynikitin wrote on Sat, 31 October 2015 15:06
Try DeQtf() function before call.
for ex.:
instead
Exclamation("00.-Qu├® es.pdf");
put
Exclamation(DeQtf("00.-Qu├® es.pdf"));
Thank you ,however Exclamation("00.-Qu├® es.pdf") works very well.

The problem was with Exclamation(ToSystemCharset("00.-Qu├® es.pdf")).

Best regards
Iñaki

Report message to a moderator

Re: Qtf and non UTF8 strings [message #45347 is a reply to message #45344] Sat, 31 October 2015 19:10 Go to previous message
koldo is currently offline  koldo
Messages: 3432
Registered: August 2008
Senior Veteran
mirek wrote on Sat, 31 October 2015 15:41
sergeynikitin wrote on Sat, 31 October 2015 15:06
Try DeQtf() function before call.
for ex.:
instead
Exclamation("00.-Qu├® es.pdf");
put
Exclamation(DeQtf("00.-Qu├® es.pdf"));


Well, that was not a cause here. However, speaking of DeQtf, in most cases you can use "\1" escape intead.

Exclamation("\1" + filename + "\1");

(In most case -> as long as you can be reasonably sure there is no \1 in the string... Smile

Great ! Smile

"\1" + filename + "\1" is bullet proof.

Exclamation("\1" + ToSystemCharset("00.-Qu├® es.pdf") + "\1") works perfectly.

Best regards
Iñaki

Report message to a moderator

Previous Topic: FindReplaceTool with right mouse click
Next Topic: Only one RichEdit on one Form possible?
Goto Forum:
  

[ Syndicate this forum (XML) ] [ RSS ] [ PDF ]

Current Time: Fri Apr 25 11:54:36 CEST 2025

Total time taken to generate the page: 0.00619 seconds
.:: Contact :: Home ::.

Powered by: FUDforum 3.0.5.
Copyright ©2001-2013 FUDforum Bulletin Board Software