Main place for discussions about U++ cross platform development framework.
Hi, Here's a strange issue: When I render a map using Painter, sometimes some letter does not render at all at specific scales. Alternatively, as you can see in the attached picture, it may render at false scale: When rotating the map slightly, the same text renders correctly: ...Unless when the rotation is exactly 90, 180 or 270 degrees, which causes similar scaling error. Another variant is that the letter appears to be completely absent. BUT: when zooming the map, the letter renders perfectly OK. There is no difference if Painter is in MT or ST mode. Finally, this behavior persists for that run of the program only. If I close and restart the program, the same letter will likely render OK all the time, but there may (or may not) be another letter that suffers from the same problem. I think this is some sort of initialization problem with fonts... but who knows. Best regards, Tom
Hi, Here's some more. In this picture two of the letters have shrinked in the middle of a word: When zooming in, the text gets fixed: Instead of a testcase, here's a clip of the code I use for rendering the text with Painter: Font fnt(font); fnt.Height((int)floor(height_dp)).Bold(object.bold).Italic(object.italic==1).Underline(object.selected); Pointf p(project.Transform(object.x,object.y)); double rot=object.rotation+rotation; rot+=object.no_rotate?-GetViewRotation():0; painter.Begin(); painter.Translate(p); if(rot) painter.Rotate(-rot*DEGR2RAD); switch(object.italic){ case 2: painter.Transform(Xform2D::Sheer(-0.3)); break; case 3: painter.Transform(Xform2D::Sheer(0.2)); break; } Vector<String> lines=Split(text,10); for(int row=0;row<lines.GetCount();row++){ Size sz=GetTextSize(lines[row],fnt); Pointf delta(0,0); switch(object.attpoint){ case 1: // 1 = Top left break; case 2: // Top Center delta.x=-sz.cx/2; break; case 3: // Top Right delta.x=-sz.cx; break; case 4: // Middle Left delta.y=-sz.cy/2; break; case 5: // Middle Center delta.y=-sz.cy/2; delta.x=-sz.cx/2; break; case 6: // Middle Right delta.y=-sz.cy/2; delta.x=-sz.cx; break; default: case 7: // Bottom Left delta.y=-sz.cy; break; case 8: // Bottom Center delta.y=-sz.cy; delta.x=-sz.cx/2; break; case 9: // Bottom Right delta.y=-sz.cy; delta.x=-sz.cx; break; } delta.y+=row*height_dp*1.5; // sz.cy painter.Text(delta,lines[row],fnt); } if(edgecolor.a) painter.Stroke(height_dp*0.1,edgecolor); painter.Fill(textcolor); painter.End(); Best regards, Tom
I can see nothing obvious, but it feels like caching issue. Please try this: void ApproximateChar(LinearPathConsumer& t, Pointf at, int ch, Font fnt, double tolerance) { PAINTER_TIMING("ApproximateChar"); Value v; INTERLOCKED { PAINTER_TIMING("ApproximateChar::Fetch"); static LRUCache<Value, GlyphKey> cache; cache.Shrink(500000); sMakeGlyph h; h.gk.fnt = fnt; h.gk.chr = ch; h.gk.tolerance = tolerance; v = cache.Get(h); } #if 1 GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; PaintCharacter(chp, Pointf(0, 0), ch, fnt); Vector<float>& g = chp.glyph; #else const Vector<float>& g = ValueTo< Vector<float> >(v); #endif int i = 0; while(i < g.GetCount()) { Pointf p; p.x = g[i++]; if(p.x > 1e30) { p.x = g[i++]; p.y = g[i++]; t.Move(p + at); } else { PAINTER_TIMING("ApproximateChar::Line"); p.y = g[i++]; t.Line(p + at); } } } (This switches caching off.) Mirek
Hi, Thanks! I will check this too tomorrow! Anyway, if this does not sort out easily, I can store the problematic sequence in a Painting and serialize it in a file for you to check out. (The same procedure as last time, when there was a complex Painter issue.) Thanks, Tom
Tom1 wrote on Thu, 10 January 2019 17:26 Hi, Thanks! I will check this too tomorrow! Anyway, if this does not sort out easily, I can store the problematic sequence in a Painting and serialize it in a file for you to check out. (The same procedure as last time, when there was a complex Painter issue.) Thanks, Tom Yes, good idea. Also, if my code change does not solve the problem, try this: void BufferPainter::CharacterOp(const Pointf& p, int ch, Font fnt) { LLOG("@ CharacterOp " << p << ", " << ch << ", " << fnt); #if 1 DoMove0(); PaintCharacter(*this, p, ch, fnt); #else move = current = EndPoint(p, false); auto& m = PathAdd<CharData>(CHAR); m.p = EndPoint(p, false); m.ch = ch; m.fnt = fnt; path_info->ischar = true; EvenOdd(); #endif } (It is similar - removing caching - just on another level)
Hi Mirek, The change in ApproximateChar() appears to fix the issue. However, as can be expected, the rendering speed drops dramatically. I will next revert the ApproximateChar() changes and then test the BufferPainter::CharacterOp() change... Best regards, Tom
Hi, Also the change in BufferPainter::CharacterOp() seems to fix the issue. Similarly, the rendering speed drops. The special thing about these maps I'm rendering is that they have an extremely small change in the text rotation thorough out the map page as they have been transformed from one coordinate projection to another (Transverse Mercator to Mercator in this case) and this involves a change in text rotation by the amount of meridian convergence value. This boils down to just parts of a degree through out the entire map width. I do not know if the caching gets affected by this or not. I also generated a serialized painting, but it did not exhibit the issue. I think this is because the history for the cache is different and therefore the problem does not show up. Best regards, Tom
So I am looking into caching right now.. One possibility is that this is caused by font replacements. What is the Font use? If it is anything else than arial, does changing the font to Arial fix the issue? Mirek
The font was "font=Font(Font::SANSSERIF,12);" Changed to "font=Font(Font::ARIAL,12);", but nothing changes. (I think the enum makes these two the same.) BR, Tom
Could you add these DLOGs struct GlyphKey { Font fnt; int chr; double tolerance; bool operator==(const GlyphKey& b) const { DLOG("*** operator=="); DDUMP(fnt); DDUMP(b.fnt); DDUMP(chr); DDUMP(b.chr); return fnt == b.fnt && chr == b.chr && tolerance == b.tolerance; } unsigned GetHashValue() const { return CombineHash(fnt, chr, tolerance); } }; struct sMakeGlyph : LRUCache<Value, GlyphKey>::Maker { GlyphKey gk; GlyphKey Key() const { return gk; } int Make(Value& v) const { GlyphPainter gp; gp.move = gp.pos = Null; gp.tolerance = gk.tolerance; DLOG("*** Make"); DDUMP(GetHashValue(gk)); DDUMP(gk.fnt); DDUMP(gk.chr); PaintCharacter(gp, Pointf(0, 0), gk.chr, gk.fnt); int sz = gp.glyph.GetCount() * 4; v = RawPickToValue(pick(gp.glyph)); return sz; } }; void ApproximateChar(LinearPathConsumer& t, Pointf at, int ch, Font fnt, double tolerance) { PAINTER_TIMING("ApproximateChar"); Value v; INTERLOCKED { DLOG("==== ApproximateChar " << ch << " " << fnt); PAINTER_TIMING("ApproximateChar::Fetch"); static LRUCache<Value, GlyphKey> cache; cache.Shrink(500000); sMakeGlyph h; h.gk.fnt = fnt; h.gk.chr = ch; h.gk.tolerance = tolerance; DDUMP(GetHashValue(h.gk)); v = cache.Get(h); } #if 0 GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; PaintCharacter(chp, Pointf(0, 0), ch, fnt); const Vector<float>& g = chp.glyph; #else const Vector<float>& g = ValueTo< Vector<float> >(v); #endif int i = 0; while(i < g.GetCount()) { Pointf p; p.x = g[i++]; if(p.x > 1e30) { p.x = g[i++]; p.y = g[i++]; t.Move(p + at); } else { PAINTER_TIMING("ApproximateChar::Line"); p.y = g[i++]; t.Line(p + at); } } } then bring it into error state, send me the screenshot and .log? Thanks, Mirek
Hi, Here you go! Please find attached a the log, a screenshot and the serialized painting of the same. As you can see in the painting, the text should read 'Perttilä', not ' erttilä'. (You can view the painting using the PainterBench -tool.) BR, Tom PS: The DLOG and DDUMP cause errors in RELEASE mode, but not in DEBUG mode though: ----- Painter ( GUI PROTECT MSC17X64 MSC WIN32 ) (8 / 32) RenderChar.cpp C:\upp-12610\upp.src\uppsrc\Painter\RenderChar.cpp(96): error C2018: unknown character '0x40' C:\upp-12610\upp.src\uppsrc\Painter\RenderChar.cpp(97): error C2018: unknown character '0x40' C:\upp-12610\upp.src\uppsrc\Painter\RenderChar.cpp(98): error C2018: unknown character '0x40' C:\upp-12610\upp.src\uppsrc\Painter\RenderChar.cpp(99): error C2018: unknown character '0x40' C:\upp-12610\upp.src\uppsrc\Painter\RenderChar.cpp(100): error C2018: unknown character '0x40' C:\upp-12610\upp.src\uppsrc\Painter\RenderChar.cpp(116): error C2018: unknown character '0x40' C:\upp-12610\upp.src\uppsrc\Painter\RenderChar.cpp(117): error C2018: unknown character '0x40' C:\upp-12610\upp.src\uppsrc\Painter\RenderChar.cpp(118): error C2018: unknown character '0x40' C:\upp-12610\upp.src\uppsrc\Painter\RenderChar.cpp(119): error C2018: unknown character '0x40' C:\upp-12610\upp.src\uppsrc\Painter\RenderChar.cpp(132): error C2018: unknown character '0x40' C:\upp-12610\upp.src\uppsrc\Painter\RenderChar.cpp(140): error C2018: unknown character '0x40' Painter: 1 file(s) built in (0:00.90), 902 msecs / file, duration = 910 msecs, parallelization 0%
Tom1 wrote on Fri, 11 January 2019 11:46 Hi, PS: The DLOG and DDUMP cause errors in RELEASE mode, but not in DEBUG mode though: That is by design. It forces you to remove LOGs that you have inserted for debugging purposes. Not ideal, but better than to have logfile overcrowded with forgotten debugging info that you have needed 5 years ago
mirek wrote on Fri, 11 January 2019 13:10 Tom1 wrote on Fri, 11 January 2019 11:46 Hi, PS: The DLOG and DDUMP cause errors in RELEASE mode, but not in DEBUG mode though: That is by design. It forces you to remove LOGs that you have inserted for debugging purposes. Not ideal, but better than to have logfile overcrowded with forgotten debugging info that you have needed 5 years ago Good point!
No luck... Let us add more logging: struct GlyphKey { Font fnt; int chr; double tolerance; bool operator==(const GlyphKey& b) const { DLOG("*** operator=="); DDUMP(fnt); DDUMP(b.fnt); DDUMP(chr); DDUMP(b.chr); return fnt == b.fnt && chr == b.chr && tolerance == b.tolerance; } unsigned GetHashValue() const { return CombineHash(fnt, chr, tolerance); } }; struct sMakeGlyph : LRUCache<Value, GlyphKey>::Maker { GlyphKey gk; GlyphKey Key() const { return gk; } int Make(Value& v) const { GlyphPainter gp; gp.move = gp.pos = Null; gp.tolerance = gk.tolerance; DLOG("*** Make"); DDUMP(GetHashValue(gk)); DDUMP(gk.fnt); DDUMP(gk.chr); PaintCharacter(gp, Pointf(0, 0), gk.chr, gk.fnt); int sz = gp.glyph.GetCount() * 4; v = RawPickToValue(pick(gp.glyph)); return sz; } }; void ApproximateChar(LinearPathConsumer& t, Pointf at, int ch, Font fnt, double tolerance) { PAINTER_TIMING("ApproximateChar"); Value v; INTERLOCKED { DLOG("==== ApproximateChar " << ch << " " << fnt); PAINTER_TIMING("ApproximateChar::Fetch"); static LRUCache<Value, GlyphKey> cache; cache.Shrink(500000); sMakeGlyph h; h.gk.fnt = fnt; h.gk.chr = ch; h.gk.tolerance = tolerance; DDUMP(GetHashValue(h.gk)); v = cache.Get(h); DDUMP(ValueTo< Vector<float> >(v)); } #if 0 GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; PaintCharacter(chp, Pointf(0, 0), ch, fnt); const Vector<float>& g = chp.glyph; #else const Vector<float>& g = ValueTo< Vector<float> >(v); #endif int i = 0; while(i < g.GetCount()) { Pointf p; p.x = g[i++]; if(p.x > 1e30) { p.x = g[i++]; p.y = g[i++]; t.Move(p + at); } else { PAINTER_TIMING("ApproximateChar::Line"); p.y = g[i++]; t.Line(p + at); } } } the same (no need to send painting, just .log and tell me what letter is missing / wrong). Thanks, Mirek
Hi, Here's the new log. No images or paintings this time The failed letter was a capital 'U' this time (at false size). Best regards, Tom
Tom1 wrote on Fri, 11 January 2019 14:33 Hi, Interestingly it always has two operator== hash compares until a match is found. (I repeated the rendering multiple times to find the spot in log more easily.) Is this multiple hash match affecting the caching in some way? That is hashcode collision. Nothing to worry about. Quote: PS: When a wrong cache item is picked for a character, it may have wrong size and wrong Italic/Normal coding, but it always is the correct Wrong italic/normal bit is new - is that sure? Mirek
Hi, The first message in this thread portrays that: The 'Örviksudden' should have been in italic, but was instead shown with straight up 'Ö', if you look at it carefully. I have seen this in other tests too occasionally, so I can confirm it. Best regards, Tom
Still no clues... Anyway, let us modify logging a bit: void ApproximateChar(LinearPathConsumer& t, Pointf at, int ch, Font fnt, double tolerance) { PAINTER_TIMING("ApproximateChar"); Value v; INTERLOCKED { DLOG("==== ApproximateChar " << ch << " " << (char)ch << " " << fnt << ", tolerance: " << tolerance); PAINTER_TIMING("ApproximateChar::Fetch"); static LRUCache<Value, GlyphKey> cache; cache.Shrink(500000); sMakeGlyph h; h.gk.fnt = fnt; h.gk.chr = ch; h.gk.tolerance = tolerance; v = cache.Get(h); DDUMP(ValueTo< Vector<float> >(v)); #if 1 GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; PaintCharacter(chp, Pointf(0, 0), ch, fnt); DDUMP(chp.glyph); ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif } const Vector<float>& g = ValueTo< Vector<float> >(v); int i = 0; while(i < g.GetCount()) { Pointf p; p.x = g[i++]; if(p.x > 1e30) { p.x = g[i++]; p.y = g[i++]; t.Move(p + at); } else { PAINTER_TIMING("ApproximateChar::Line"); p.y = g[i++]; t.Line(p + at); } } } (If error is here, it should also assert....)
Tom1 wrote on Fri, 11 January 2019 15:33 Hi, The first message in this thread portrays that: The 'Örviksudden' should have been in italic, but was instead shown with straight up 'Ö', if you look at it carefully. I have seen this in other tests too occasionally, so I can confirm it. Well, alternative theory is that some letters are rendered with wrong transformation matrix... I guess random matrix would explain both wrong size and "unitalicizing"..... Anyway, it would probably also create some more profound artefacts that you have not observed yet (?) Mirek
mirek wrote on Fri, 11 January 2019 16:43 Still no clues... Anyway, let us modify logging a bit: void ApproximateChar(LinearPathConsumer& t, Pointf at, int ch, Font fnt, double tolerance) { PAINTER_TIMING("ApproximateChar"); Value v; INTERLOCKED { DLOG("==== ApproximateChar " << ch << " " << (char)ch << " " << fnt << ", tolerance: " << tolerance); PAINTER_TIMING("ApproximateChar::Fetch"); static LRUCache<Value, GlyphKey> cache; cache.Shrink(500000); sMakeGlyph h; h.gk.fnt = fnt; h.gk.chr = ch; h.gk.tolerance = tolerance; v = cache.Get(h); DDUMP(ValueTo< Vector<float> >(v)); #if 1 GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; PaintCharacter(chp, Pointf(0, 0), ch, fnt); DDUMP(chp.glyph); ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif } const Vector<float>& g = ValueTo< Vector<float> >(v); int i = 0; while(i < g.GetCount()) { Pointf p; p.x = g[i++]; if(p.x > 1e30) { p.x = g[i++]; p.y = g[i++]; t.Move(p + at); } else { PAINTER_TIMING("ApproximateChar::Line"); p.y = g[i++]; t.Line(p + at); } } } (If error is here, it should also assert....) Strangely enough, this did not exhibit the problem at all. There are no buggy letters, as far as I could tell from looking around the map for 30 minutes. No asserts either, but that is to be expected as the problem did not surface here... I have been running with MSBT17x64 but I must next look at 32-bit compiler to see if it behaves differently. And maybe test on Linux too to see if the behavior changes. I will continue with this on Monday... As for the transformation matrix, I cannot see a chance for it to change between successive letters. That would require another thread to break the matrix, but this issue happens in ST too, so I think it is next to impossible. Anyway, thanks for your efforts on this so far and have a nice weekend! Tom
IDK, I see no issues and as the last try I a have adapted our benchmarking snippet to test zooming in hope that it will reproduce the caching error: #include <CtrlLib/CtrlLib.h> #include <Painter/Painter.h> using namespace Upp; class PainterText : public TopWindow { public: Painting p; FileSel fs; BufferPainter bpainter; double scale = 2; void Open(){ if(fs.ExecuteOpen("Select a painting to view")){ p.Clear(); p.Serialize(FileIn(fs.Get())); } } virtual void MouseWheel(Point p, int zdelta, dword keyflags) { if(zdelta < 0) scale *= 0.8; else scale /= 0.8; Refresh(); } virtual bool Key(dword key, int count){ Refresh(); switch(key){ case K_CTRL_O: Open(); return true; } return false; } typedef PainterText CLASSNAME; PainterText(){ Sizeable(); p.Serialize(FileIn("C:/xxx/PainteTest/T5.painting")); } virtual void Paint(Draw &draw){ ImageBuffer ib(GetSize()); { bpainter.Create(ib); bpainter.Co(false); bpainter.PreClipDashed(); bpainter.Clear(White()); bpainter.EvenOdd(); bpainter.Scale(scale); bpainter.Paint(p); bpainter.Finish(); } SetSurface(draw,Rect(ib.GetSize()),ib,ib.GetSize(),Point(0,0)); } }; GUI_APP_MAIN { PainterText().Run(); } but nothing.... Is not it possible that some new addition to your code is overwritting the cachec? Some dangling pointer perhaps? Mirek
Tom1 wrote on Fri, 11 January 2019 18:50 mirek wrote on Fri, 11 January 2019 16:43 #if 1 GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; PaintCharacter(chp, Pointf(0, 0), ch, fnt); DDUMP(chp.glyph); ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif (If error is here, it should also assert....) Strangely enough, this did not exhibit the problem at all. There are no buggy letters, as far as I could tell from looking around the map for 30 minutes. No asserts either, but that is to be expected as the problem did not surface here... Well, that is something we can work with... First of all, it would be nice to make sure this really makes differences. Change to #if 0 to resolve this. If this is positive, I would set back to #if 1, retest, if problem is gone, then start removing lines of testing code from the last one until the problem reappears. That should give us some clue... Mirek
Hi, Yes, I will follow your instructions... starting on Monday. The absolute worst case scenario for me is if I have any pointer issues and corruption of memory thereby. Late last night I started to think that I may have turned off PROTECT flag simultaneously when switching again to DEBUG mode for the test. This may have contributed to disappearance of the problem too. This is something I will have to check first thing on Monday. I mean the effect of working with and without PROTECT. Anyway, I will keep digging. And I wish to thank you for your effort on this. (Hope I have not wasted your time with some stupid mistake of my own.) Thanks and best regards, Tom
Hi Mirek, OK, here's the situation. Using PROTECT or not does not have any effect on this issue. Neither does the compiler; MSBT17 and MSBT17x64 both work the same. After starting to comment out the lines from the bottom of the sequence, PaintCharacter() proved to be the line required for correct operation. Commenting it out it brought the problem back: #if 1 GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; PaintCharacter(chp, Pointf(0, 0), ch, fnt); // <<-- Required for correct rendering // DDUMP(chp.glyph); // ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif Any suggestions, how I should proceed? Best regards, Tom
Tom1 wrote on Mon, 14 January 2019 09:58 Hi Mirek, OK, here's the situation. Using PROTECT or not does not have any effect on this issue. Neither does the compiler; MSBT17 and MSBT17x64 both work the same. After starting to comment out the lines from the bottom of the sequence, PaintCharacter() proved to be the line required for correct operation. Commenting it out it brought the problem back: #if 1 GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; PaintCharacter(chp, Pointf(0, 0), ch, fnt); // <<-- Required for correct rendering // DDUMP(chp.glyph); // ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif Any suggestions, how I should proceed? Best regards, Tom Try this: #ifdef 1 DLOG("==== ApproximateChar " << ch << " " << (char)ch << " " << fnt << ", tolerance: " << tolerance); DDUMP(ValueTo< Vector<float> >(v)); GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; extern HFONT GetWin32Font(Font fnt, int angle); GetWin32Font(fnt, 0); // PaintCharacter(chp, Pointf(0, 0), ch, fnt); // DDUMP(chp.glyph); // ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif GetWin32Font is called by PaintCharacter and has some caching inside too, maybe that is the one that is causing the trouble. Of course, if calling it fixes the problem, try to comment and uncomment...
mirek wrote on Mon, 14 January 2019 11:13 ... Try this: #ifdef 1 DLOG("==== ApproximateChar " << ch << " " << (char)ch << " " << fnt << ", tolerance: " << tolerance); DDUMP(ValueTo< Vector<float> >(v)); GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; extern HFONT GetWin32Font(Font fnt, int angle); GetWin32Font(fnt, 0); // PaintCharacter(chp, Pointf(0, 0), ch, fnt); // DDUMP(chp.glyph); // ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif GetWin32Font is called by PaintCharacter and has some caching inside too, maybe that is the one that is causing the trouble. Of course, if calling it fixes the problem, try to comment and uncomment... This does not fix the issue. BR, Tom
Tom1 wrote on Mon, 14 January 2019 10:18 mirek wrote on Mon, 14 January 2019 11:13 ... Try this: #ifdef 1 DLOG("==== ApproximateChar " << ch << " " << (char)ch << " " << fnt << ", tolerance: " << tolerance); DDUMP(ValueTo< Vector<float> >(v)); GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; extern HFONT GetWin32Font(Font fnt, int angle); GetWin32Font(fnt, 0); // PaintCharacter(chp, Pointf(0, 0), ch, fnt); // DDUMP(chp.glyph); // ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif GetWin32Font is called by PaintCharacter and has some caching inside too, maybe that is the one that is causing the trouble. Of course, if calling it fixes the problem, try to comment and uncomment... This does not fix the issue. BR, Tom What happens if you do void PaintCharacter(Painter& sw, const Pointf& p, int chr, Font font) { GlyphInfo gi = GetGlyphInfo(font, chr); PaintCharPath pw; pw.sw = &sw; if(gi.IsNormal()) font.Render(pw, p.x, p.y, chr); /* else if(gi.IsReplaced()) { Font fnt = font; fnt.Face(gi.lspc); fnt.Height(gi.rspc); fnt.Render(pw, p.x, p.y + font.GetAscent() - fnt.GetAscent(), chr); } else if(gi.IsComposed()) { ComposedGlyph cg; Compose(font, chr, cg); font.Render(pw, p.x, p.y, cg.basic_char); sw.Div(); cg.mark_font.Render(pw, p.x + cg.mark_pos.x, p.y + cg.mark_pos.y, cg.mark_char); }*/ sw.EvenOdd(true); } This should have no impact at all; we are testing branches that do character replacements if characters are missing. But I guess it is worth checking (as everything else fails).
Another idea to test: #ifdef 1 DLOG("==== ApproximateChar " << ch << " " << (char)ch << " " << fnt << ", tolerance: " << tolerance); DDUMP(ValueTo< Vector<float> >(v)); GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; GetGlyphInfo(fnt, ch); // GetWin32Font(fnt, 0); // PaintCharacter(chp, Pointf(0, 0), ch, fnt); DDUMP(chp.glyph); ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif Does this fix the issue? (GetGlyphInfo is another caching function called from PaintCharacter). Mirek
Yes, this does not have any effect what so ever. (I mean the Painter.cpp :: PaintCharacter() change.) BR, Tom
mirek wrote on Mon, 14 January 2019 11:28 Another idea to test: #ifdef 1 DLOG("==== ApproximateChar " << ch << " " << (char)ch << " " << fnt << ", tolerance: " << tolerance); DDUMP(ValueTo< Vector<float> >(v)); GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; GetGlyphInfo(fnt, ch); // GetWin32Font(fnt, 0); // PaintCharacter(chp, Pointf(0, 0), ch, fnt); DDUMP(chp.glyph); ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif Does this fix the issue? (GetGlyphInfo is another caching function called from PaintCharacter). Mirek This does not fix the issue either. (Additionally I had to comment out the ASSERT, as the chp.glyph is not initialized to compare correctly with v.) BR, Tom
What about this: #ifdef 1 DLOG("==== ApproximateChar " << ch << " " << (char)ch << " " << fnt << ", tolerance: " << tolerance); DDUMP(ValueTo< Vector<float> >(v)); GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; struct PaintCharPath : FontGlyphConsumer { Painter *sw; virtual void Move(Pointf p) { sw->Move(p); } virtual void Line(Pointf p) { sw->Line(p); } virtual void Quadratic(Pointf p1, Pointf p2) { sw->Quadratic(p1, p2); } virtual void Cubic(Pointf p1, Pointf p2, Pointf p3) { sw->Cubic(p1, p2, p3); } virtual void Close() { sw->Close(); } } pw; pw.sw = &chp; fnt.Render(pw, 0, 0, ch); // PaintCharacter(chp, Pointf(0, 0), ch, fnt); // DDUMP(chp.glyph); // ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif (We are trying to identify which part of PaintCharacter makes the difference...)
Tried. Now the error appears if "fnt.Render(pw, 0, 0, ch);" is commented out from the above. BR, Tom
Tom1 wrote on Mon, 14 January 2019 11:01 Tried. Now the error appears if "fnt.Render(pw, 0, 0, ch);" is commented out from the above. BR, Tom Getting closer, but still no clue.... #ifdef 1 DLOG("==== ApproximateChar " << ch << " " << (char)ch << " " << fnt << ", tolerance: " << tolerance); DDUMP(ValueTo< Vector<float> >(v)); GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; struct PaintCharPath : FontGlyphConsumer { Painter *sw; virtual void Move(Pointf p) { sw->Move(p); } virtual void Line(Pointf p) { sw->Line(p); } virtual void Quadratic(Pointf p1, Pointf p2) { sw->Quadratic(p1, p2); } virtual void Cubic(Pointf p1, Pointf p2, Pointf p3) { sw->Cubic(p1, p2, p3); } virtual void Close() { sw->Close(); } } pw; pw.sw = &chp; void RenderCharacterSys(FontGlyphConsumer& sw, double x, double y, int ch, Font fnt); RenderCharacterSys(pw, 0, 0, ch, fnt); // fnt.Render(pw, 0, 0, ch); // PaintCharacter(chp, Pointf(0, 0), ch, fnt); // DDUMP(chp.glyph); // ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif Now if RenderCharacterSys here is our 'toggle', I suggest start commenting out its internals to see what really makes the difference... Mirek
Oh, sorry about the part about "commenting out" - that would not work (because we are using it for the desired rendering as well). What we need to do is to copy it and THEN start removing... double fx_to_dbl(const FIXED& p); Pointf fx_to_dbl(const Pointf& pp, const POINTFX& p); struct sMakeGlyph : LRUCache<Value, GlyphKey>::Maker { GlyphKey gk; GlyphKey Key() const { return gk; } int Make(Value& v) const { GlyphPainter gp; gp.move = gp.pos = Null; gp.tolerance = gk.tolerance; PaintCharacter(gp, Pointf(0, 0), gk.chr, gk.fnt); int sz = gp.glyph.GetCount() * 4; v = RawPickToValue(pick(gp.glyph)); return sz; } }; void RenderCharPath2(const char* gbuf, unsigned total_size, FontGlyphConsumer& sw, double xx, double yy) { const char* cur_glyph = gbuf; const char* end_glyph = gbuf + total_size; Pointf pp(xx, yy); while(cur_glyph < end_glyph) { const TTPOLYGONHEADER* th = (TTPOLYGONHEADER*)cur_glyph; const char* end_poly = cur_glyph + th->cb; const char* cur_poly = cur_glyph + sizeof(TTPOLYGONHEADER); sw.Move(fx_to_dbl(pp, th->pfxStart)); while(cur_poly < end_poly) { const TTPOLYCURVE* pc = (const TTPOLYCURVE*)cur_poly; if (pc->wType == TT_PRIM_LINE) for(int i = 0; i < pc->cpfx; i++) sw.Line(fx_to_dbl(pp, pc->apfx[i])); if (pc->wType == TT_PRIM_QSPLINE) for(int u = 0; u < pc->cpfx - 1; u++) { Pointf b = fx_to_dbl(pp, pc->apfx[u]); Pointf c = fx_to_dbl(pp, pc->apfx[u + 1]); if (u < pc->cpfx - 2) c = Mid(b, c); sw.Quadratic(b, c); } cur_poly += sizeof(WORD) * 2 + sizeof(POINTFX) * pc->cpfx; } sw.Close(); cur_glyph += th->cb; } } HFONT GetWin32Font(Font fnt, int angle); HDC Win32_IC(); void RenderCharacterSys2(FontGlyphConsumer& sw, double x, double y, int ch, Font fnt) { HFONT hfont = GetWin32Font(fnt, 0); if(hfont) { HDC hdc = Win32_IC(); HFONT ohfont = (HFONT) ::SelectObject(hdc, hfont); GLYPHMETRICS gm; MAT2 m_matrix; memset(&m_matrix, 0, sizeof(m_matrix)); m_matrix.eM11.value = 1; m_matrix.eM22.value = 1; int gsz = GetGlyphOutlineW(hdc, ch, GGO_NATIVE, &gm, 0, NULL, &m_matrix); if(gsz < 0) return; StringBuffer gb(gsz); gsz = GetGlyphOutlineW(hdc, ch, GGO_NATIVE, &gm, gsz, ~gb, &m_matrix); if(gsz < 0) return; RenderCharPath2(~gb, gsz, sw, x, y + fnt.GetAscent()); ::SelectObject(hdc, ohfont); } } void ApproximateChar(LinearPathConsumer& t, Pointf at, int ch, Font fnt, double tolerance) { PAINTER_TIMING("ApproximateChar"); Value v; INTERLOCKED { PAINTER_TIMING("ApproximateChar::Fetch"); static LRUCache<Value, GlyphKey> cache; cache.Shrink(500000); sMakeGlyph h; h.gk.fnt = fnt; h.gk.chr = ch; h.gk.tolerance = tolerance; v = cache.Get(h); #ifdef _DEBUG DLOG("==== ApproximateChar " << ch << " " << (char)ch << " " << fnt << ", tolerance: " << tolerance); DDUMP(ValueTo< Vector<float> >(v)); GlyphPainter chp; chp.move = chp.pos = Null; chp.tolerance = tolerance; struct PaintCharPath : FontGlyphConsumer { Painter *sw; virtual void Move(Pointf p) { sw->Move(p); } virtual void Line(Pointf p) { sw->Line(p); } virtual void Quadratic(Pointf p1, Pointf p2) { sw->Quadratic(p1, p2); } virtual void Cubic(Pointf p1, Pointf p2, Pointf p3) { sw->Cubic(p1, p2, p3); } virtual void Close() { sw->Close(); } } pw; pw.sw = &chp; RenderCharacterSys2(pw, 0, 0, ch, fnt); // fnt.Render(pw, 0, 0, ch); // PaintCharacter(chp, Pointf(0, 0), ch, fnt); // DDUMP(chp.glyph); // ASSERT(ValueTo< Vector<float> >(v) == chp.glyph); #endif } const Vector<float>& g = ValueTo< Vector<float> >(v); int i = 0; while(i < g.GetCount()) { Pointf p; p.x = g[i++]; if(p.x > 1e30) { p.x = g[i++]; p.y = g[i++]; t.Move(p + at); } else { PAINTER_TIMING("ApproximateChar::Line"); p.y = g[i++]; t.Line(p + at); } } } It this still 'toggles', start commenting from RenderCharPath2 call up...
Commenting out this breaks rendering: // RenderCharPath2(~gb, gsz, sw, x, y + fnt.GetAscent()); I'll try to chop that one to slices next... BR, Tom
Also try RenderCharPath2(~gb, gsz, sw, x, y/* + fnt.GetAscent()*/);
Thinking about it, I have suspiction that maybe RawPickToValue might have to do something with all that... What about this: struct sMakeGlyph : LRUCache<String, GlyphKey>::Maker { GlyphKey gk; GlyphKey Key() const { return gk; } int Make(String& v) const { GlyphPainter gp; gp.move = gp.pos = Null; gp.tolerance = gk.tolerance; PaintCharacter(gp, Pointf(0, 0), gk.chr, gk.fnt); int sz = gp.glyph.GetCount() * 4; v.Set((char *)gp.glyph.begin(), sizeof(float) * gp.glyph.GetCount()); return sz; } }; void ApproximateChar(LinearPathConsumer& t, Pointf at, int ch, Font fnt, double tolerance) { PAINTER_TIMING("ApproximateChar"); String v; INTERLOCKED { PAINTER_TIMING("ApproximateChar::Fetch"); static LRUCache<String, GlyphKey> cache; cache.Shrink(500000); sMakeGlyph h; h.gk.fnt = fnt; h.gk.chr = ch; h.gk.tolerance = tolerance; v = cache.Get(h); } int i = 0; int count = v.GetCount() / sizeof(float); const float *g = (const float *)~v; while(i < count) { Pointf p; p.x = g[i++]; if(p.x > 1e30) { p.x = g[i++]; p.y = g[i++]; t.Move(p + at); } else { PAINTER_TIMING("ApproximateChar::Line"); p.y = g[i++]; t.Line(p + at); } } }
mirek wrote on Mon, 14 January 2019 12:32 Also try RenderCharPath2(~gb, gsz, sw, x, y/* + fnt.GetAscent()*/); This does not make any difference. Tom
This shows the problem: void RenderCharPath2(const char* gbuf, unsigned total_size, FontGlyphConsumer& sw, double xx, double yy) { const char* cur_glyph = gbuf; const char* end_glyph = gbuf + total_size; Pointf pp(xx, yy); while(cur_glyph < end_glyph) { const TTPOLYGONHEADER* th = (TTPOLYGONHEADER*)cur_glyph; const char* end_poly = cur_glyph + th->cb; const char* cur_poly = cur_glyph + sizeof(TTPOLYGONHEADER); sw.Move(fx_to_dbl(pp, th->pfxStart)); /* while(cur_poly < end_poly) { const TTPOLYCURVE* pc = (const TTPOLYCURVE*)cur_poly; if (pc->wType == TT_PRIM_LINE) for(int i = 0; i < pc->cpfx; i++) sw.Line(fx_to_dbl(pp, pc->apfx[i])); if (pc->wType == TT_PRIM_QSPLINE) for(int u = 0; u < pc->cpfx - 1; u++) { Pointf b = fx_to_dbl(pp, pc->apfx[u]); Pointf c = fx_to_dbl(pp, pc->apfx[u + 1]); if (u < pc->cpfx - 2) c = Mid(b, c); sw.Quadratic(b, c); } cur_poly += sizeof(WORD) * 2 + sizeof(POINTFX) * pc->cpfx; } */ sw.Close(); cur_glyph += th->cb; } } This renders OK: void RenderCharPath2(const char* gbuf, unsigned total_size, FontGlyphConsumer& sw, double xx, double yy) { const char* cur_glyph = gbuf; const char* end_glyph = gbuf + total_size; Pointf pp(xx, yy); while(cur_glyph < end_glyph) { const TTPOLYGONHEADER* th = (TTPOLYGONHEADER*)cur_glyph; const char* end_poly = cur_glyph + th->cb; const char* cur_poly = cur_glyph + sizeof(TTPOLYGONHEADER); sw.Move(fx_to_dbl(pp, th->pfxStart)); while(cur_poly < end_poly) { const TTPOLYCURVE* pc = (const TTPOLYCURVE*)cur_poly; /* if (pc->wType == TT_PRIM_LINE) for(int i = 0; i < pc->cpfx; i++) sw.Line(fx_to_dbl(pp, pc->apfx[i])); if (pc->wType == TT_PRIM_QSPLINE) for(int u = 0; u < pc->cpfx - 1; u++) { Pointf b = fx_to_dbl(pp, pc->apfx[u]); Pointf c = fx_to_dbl(pp, pc->apfx[u + 1]); if (u < pc->cpfx - 2) c = Mid(b, c); sw.Quadratic(b, c); } */ cur_poly += sizeof(WORD) * 2 + sizeof(POINTFX) * pc->cpfx; } sw.Close(); cur_glyph += th->cb; } } I will now check the next suggestion with raw pick. BR, Tom
Tom1 wrote on Mon, 14 January 2019 11:52 This shows the problem: BR, Tom Now that is crazy, is not it? Must be some overlooked undefined behaviour somewhere.
mirek wrote on Mon, 14 January 2019 12:41 Thinking about it, I have suspiction that maybe RawPickToValue might have to do something with all that... What about this: struct sMakeGlyph : LRUCache<String, GlyphKey>::Maker { GlyphKey gk; GlyphKey Key() const { return gk; } int Make(String& v) const { GlyphPainter gp; gp.move = gp.pos = Null; gp.tolerance = gk.tolerance; PaintCharacter(gp, Pointf(0, 0), gk.chr, gk.fnt); int sz = gp.glyph.GetCount() * 4; v.Set((char *)gp.glyph.begin(), sizeof(float) * gp.glyph.GetCount()); return sz; } }; void ApproximateChar(LinearPathConsumer& t, Pointf at, int ch, Font fnt, double tolerance) { PAINTER_TIMING("ApproximateChar"); String v; INTERLOCKED { PAINTER_TIMING("ApproximateChar::Fetch"); static LRUCache<String, GlyphKey> cache; cache.Shrink(500000); sMakeGlyph h; h.gk.fnt = fnt; h.gk.chr = ch; h.gk.tolerance = tolerance; v = cache.Get(h); } int i = 0; int count = v.GetCount() / sizeof(float); const float *g = (const float *)~v; while(i < count) { Pointf p; p.x = g[i++]; if(p.x > 1e30) { p.x = g[i++]; p.y = g[i++]; t.Move(p + at); } else { PAINTER_TIMING("ApproximateChar::Line"); p.y = g[i++]; t.Line(p + at); } } } The problem is still there with the above code. BR, Tom
Are you 100% sure that there is no other thread running using Painter (perhaps painting something entirely different) or maybe even Draw or Font? No background threads? Mirek
Have you tried this on another computer? (Blaming HW is the last thing I usually do, but this is way too weird...) Mirek
As the difference IMO amounts to some waste CPU cycles only, maybe we could try to insert some small delay instead of RenderCharacterSys2....
According to Windows Task Manager, there are 30 threads running on this process. But my GUI is simply running on main thread. There should be nothing graphical going on in any of my background threads. Here's an interesting part: I said previously that ST/MT in Painter did not make any difference. Well, it did not when I had started the program originally with Painter MT enabled and then switched 'live' to ST to test for the error effect. BUT: Now I started with Painter fixed in ST and: Error is gone... Sounds like in my original MT/ST test the font caching in Painter had already happened in MT before I switched it to ST. I need to dig deeper on this. BR, Tom
mirek wrote on Mon, 14 January 2019 13:14 Have you tried this on another computer? (Blaming HW is the last thing I usually do, but this is way too weird...) Mirek Need to try that too.
Tom1 wrote on Mon, 14 January 2019 12:21 According to Windows Task Manager, there are 30 threads running on this process. But my GUI is simply running on main thread. There should be nothing graphical going on in any of my background threads. Here's an interesting part: I said previously that ST/MT in Painter did not make any difference. Well, it did not when I had started the program originally with Painter MT enabled and then switched 'live' to ST to test for the error effect. BUT: Now I started with Painter fixed in ST and: Error is gone... Sounds like in my original MT/ST test the font caching in Painter had already happened in MT before I switched it to ST. I need to dig deeper on this. BR, Tom Well, that makes better sense - as the only possible difference there seem to be the delay, this would point to some sort of race condition. Probably the wrong glyph was cached BEFORE ST was activated.... Maybe there is a problem with INTERLOCKED. Can you try to replace it with explicit mutex? Mirek
Replaced INTERLOCKED with StaticMutex, but no change... I Will continue testing. BR, Tom
Try this pls: Buffer<ClippingLine> BufferPainter::RenderPath(double width, Event<One<SpanSource>&> ss, const RGBA& color) { PAINTER_TIMING("RenderPath"); Buffer<ClippingLine> newclip; if(width == FILL) Close(); current = Null; if(width == 0 || !ss && color.a == 0 && width >= FILL) return newclip; if(pathattr.mtx_serial != preclip_mtx_serial) { if(dopreclip) { Pointf tl, br, a; Xform2D imx = Inverse(pathattr.mtx); tl = br = imx.Transform(0, 0); a = imx.Transform(size.cx, 0); tl = min(a, tl); br = max(a, br); a = imx.Transform(0, size.cy); tl = min(a, tl); br = max(a, br); a = imx.Transform(size.cx, size.cy); tl = min(a, tl); br = max(a, br); preclip = Rectf(tl, br); } else preclip = Null; regular = pathattr.mtx.IsRegular(); preclip_mtx_serial = pathattr.mtx_serial; } if(co) { if(width >= FILL && !ss && !alt && findarg(mode, MODE_ANTIALIASED, MODE_SUBPIXEL) >= 0 && !path_info->ischar) { (The difference is && !path_info->ischar in the last line). Also, is there a difference DEBUG vs RELEASE?
mirek wrote on Mon, 14 January 2019 13:14 Have you tried this on another computer? (Blaming HW is the last thing I usually do, but this is way too weird...) Mirek Tried with my old reliable notebook, but it's the same. BR, Tom
mirek wrote on Mon, 14 January 2019 14:23 Try this pls: Buffer<ClippingLine> BufferPainter::RenderPath(double width, Event<One<SpanSource>&> ss, const RGBA& color) { PAINTER_TIMING("RenderPath"); Buffer<ClippingLine> newclip; if(width == FILL) Close(); current = Null; if(width == 0 || !ss && color.a == 0 && width >= FILL) return newclip; if(pathattr.mtx_serial != preclip_mtx_serial) { if(dopreclip) { Pointf tl, br, a; Xform2D imx = Inverse(pathattr.mtx); tl = br = imx.Transform(0, 0); a = imx.Transform(size.cx, 0); tl = min(a, tl); br = max(a, br); a = imx.Transform(0, size.cy); tl = min(a, tl); br = max(a, br); a = imx.Transform(size.cx, size.cy); tl = min(a, tl); br = max(a, br); preclip = Rectf(tl, br); } else preclip = Null; regular = pathattr.mtx.IsRegular(); preclip_mtx_serial = pathattr.mtx_serial; } if(co) { if(width >= FILL && !ss && !alt && findarg(mode, MODE_ANTIALIASED, MODE_SUBPIXEL) >= 0 && !path_info->ischar) { (The difference is && !path_info->ischar in the last line). Also, is there a difference DEBUG vs RELEASE? In RELEASE mode I could not find a single bad letter in 15 minutes. I think this is it! Will try now with DEBUG... BR, Tom
Not a single false character in DEBUG mode either. I should have found something already if the problem still existed. So, is this the fix or means for problem isolation? Best regards, Tom
Tom1 wrote on Mon, 14 January 2019 14:11 Not a single false character in DEBUG mode either. I should have found something already if the problem still existed. So, is this the fix or means for problem isolation? Best regards, Tom This basically disables MT for character rendering... So for now, please revert it and try two separate things for me: void Painter::TextOp(const Pointf& p, const wchar *text, Font fnt, int n, const double *dx) { if(n == 0) { Move(0, 0); return; } FontInfo fi = fnt.Info(); double x = p.x; while(n) { int ch = *text++; Character(x, p.y, ch, fnt); // Div(); if(dx) x += *dx++; else x += fi[ch]; n--; } if(fnt.IsUnderline() || fnt.IsStrikeout()) { double a = fnt.GetAscent(); double cy = max(a / 16, 1.0); double cx = x - p.x; if(fnt.IsUnderline()) Rectangle(p.x, p.y + a + cy, cx, cy); if(fnt.IsStrikeout()) Rectangle(p.x, p.y + 2 * a / 3, cx, cy); } } (This might lead to some new artifacts where characters overlap, that is expected, look for the original problem (wrong characters). void BufferPainter::RenderPathSegments(LinearPathConsumer *g, const Vector<byte>& path, const SimpleAttr *attr, double tolerance) { Pointf pos = Pointf(0, 0); const byte *data = path.begin(); const byte *end = path.end(); while(data < end) { const LinearData *d = (LinearData *)data; switch(d->type) { case MOVE: { g->Move(pos = attr ? attr->mtx.Transform(d->p) : d->p); data += sizeof(LinearData); break; } case LINE: { PAINTER_TIMING("LINE"); g->Line(pos = attr ? attr->mtx.Transform(d->p) : d->p); data += sizeof(LinearData); break; } case QUADRATIC: { PAINTER_TIMING("QUADRATIC"); const QuadraticData *d = (QuadraticData *)data; if(attr) { Pointf p = attr->mtx.Transform(d->p); ApproximateQuadratic(*g, pos, attr->mtx.Transform(d->p1), p, tolerance); pos = p; } else { ApproximateQuadratic(*g, pos, d->p1, d->p, tolerance); pos = d->p; } data += sizeof(QuadraticData); break; } case CUBIC: { PAINTER_TIMING("CUBIC"); const CubicData *d = (CubicData *)data; if(attr) { Pointf p = attr->mtx.Transform(d->p); ApproximateCubic(*g, pos, attr->mtx.Transform(d->p1), attr->mtx.Transform(d->p2), p, tolerance); pos = p; } else { ApproximateCubic(*g, pos, d->p1, d->p2, d->p, tolerance); pos = d->p; } data += sizeof(CubicData); break; } case CHAR: { const CharData *ch = (CharData *)data; INTERLOCKED ApproximateChar(*g, ch->p, ch->ch, ch->fnt, tolerance); data += sizeof(CharData); break; } default: NEVER(); g->End(); return; } } g->End(); } (Additional mutex in for CHAR). Thanks, Mirek
Neither of the above help... false letters can still be found. BR, Tom Update: I see you updated something in posting. I need to re-test.
OK, now it looks like "void Painter::TextOp(const Pointf& p, const wchar *text, Font fnt, int n, const double *dx)" -fix would correct the issue. The other does not have any effect. BR, Tom
No, false positive. The problem is still there even after the "void Painter::TextOp(const Pointf& p, const wchar *text, Font fnt, int n, const double *dx)" -fix. BR, Tom
Tom1 wrote on Mon, 14 January 2019 15:38 No, false positive. The problem is still there even after the "void Painter::TextOp(const Pointf& p, const wchar *text, Font fnt, int n, const double *dx)" -fix. BR, Tom Well, for today I am out of ideas.... Will continue tomorrow, OK? Perhaps one last question: This probably must have started recently, probably after the last round of optimizations. Is that correct? Mirek
Hi, Yes, I'm also out of time for today. Tomorrow I will have time until about 14:00 EET. The first time I noticed this was just last week when I reported it. I had not updated from SVN in a while before that though, so I may need to go back a bit further in time to find that. One potential reason for finding this issue is that I had just recently reached a point in my own code where I started to rotate the texts according to the original scale and rotation, both of which change constantly over the projection surface for maps where a change from one map projection to another has been carried out. In effect this has caused an overwhelming increase in cached fonts here in my application. I think the reason is that when tolerance changes even by the smallest fraction of a double, it necessitates a new cache entry. In my case there will be a huge amount of cache entries. Actually, if the tolerance parameter were made more discrete, e.g. through classing to slots for caching, the problem might not show up at all or at least do it extremely rarely. Then again, I think, the potential issue with caching should be corrected first, if real, and only thereafter should the optimization through classing be implemented. I might have it all wrong here, but this is just a thought. Anyway, Mirek, thanks for your efforts today on the task! Best regards, Tom
Tom1 wrote on Mon, 14 January 2019 17:21 One potential reason for finding this issue is that I had just recently reached a point in my own code where I started to rotate the texts according to the original scale and rotation, both of which change constantly over the projection surface for maps where a change from one map projection to another has been carried out. In effect this has caused an overwhelming increase in cached fonts here in my application. Rotated texts in Painter are rotated by Painter matrix, they are cached in zero angle position... Anyway, good info, I will think about this all... Mirek
mirek wrote on Mon, 14 January 2019 18:45 Tom1 wrote on Mon, 14 January 2019 17:21 One potential reason for finding this issue is that I had just recently reached a point in my own code where I started to rotate the texts according to the original scale and rotation, both of which change constantly over the projection surface for maps where a change from one map projection to another has been carried out. In effect this has caused an overwhelming increase in cached fonts here in my application. Rotated texts in Painter are rotated by Painter matrix, they are cached in zero angle position... Anyway, good info, I will think about this all... Mirek Well, yes, they are indeed cached in zero angle position. I should have expressed myself more clearly to make the point: Both fine scaling and fine rotation changes ultimately have an effect on double tolerance value solved using the Painter matrix scale and stored as caching key. This is also why rotating the image sometimes corrected the problem. Please see my early screenshots in this thread. (The changes would have been just at a magnitude of rounding errors i.e. just some LSBs.) BR, Tom
Tom1 wrote on Mon, 14 January 2019 18:31 mirek wrote on Mon, 14 January 2019 18:45 Tom1 wrote on Mon, 14 January 2019 17:21 One potential reason for finding this issue is that I had just recently reached a point in my own code where I started to rotate the texts according to the original scale and rotation, both of which change constantly over the projection surface for maps where a change from one map projection to another has been carried out. In effect this has caused an overwhelming increase in cached fonts here in my application. Rotated texts in Painter are rotated by Painter matrix, they are cached in zero angle position... Anyway, good info, I will think about this all... Mirek Well, yes, they are indeed cached in zero angle position. I should have expressed myself more clearly to make the point: Both fine scaling and fine rotation changes ultimately have an effect on double tolerance value solved using the Painter matrix scale and stored as caching key. This is also why rotating the image sometimes corrected the problem. Please see my early screenshots in this thread. (The changes would have been just at a magnitude of rounding errors i.e. just some LSBs.) BR, Tom Well, yes, thats something I have missed so far. This is true and actually something to fix (probably round tolerance to like 10%).
EDIT: Perhaps check the possible fix first - the last post "OK, now I see possible bug, hard to say it is related, but please test:" and do testing only if it fails. To test: void BufferPainter::FinishPathJob() { if(jobcount == 0) return; /* CoWork co; co * [&] { for(;;) { int i = co.Next(); if(i >= jobcount) break; CoJob& b = cojob[i]; b.rasterizer.Reset(); PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, b.regular); if(!j.preclipped) { b.evenodd = j.evenodd; BufferPainter::RenderPathSegments(j.g, b.path_info->path[b.subpath], j.regular ? &b.attr : NULL, j.tolerance); } } };*/ for(int i = 0; i < jobcount; i++) { CoJob& b = cojob[i]; b.rasterizer.Reset(); PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, b.regular); if(!j.preclipped) { b.evenodd = j.evenodd; BufferPainter::RenderPathSegments(j.g, b.path_info->path[b.subpath], j.regular ? &b.attr : NULL, j.tolerance); } } FinishFillJob(); fillcount = jobcount; Swap(cofill, cojob); // Swap to keep allocated rasters (instead of pick) (Changing the path rendering to run in ST)
Worth testing how much tolerance has to do with it: BufferPainter::PathJob::PathJob(Rasterizer& rasterizer, double width, const PathInfo *path_info, const SimpleAttr& attr, const Rectf& preclip, bool isregular) : trans(attr.mtx) { evenodd = attr.evenodd; regular = isregular && width < 0 && !path_info->ischar; g = &rasterizer; if(!IsNull(preclip.left) && !path_info->ischar) { double ex = max(width, 0.0) * (1 + attr.miter_limit); if(path_info->path_max.y + ex < preclip.top || path_info->path_min.y - ex > preclip.bottom || path_info->path_max.x + ex < preclip.left || path_info->path_min.x - ex > preclip.right) { preclipped = true; return; } } preclipped = false; if(regular) tolerance = 0.3; else { trans.target = g; g = &trans; tolerance = 0.3 / attr.mtx.GetScale(); } tolerance = 0.3; // add this...
OK, now I see possible bug, hard to say it is related, but please test: void BufferPainter::FinishPathJob() { if(jobcount == 0) return; { CoWork co; co * [&] { for(;;) { int i = co.Next(); if(i >= jobcount) break; CoJob& b = cojob[i]; b.rasterizer.Reset(); PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, b.regular); if(!j.preclipped) { b.evenodd = j.evenodd; BufferPainter::RenderPathSegments(j.g, b.path_info->path[b.subpath], j.regular ? &b.attr : NULL, j.tolerance); } } }; } FinishFillJob(); (The difference is that CoWork is now inside block).
mirek wrote on Mon, 14 January 2019 21:56 OK, now I see possible bug, hard to say it is related, but please test: void BufferPainter::FinishPathJob() { if(jobcount == 0) return; { CoWork co; co * [&] { for(;;) { int i = co.Next(); if(i >= jobcount) break; CoJob& b = cojob[i]; b.rasterizer.Reset(); PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, b.regular); if(!j.preclipped) { b.evenodd = j.evenodd; BufferPainter::RenderPathSegments(j.g, b.path_info->path[b.subpath], j.regular ? &b.attr : NULL, j.tolerance); } } }; } FinishFillJob(); (The difference is that CoWork is now inside block). Good morning, This does not help. The problem is still there. BR, Tom
mirek wrote on Mon, 14 January 2019 20:53 EDIT: Perhaps check the possible fix first - the last post "OK, now I see possible bug, hard to say it is related, but please test:" and do testing only if it fails. To test: void BufferPainter::FinishPathJob() { if(jobcount == 0) return; /* CoWork co; co * [&] { for(;;) { int i = co.Next(); if(i >= jobcount) break; CoJob& b = cojob[i]; b.rasterizer.Reset(); PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, b.regular); if(!j.preclipped) { b.evenodd = j.evenodd; BufferPainter::RenderPathSegments(j.g, b.path_info->path[b.subpath], j.regular ? &b.attr : NULL, j.tolerance); } } };*/ for(int i = 0; i < jobcount; i++) { CoJob& b = cojob[i]; b.rasterizer.Reset(); PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, b.regular); if(!j.preclipped) { b.evenodd = j.evenodd; BufferPainter::RenderPathSegments(j.g, b.path_info->path[b.subpath], j.regular ? &b.attr : NULL, j.tolerance); } } FinishFillJob(); fillcount = jobcount; Swap(cofill, cojob); // Swap to keep allocated rasters (instead of pick) (Changing the path rendering to run in ST) This fixes the issue. I do not know if this has anything to do with it, but I have noticed that I cannot pass reference variables as parameters to CoWork::Do() -called functions. Compiler does not complain, but the code simply does not work correctly. I have had to switch to using pointers as parameters instead of reference variables in such functions. BR, Tom
Hi, As for forcing "tolerance = 0.3;" likely decreases the number of cache entries to a fraction of the original in this case, the likelihood to see any issues drops equally. And so it seems: I cannot see a single erroneous character. Then again, I can check about 1000 names in a reasonable time on screen and if the problem is there, I will see just a few errors. I think that it becomes just a few errors in a million now, so I'm not 'lucky enough' to find them. BR, Tom
Tom1 wrote on Tue, 15 January 2019 08:53 I do not know if this has anything to do with it, but I have noticed that I cannot pass reference variables as parameters to CoWork::Do() -called functions. Compiler does not complain, but the code simply does not work correctly. I have had to switch to using pointers as parameters instead of reference variables in such functions. That should not be issue here. Now all depends on details, but you always have to be careful that referenced variable exists until CoWork Finish or destructor... If you think that you have non-working example of different nature, please start a new thread a post it... Mirek
Tom1 wrote on Tue, 15 January 2019 09:11 Hi, As for forcing "tolerance = 0.3;" likely decreases the number of cache entries to a fraction of the original in this case, the likelihood to see any issues drops equally. And so it seems: I cannot see a single erroneous character. Then again, I can check about 1000 names in a reasonable time on screen and if the problem is there, I will see just a few errors. I think that it becomes just a few errors in a million now, so I'm not 'lucky enough' to find them. BR, Tom Understood, I mainly just wanted to know if the hypothesis is correct... (Revert the change, later I will introduce a nice simple fix to round tolerance, but we need to find the bug first). Mirek
I am also trying to reproduce the issue here; anyway the paintings I have here only has a couple o texts in it. Now I am not sure you have any level-of-detail systeme there, but if not, would it be possible to send me extremely zoomed out painting file so that it has more texts in it? (That said, when testing, do you start with such zoomed-out situation?)
void BufferPainter::FinishPathJob() { if(jobcount == 0) return; CoWork co; co * [&] { for(;;) { int i = co.Next(); if(i >= jobcount) break; INTERLOCKED { CoJob& b = cojob[i]; b.rasterizer.Reset(); PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, b.regular); if(!j.preclipped) { b.evenodd = j.evenodd; BufferPainter::RenderPathSegments(j.g, b.path_info->path[b.subpath], j.regular ? &b.attr : NULL, j.tolerance); } } } }; If problem disappears, try to move INTERLOCKED down line by line (until if) to see when/if it reappears.
mirek wrote on Tue, 15 January 2019 10:15 Tom1 wrote on Tue, 15 January 2019 08:53 I do not know if this has anything to do with it, but I have noticed that I cannot pass reference variables as parameters to CoWork::Do() -called functions. Compiler does not complain, but the code simply does not work correctly. I have had to switch to using pointers as parameters instead of reference variables in such functions. That should not be issue here. Now all depends on details, but you always have to be careful that referenced variable exists until CoWork Finish or destructor... If you think that you have non-working example of different nature, please start a new thread a post it... Mirek OK, will do when I have some time and can recall the accurate details of my problem with it to create a testcase. BR, Tom
mirek wrote on Tue, 15 January 2019 10:23 void BufferPainter::FinishPathJob() { if(jobcount == 0) return; CoWork co; co * [&] { for(;;) { int i = co.Next(); if(i >= jobcount) break; INTERLOCKED { CoJob& b = cojob[i]; b.rasterizer.Reset(); PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, b.regular); if(!j.preclipped) { b.evenodd = j.evenodd; BufferPainter::RenderPathSegments(j.g, b.path_info->path[b.subpath], j.regular ? &b.attr : NULL, j.tolerance); } } } }; If problem disappears, try to move INTERLOCKED down line by line (until if) to see when/if it reappears. This does not help. BR, Tom
What about locking the whole lambda body? Also to test in the same loop: PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, false); (change the last parameter to 'false')
mirek wrote on Tue, 15 January 2019 10:21 I am also trying to reproduce the issue here; anyway the paintings I have here only has a couple o texts in it. Now I am not sure you have any level-of-detail systeme there, but if not, would it be possible to send me extremely zoomed out painting file so that it has more texts in it? (That said, when testing, do you start with such zoomed-out situation?) I do have a sort of level-of-detail system here, but I can squeeze quite a bunch of details on one UHD view still using smaller than readable fonts. You can then scale the painting to a readable level. Anyway, can you PM me your e-mail address, so that I can send you the painting. (It's still over 7MB when compressed with 7-zip at ultra level.) BR, Tom
mirek wrote on Tue, 15 January 2019 10:38 What about locking the whole lambda body? Also to test in the same loop: PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, false); (change the last parameter to 'false') void BufferPainter::FinishPathJob() { if(jobcount == 0) return; CoWork co; co * [&] { INTERLOCKED { for(;;) { int i = co.Next(); if(i >= jobcount) break; CoJob& b = cojob[i]; b.rasterizer.Reset(); PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, b.regular); if(!j.preclipped) { b.evenodd = j.evenodd; BufferPainter::RenderPathSegments(j.g, b.path_info->path[b.subpath], j.regular ? &b.attr : NULL, j.tolerance); } } } }; This works OK. I will now test with 'false' instead of 'b.regular'. BR, Tom
BTW, here is my testing code where I am trying to reproduce the issue: #include <CtrlLib/CtrlLib.h> #include <Painter/Painter.h> using namespace Upp; class PainterText : public TopWindow { public: Painting p; FileSel fs; BufferPainter bpainter; double scale = 0.4; int rotation = 0; Pointf translate = Pointf(0, 0); Pointf start; void Open(){ if(fs.ExecuteOpen("Select a painting to view")) { p.Clear(); p.Serialize(FileIn(fs.Get())); } } virtual void LeftDown(Point p, dword) { start = (Pointf)p * scale; SetCapture(); } virtual void MouseMove(Point p, dword keyflags) { if(HasCapture()) { Pointf pos = (Pointf)p * scale; translate += (pos - start); start = pos; Refresh(); } } virtual void MouseWheel(Point p, int zdelta, dword keyflags) { if(keyflags & K_CTRL) rotation += sgn(zdelta); else { if(zdelta < 0) scale *= 0.9; else scale /= 0.9; } Refresh(); } virtual bool Key(dword key, int count){ Refresh(); switch(key){ case K_CTRL_O: Open(); return true; } return false; } typedef PainterText CLASSNAME; PainterText(){ Sizeable(); p.Serialize(FileIn("C:/xxx/PainteTest/T5.painting")); } virtual void Paint(Draw &draw){ ImageBuffer ib(GetSize()); { bpainter.Create(ib); bpainter.Co(true); bpainter.PreClipDashed(); bpainter.Clear(White()); bpainter.EvenOdd(); bpainter.Scale(scale); bpainter.Rotate(rotation * .1); bpainter.Translate(translate); bpainter.Paint(p); bpainter.Finish(); } SetSurface(draw,Rect(ib.GetSize()),ib,ib.GetSize(),Point(0,0)); } }; GUI_APP_MAIN { PainterText().Run(); } Perhaps you could try with the painting file you have produced - this is to ensure that I am doing the same thing...
mirek wrote on Tue, 15 January 2019 10:38 What about locking the whole lambda body? Also to test in the same loop: PathJob j(b.rasterizer, b.width, b.path_info, b.attr, b.preclip, false); (change the last parameter to 'false') Changing the last parameter to 'false' does not work. BR, Tom
CharEntry fc_cache[512]; GlyphInfo GetGlyphInfo(Font font, int chr) { INTERLOCKED { font.RealizeStd(); unsigned hash = GlyphHash(font, chr); CharEntry& e = fc_cache[hash & 511]; if(e.font != font.AsInt64() || e.chr != chr) e = GetGlyphEntry(font, chr, hash); return e.info; } } (removal of thread__ for fc_cache)
FontEntry fi_cache[63]; const CommonFontInfo& GetFontInfo(Font font) { INTERLOCKED { font.RealizeStd(); unsigned hash = FoldHash(font.GetHashValue()) % 63; FontEntry& e = fi_cache[hash]; if(e.font != font.AsInt64()) { Mutex::Lock __(sFontLock); e.font = font.AsInt64(); e.info = GetFontInfoSys(font); } return e.info; } } (Removal of thread__ for fi_cache)
Hi, Removal of thread__ for fc_cache and fi_cache do not help. The issue is still there. - As for the testcode, I could not spot any false characters there, but the granularity due to significant zooming ruins part of the larger texts. BR, Tom
That testing code I have posted works better with virtual void LeftDown(Point p, dword) { start = (Pointf)p / scale; SetCapture(); } virtual void MouseMove(Point p, dword keyflags) { if(HasCapture()) { Pointf pos = (Pointf)p / scale; translate += (pos - start); start = pos; Refresh(); } }
Yes And: PainterText().MinimizeBox().MaximizeBox().Sizeable().Run(); Is nice to have too to fill the screen. But I still can't reproduce the issue with this. One thing that caught my attention, though, is that the large bold upright texts show up extremely coarse, which is kind of expected due to the original zoom level fitting it all in one UHD screen. But why on earth are those smaller non-bold italic / upright fonts perfectly rendered although they are even smaller in the original view? BR, Tom
Yes, I have noticed it too. I am not so sure original zoom level has anything to do with that though... (it should get new tolerance value)
As to this issue, is it possible that all other texts are using your custom "Sheer" italic? It actually looks like setting tolerance to 0.3 is not a good idea... EDIT: tolerance is ok
Is it by any means possible that missing letter are from those 'ugly' texts?
mirek wrote on Tue, 15 January 2019 12:23 As to this issue, is it possible that all other texts are using your custom "Sheer" italic? It actually looks like setting tolerance to 0.3 is not a good idea... EDIT: tolerance is ok My custom reverse italic (using sheer to lean back) is using standard upright font and the normal italic is using standard italic served by Painter (as is without custom sheer). The ugly characters appear when the font is bold. The names of lakes and other water areas (light blue) are using this reverse italic. Smaller place names are in plain italic. Larger place names are in straight-up bold. Municipalities are in even larger straight-up capital non-bold letters. Best regards, Tom
Interesting, I have tried to hunt down 'ugly text issue' like this: void Painter::TextOp(const Pointf& p, const wchar *text, Font fnt, int n, const double *dx) { DDUMP(WString(text, n)); And interestingly, it looks like those are not painted as Text... or at least not stored into painting file as Text. Is that somehow possible that you have some separate path for them?
mirek wrote on Tue, 15 January 2019 12:27 Is it by any means possible that missing letter are from those 'ugly' texts? I do not know about the missing letters, but the wrong size / wrong italicizing letters tend to come at same size and italicizing as many of the other texts that already occur on the view. BR, Tom
I have 1 h 8'... Anyway, I need to pull back that 'Bold' from some place names. E.g. 'Paloniemi', which is rendered ugly, is just slightly larger straight up text than the more common place names which are italic... not even bold! Best regards, Tom
Tom1 wrote on Tue, 15 January 2019 12:54 I have 1 h 8'... Anyway, I need to pull back that 'Bold' from some place names. E.g. 'Paloniemi', which is rendered ugly, is just slightly larger straight up text than the more common place names which are italic... not even bold! Best regards, Tom OK, now I know: There is no separate path for those ugly texts. But I suppose the font size causes some conditional conversion to polygons in Painting. I forced all fonts to be of height 12, and then applied suitable Scale to Painter. As a result all the fonts were (ugly) rendered with precision that was insufficient when zooming in. Then I did this using height 75 and again suitable scale to painter. This time all the texts rendered beautifully in Painting. So I guess the storage format and precision in Painting is dependent on font size. BR, Tom EDIT: I changed font height to 1 and tried again: This time all fonts rendered perfectly!! But, the inter-character spacing precision in words was terrible. EDIT: I think I understand why I cannot send the problem in a painting to you: The Painting is generated ST and characters are partly rendered already, so the error will not show up. I must generate a file with only unrendered characters, which get rendered at your end only.
Tom1 wrote on Tue, 15 January 2019 11:19 Hi, Removal of thread__ for fc_cache and fi_cache do not help. The issue is still there. - As for the testcode, I could not spot any false characters there, but the granularity due to significant zooming ruins part of the larger texts. BR, Tom Hi! It looks like this change of removing thread__ for fc_cache and fi_cache did not get compiled for me as I was simultaneously working on the testcode. Now I recompiled all and tried this once more and BANG! The problem was gone from my own program. And I also got it back by adding the thread__ ahead of those two caches. I think this solved the issue! I must fly now... Will be back for more testing tomorrow. A million thanks and best regards, Tom
Tom1 wrote on Tue, 15 January 2019 12:27 So I guess the storage format and precision in Painting is dependent on font size. Except it is not, AFAIK... Besides, in this situation you are just using PaintingPainter, which really just directly stores Painter commands (including TextOp and CharacterOp). And the problem is that those ugly texts get stored as curves. Maybe I am blind, but I do not see a code path that could lead to THAT. Mirek
Tom1 wrote on Tue, 15 January 2019 13:28 Tom1 wrote on Tue, 15 January 2019 11:19 Hi, Removal of thread__ for fc_cache and fi_cache do not help. The issue is still there. - As for the testcode, I could not spot any false characters there, but the granularity due to significant zooming ruins part of the larger texts. BR, Tom Hi! It looks like this change of removing thread__ for fc_cache and fi_cache did not get compiled for me as I was simultaneously working on the testcode. Now I recompiled all and tried this once more and BANG! The problem was gone from my own program. And I also got it back by adding the thread__ ahead of those two caches. I think this solved the issue! I must fly now... Will be back for more testing tomorrow. A million thanks and best regards, Tom Obviously the next step is to try whether it is caused by first, second or both... Mirek
Upon further investigation we can forget about ugly texts. Most likely these are just results of font hinting process in Win32. (As to what got me so confused, those texts are so mangled that I misread the name and then failed to find it in the list...) EDIT: Hinting confirmed. I have added |GGO_UNHINTED to the GetGlyphOutlineW call and problem disappeared... (now in trunk).
Ok, I will try both one at a time tomorrow. BR, Tom
Good morning Mirek, Removing "thread__" from "thread__ CharEntry fc_cache[512];" fixes it. Removing "thread__" from "thread__ FontEntry fi_cache[63];" does not have any effect on the problem. Best regards, Tom BTW: Why should the caches be thread_local anyway? Isn't it true that Painter will use the same fonts with a whole bunch of threads when painting in MT? Having them common for all threads would have the used fonts cached after first hit and available to all threads. (Well, I'm likely missing something important here, but you'll probably educate me on that shortly...)
Tom1 wrote on Wed, 16 January 2019 08:54 Good morning Mirek, Removing "thread__" from "thread__ CharEntry fc_cache[512];" fixes it. Removing "thread__" from "thread__ FontEntry fi_cache[63];" does not have any effect on the problem. Best regards, Tom BTW: Why should the caches be thread_local anyway? Isn't it true that Painter will use the same fonts with a whole bunch of threads when painting in MT? Having them common for all threads would have the used fonts cached after first hit and available to all threads. (Well, I'm likely missing something important here, but you'll probably educate me on that shortly...) Only to avoid confusion, how the code for CharEntry exactly looks now for you? And again, the rest of the code is absolutely the same as trunk? Mirek