Overview
Examples
Screenshots
Comparisons
Applications
Download
Documentation
Tutorials
Bazaar
Status & Roadmap
FAQ
Authors & License
Forums
Funding Ultimate++
Search on this site
Search in forums












SourceForge.net Logo
Home » Developing U++ » U++ Developers corner » Optimizing DrawImage across platforms
Optimizing DrawImage across platforms [message #53611] Wed, 15 April 2020 09:52 Go to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
Hi,

There is an issue with ViewDraw when using the GTK3 backend on Linux. The following testcase works correctly on Windows (GUI) and Linux (GUI X11) but not on Linux (GUI) using GTK3. The cross should follow the cursor on the white background, but on GTK3 the entire window goes black and only the area updated with ViewDraw is valid afterwards.

#include <CtrlLib/CtrlLib.h>

using namespace Upp;

class Testcase5 : public TopWindow {
public:
	typedef Testcase5 CLASSNAME;
	
	Rect loc;
	
	Testcase5(){
		loc.SetNull();
	}

	void Paint(Draw &w){
		w.DrawRect(Rect(GetSize()),White());
		loc.SetNull();
	}
	
	void MouseMove(Point p, dword keyflags){
		ImageBuffer sb(32,32);
		BufferPainter sbp(sb);
		sbp.Clear(RGBAZero());
		sbp.Move(0,0).Line(31,31);
		sbp.Move(0,31).Line(31,0);
		sbp.Stroke(5,Gray());
		
		ViewDraw draw(this);
		if(!loc.IsNullInstance()) draw.DrawRect(loc,White());
		loc.Set(p.x-16,p.y-16,p.x+16,p.y+16);
		draw.DrawImage(loc.left,loc.top,Image(sb));
	}
};


GUI_APP_MAIN
{
	Testcase5().Run();
}


Best regards,

Tom
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53615 is a reply to message #53611] Wed, 15 April 2020 14:35 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 14154
Registered: November 2005
Ultimate Member
Well, this is what is possible with gtk3. Let us say that what happens to areas that you do not paint to is implementation defined.

If this is a showstopper, I could probably add ViewDraw constructor that actually specifies the area of Ctrl that you wish to repaint...

Mirek
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53616 is a reply to message #53615] Wed, 15 April 2020 15:15 Go to previous messageGo to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
Hi Mirek,

Well, yes.. I will have some small targets shown on a map display and updated between once and five times per second depending on their positioning rate. It would be terrible waste to update the entire window area each time any of the targets move.

So, if it is not too much work, I would certainly prefer a way to update just a small rectangle within the Ctrl. Painting the entire map possibly tens of times per second is just too much.

I assume, this ViewDraw constructor will then apply to all backends?

Thanks and best regards,

Tom
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53621 is a reply to message #53616] Wed, 15 April 2020 22:21 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 14154
Registered: November 2005
Ultimate Member
So I have got both good news and bad news...

It is done for X11, Gtk, Win32.

However, while trying to do that on MacOS, I have found that even existing ViewDraw stopped working. Unfortunately, it seems like MacOS issue, critical method is broken, as other people found too:

https://cgit.freedesktop.org/libreoffice/core/commit/?id=959 e8ae7ea33ce94dd80ee8ea172b6db64593873

Given this, it actually seems impossible to implement ViewDraw for MacOS... It looks like we should actually deprecate it as well; simply use Refresh and optimized Paint....

Mirek
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53630 is a reply to message #53621] Thu, 16 April 2020 21:08 Go to previous messageGo to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
Hi Mirek,

Thank you very much for solving this issue for me! Smile

I did some further tuning of the render/paint structure using the new ViewDraw and ended up with a nice target update time of 60-150 micro seconds on both Windows and Linux GTK3 with the following code:

#include <CtrlLib/CtrlLib.h>

using namespace Upp;

class Testcase5 : public TopWindow {
public:
	typedef Testcase5 CLASSNAME;
	
	Rect loc;
	
	ImageBuffer map;
	
	Testcase5(){
		loc.SetNull();
	}

	void Paint(Draw &w){
		int64 t0=usecs();
		
		SetSurface(w,Rect(map.GetSize()),map,map.GetSize(),Point(0,0));
		loc.SetNull();
		
		int64 t1=usecs();
		Title(Format("Paint took %lld us",t1-t0));
	}
	
	void Layout(){
		Size sz=GetSize();
		map.Create(sz);
		BufferPainter bp(map);
		bp.RectPath(Rect(sz));
		bp.Fill(Pointf(0,0),Green(),Pointf(sz.cx-1,sz.cy-1),Yellow());
		bp.Move(0,0).Line(Pointf(sz.cx-1,sz.cy-1)).Stroke(1,Black());
		bp.Move(0,sz.cy-1).Line(Pointf(sz.cx-1,0)).Stroke(1,Black());
	}
	
	void MouseMove(Point p, dword keyflags){
		int64 t0=usecs();
		
		Rect area;
		area.SetNull();
		if(!loc.IsNullInstance()) area.Union(loc);
		loc.Set(p.x-16,p.y-16,p.x+16,p.y+16);
		area.Union(loc);
		area.Inflate(2);
		area.Intersect(Rect(GetSize()));
		
		ImageBuffer sb(area.GetSize());
		for(int y=0;y<area.Height();y++) memcpy(sb[y],&map[y+area.top][area.left],sizeof(RGBA)*area.Width());

		BufferPainter sbp(sb);
		sbp.Translate(loc.left-area.left,loc.top-area.top);
		sbp.Move(0,0).Line(31,31);
		sbp.Move(0,31).Line(31,0);
		sbp.Stroke(5,Black());
		
		ViewDraw draw(this,area);
		SetSurface(draw,area.GetSize(),sb,sb.GetSize(),Point(0,0));
		
		int64 t1=usecs();
		Title(Format("Move took %lld us",t1-t0));
		
	}
};


GUI_APP_MAIN
{
	Testcase5().Sizeable().MaximizeBox().MinimizeBox().Run();
}


I do not need support for this feature in Mac, but what worries me is the potential deprecation of ViewDraw... Is there a real risk that you would drop ViewDraw entirely? As you can see from running the example above, an optimized Paint (as above, drawing a readily rendered ImageBuffer with SetSurface()) of the entire view on a 4K screen takes something like 6-12 ms, the small update with ViewDraw runs at around just 1 % of that time. (That's the difference between success and failure for my task.)

How would similar code and its performance look with what you refer to as 'Refresh and optimized Paint'?

Thanks and best regards,

Tom
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53631 is a reply to message #53630] Thu, 16 April 2020 22:06 Go to previous messageGo to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
Hi,

Just noticed a related issue: RectTracker (using ViewDraw) is not working in GTK3, while it is OK in X11 and Win32. This is also one important feature I need in my application. (On top of that I also use an inherited LineTracker to visualize drawing of straight lines.)

Best regards,

Tom
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53633 is a reply to message #53630] Fri, 17 April 2020 00:41 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 14154
Registered: November 2005
Ultimate Member
Tom1 wrote on Thu, 16 April 2020 21:08
How would similar code and its performance look with what you refer to as 'Refresh and optimized Paint'?


So I guess we are in similar situation, painting maps and complex polygon. What we actually do is that we paint the map to Image, then in Paint we just put this image on screen and all "indicators" paint over that map.

Mirek
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53635 is a reply to message #53633] Fri, 17 April 2020 13:20 Go to previous messageGo to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
mirek wrote on Fri, 17 April 2020 01:41
Tom1 wrote on Thu, 16 April 2020 21:08
How would similar code and its performance look with what you refer to as 'Refresh and optimized Paint'?


So I guess we are in similar situation, painting maps and complex polygon. What we actually do is that we paint the map to Image, then in Paint we just put this image on screen and all "indicators" paint over that map.

Mirek


Well, so it seems. This basically sounds the same what I'm doing. My chain is vector map > Painter > ImageBuffer > SetSurface. And then the small moving targets or overlay texts/objects/indicators on top of that with a higher update rate. Just like my example above. The situation reminds me of sprites in the old times, which would fit in nicely. Smile

The cost of 6-12 ms per 4K screen update (using SetSurface) for just a tiny little change is too much in my opinion. The new small-area ViewDraw based approach is far more efficient and should be here to stay even if MacOS does not support it at the moment -- or ever.

Another approach could possibly be to have 'Refresh(Rect) and Paint(Rect)/Paint(Vector<Rect>)' routine variants for partial updates via Paint. But I'm not sure if this solves the RectTracker requirements. And I would still prefer the new partial ViewDraw for the purpose...

Yet another question is, if OpenGL, Direct2D/3D, or some other backend would offer faster SetSurface capability taking the cost of 4K screen update well below millisecond level. Then again, I would like to avoid that path for now to ensure best compatibility with existing low-end hardware the clients may have.

Best regards,

Tom
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53636 is a reply to message #53635] Fri, 17 April 2020 13:38 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 14154
Registered: November 2005
Ultimate Member
Well, I think the difference might be that SetSurface is fine as long as you use it just once.

For this scenario, converting to Image first is better option, because you will repaint that one many times. Converting to Image basically means that the data gets transfered to GPU and the next time you paint it it goes from GPU memory.
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53637 is a reply to message #53636] Fri, 17 April 2020 14:36 Go to previous messageGo to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
Hi Mirek,

		int64 t0=usecs();
		w.DrawImage(0,0,image);
		int64 t1=usecs();
		Title(Format("Paint took %lld us",t1-t0));


In Windows this takes about 21 milliseconds in the first run and about the same 6.. milliseconds in the following runs as with SetSurface steadily.

In Linux with GTK3, the DrawImage() and SetSurface() both perform equally at around 12 milliseconds.

HOWEVER, Linux with X11 re-using Image is super fast! Repeated use of Image in DrawImage() takes just 5-20 microseconds, while initial DrawImage() takes around 10 milliseconds. In comparison, SetSurface() takes here over 20 milliseconds for some reason...

I guess this last DrawImage() result with X11 was the one you were referring to. Unfortunately, this is not the case with Windows and GTK3. Infact, it would change the game entirely if similar results were available with Windows and GTK3.

Best regards,

Tom

*** Please note that Linux and Windows results are not comparable with each other as they are run on different machines. ***
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53638 is a reply to message #53637] Fri, 17 April 2020 17:00 Go to previous messageGo to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
One finding:

Increasing cache size in GTK3 SysDrawImageOp improves repeated DrawImage() performance by a factor of two to about 5-6 ms per update.

void SystemDraw::SysDrawImageOp(int x, int y, const Image& img, Color color)
{
...
//	cache.Shrink(4 * 1024 * 768, 1000); // Cache must be after Paint because of PaintOnly!
	cache.Shrink(4 * 3840 * 2160, 1000); // Cache must be after Paint because of PaintOnly!
}

Best regards,

Tom
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53642 is a reply to message #53638] Sat, 18 April 2020 14:05 Go to previous messageGo to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
Hi Mirek,

I noticed an update in trunk for GTK3 SysDrawImageOp(), but for some reason a fixed value works twice as fast in repeated use.

	//Size sz = Ctrl::GetPrimaryScreenArea().GetSize();
	Size sz(3840,2160); // Twice as fast compared to the above... why?
	cache.Shrink(4 * sz.cx * sz.cy, 1000); // Cache must be after Paint because of PaintOnly!


I just cannot figure out why. I tried even using static Size, with no improvement.

Best regards,

Tom
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53643 is a reply to message #53642] Sat, 18 April 2020 15:06 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 14154
Registered: November 2005
Ultimate Member
Tom1 wrote on Sat, 18 April 2020 14:05
Hi Mirek,

I noticed an update in trunk for GTK3 SysDrawImageOp(), but for some reason a fixed value works twice as fast in repeated use.

	//Size sz = Ctrl::GetPrimaryScreenArea().GetSize();
	Size sz(3840,2160); // Twice as fast compared to the above... why?
	cache.Shrink(4 * sz.cx * sz.cy, 1000); // Cache must be after Paint because of PaintOnly!


I just cannot figure out why. I tried even using static Size, with no improvement.

Best regards,

Tom


Try DDUMP(Ctrl::GetPrimaryScreenArea()); there. Perhaps it is wrong?

In related news, I have reimplemented RectTracker to actually avoid using ViewDraw (what can I do if it does not work on MacOS, right?).

It now works by "snapshoting" master widget into Image and then using normal Paint. In the end I am quite happy about it, as this completely removes the need for RectTracker support in all hosts - it was really ugly in gtk3 and macos...

Mirek

[Updated on: Sat, 18 April 2020 15:07]

Report message to a moderator

Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53644 is a reply to message #53643] Sat, 18 April 2020 16:43 Go to previous messageGo to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
Hi Mirek,

Two things about the changes:

case WM_PAINT:
DLOG("WM_PAINT " << Name(this));

The above DLOG in Win32Proc.cpp prevents Release mode compilation.


Second: RectTracker no longer works at every drag direction. It used to allow it after:

tracker.MinSize(Size(-100000,-100000));

Well, maybe it works behind the scenes, but it does not show the rect on screen for north or west drag directions.

Best regards,

Tom
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53645 is a reply to message #53637] Sat, 18 April 2020 16:48 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 14154
Registered: November 2005
Ultimate Member
As for windows numbers, looks like Image in Win32 was overengineered, one of optimizations was that the first time it is painted, it paints with SetSurface, only for subsequent paints actually move Image to system handles.

I have now exluded all those things whith #if 0 - looks like it improved at least current RectTracker performance quite nicely...

Mirek
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53646 is a reply to message #53643] Sat, 18 April 2020 16:57 Go to previous messageGo to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
mirek wrote on Sat, 18 April 2020 16:06
Tom1 wrote on Sat, 18 April 2020 14:05
Hi Mirek,

I noticed an update in trunk for GTK3 SysDrawImageOp(), but for some reason a fixed value works twice as fast in repeated use.

	//Size sz = Ctrl::GetPrimaryScreenArea().GetSize();
	Size sz(3840,2160); // Twice as fast compared to the above... why?
	cache.Shrink(4 * sz.cx * sz.cy, 1000); // Cache must be after Paint because of PaintOnly!


I just cannot figure out why. I tried even using static Size, with no improvement.

Best regards,

Tom


Try DDUMP(Ctrl::GetPrimaryScreenArea()); there. Perhaps it is wrong?

In related news, I have reimplemented RectTracker to actually avoid using ViewDraw (what can I do if it does not work on MacOS, right?).

It now works by "snapshoting" master widget into Image and then using normal Paint. In the end I am quite happy about it, as this completely removes the need for RectTracker support in all hosts - it was really ugly in gtk3 and macos...

Mirek


I tried DUMPing and got:
Ctrl::GetPrimaryScreenArea() = [0, 0] - [3840, 2080] : (3840, 2080)

This is what I get for LinuxMint using 4K display... It obviously drops the taskbar out of PrimaryScreenArea as the screen height is 2160 in reality. However, the performance is twice as good with a fixed value ( Size(3840, 2080) ) than when calling the function. It looks like calling Ctrl::GetPrimaryScreenArea() takes quite a long time to complete.

Best regards,

Tom
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53647 is a reply to message #53645] Sat, 18 April 2020 17:05 Go to previous messageGo to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
mirek wrote on Sat, 18 April 2020 17:48
As for windows numbers, looks like Image in Win32 was overengineered, one of optimizations was that the first time it is painted, it paints with SetSurface, only for subsequent paints actually move Image to system handles.

I have now exluded all those things whith #if 0 - looks like it improved at least current RectTracker performance quite nicely...

Mirek


Sounds good... Can you put it in trunk for me to test?

BR, Tom
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53648 is a reply to message #53644] Sat, 18 April 2020 17:12 Go to previous messageGo to next message
mirek is currently offline  mirek
Messages: 14154
Registered: November 2005
Ultimate Member
Tom1 wrote on Sat, 18 April 2020 16:43
Hi Mirek,

Two things about the changes:

case WM_PAINT:
DLOG("WM_PAINT " << Name(this));

The above DLOG in Win32Proc.cpp prevents Release mode compilation.


Second: RectTracker no longer works at every drag direction. It used to allow it after:

tracker.MinSize(Size(-100000,-100000));

Well, maybe it works behind the scenes, but it does not show the rect on screen for north or west drag directions.

Best regards,

Tom


Fixed.
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53649 is a reply to message #53648] Sat, 18 April 2020 17:27 Go to previous messageGo to next message
Tom1
Messages: 1277
Registered: March 2007
Senior Contributor
mirek wrote on Sat, 18 April 2020 18:12
Tom1 wrote on Sat, 18 April 2020 16:43
Hi Mirek,

Two things about the changes:

case WM_PAINT:
DLOG("WM_PAINT " << Name(this));

The above DLOG in Win32Proc.cpp prevents Release mode compilation.


Second: RectTracker no longer works at every drag direction. It used to allow it after:

tracker.MinSize(Size(-100000,-100000));

Well, maybe it works behind the scenes, but it does not show the rect on screen for north or west drag directions.

Best regards,

Tom


Fixed.


Thanks! RectTracker works beautifully now. And also my inherited LineTracker. Smile

BR, Tom
Re: MILESTONE: gtk3 replaces gtk2 as default linux backend [message #53651 is a reply to message #53649] Sat, 18 April 2020 18:21 Go to previous messageGo to previous message
mirek is currently offline  mirek
Messages: 14154
Registered: November 2005
Ultimate Member
BTW, tested on R PI 3, seems to work fine (I mean, it is fast enough in UWord, resizing images).

Mirek
Previous Topic: Rapsberry PI - cpu dilemma
Next Topic: U++ on Telegram
Goto Forum:
  


Current Time: Tue Dec 03 01:01:31 CET 2024

Total time taken to generate the page: 0.03683 seconds