|
|
Home » Developing U++ » U++ Developers corner » Touchscreen on raspberry: problem with events
Touchscreen on raspberry: problem with events [message #48625] |
Wed, 09 August 2017 16:16  |
Giorgio
Messages: 218 Registered: August 2015
|
Experienced Member |
|
|
Hi there,
I am porting an application developed with U++ on Windows to a raspberry platform. The raspberry is a Pi 3 Model B, with the latest noobs/raspbian as operating system. The raspberry has attached a EloTouch touch screen, recognized by the kernel as a Amtel maXTouch Digitzer (probably elotouch just rebrand Amtel touchscreen). The driver used is the evdev driver.
The porting of the application was really quick: less than 20 lines of codes changed and the application was up and running on the raspberry. I tested the application with mouse and keyboard and it works as expected. Problems come in with the touch screen.
For some reason, my application does not recognize correctly the "tap". I had this problem both with buttons (more details here) and with Switch control (more details here). I guess that the problem is how upp treats the event (below there are more information on this, that I collected using evtest utility; basically, mouse uses BTN_LEFT event for the click, while the touch screen uses BTN_TOUCH for the tap). This is what works and what does not work in my application:
-Tap/click: it works only with WhenPush or WhenAction callbacks, not with the built-in event (e.g. standard Ok and Cancel buttons do not work, but if I explicitly set the WhenPush callback they work as described here)
-Double click/tap: it works (e.g. double clicking/taping on a row of a SqlArray control gets in edit mode)
-Right click emulation: it does not work (keeping pressed should triggers the right click emulation, but if I keep pressing on a SqlArray row it does not show the "Modify record..." pop up, as it happens if I right click with the mouse)
-multi-touch: not tested
On the OS all the above events (click/tap, double click/tap, right click emulation, multi touch) are treated correctly, so I guess that the problem is on U+++ side.
As I said, I guess is a problem related to the events: I tried to have a look to CtrLib/Switch.cpp and CtrLib/Switch.cpp, but I am not skilled enough to understand if they can be somehow fixed.
I think I cannot solve by myself those problems, so any help will be appreciated.
Thanks,
Gio
More information on the mouse and touch click event:
pi@raspberrypi:~ $ sudo evtest /dev/input/event2
Input driver version is 1.0.1
Input device ID: bus 0x3 vendor 0x192f product 0x416 version 0x111
Input device name: "USB Optical Mouse"
Supported events:
Event type 0 (EV_SYN)
Event type 1 (EV_KEY)
Event code 272 (BTN_LEFT)
Event code 273 (BTN_RIGHT)
Event code 274 (BTN_MIDDLE)
Event type 2 (EV_REL)
Event code 0 (REL_X)
Event code 1 (REL_Y)
Event code 6 (REL_HWHEEL)
Event code 8 (REL_WHEEL)
Event type 4 (EV_MSC)
Event code 4 (MSC_SCAN)
Properties:
Testing ... (interrupt to exit)
Event: time 1502281968.518936, -------------- EV_SYN ------------
Event: time 1502281968.646926, type 4 (EV_MSC), code 4 (MSC_SCAN), value 90001
Event: time 1502281968.646926, type 1 (EV_KEY), code 272 (BTN_LEFT), value 0
Event: time 1502281968.646926, -------------- EV_SYN ------------
pi@raspberrypi:~ $ sudo evtest /dev/input/event3
Input driver version is 1.0.1
Input device ID: bus 0x3 vendor 0x3eb product 0x8a6e version 0x111
Input device name: "Atmel Atmel maXTouch Digitizer"
Supported events:
Event type 0 (EV_SYN)
Event type 1 (EV_KEY)
Event code 330 (BTN_TOUCH)
Event type 3 (EV_ABS)
Event code 0 (ABS_X)
Value 1078
Min 0
Max 4095
Resolution 12
Event code 1 (ABS_Y)
Value 3218
Min 0
Max 4095
Resolution 15
Event code 47 (ABS_MT_SLOT)
Value 0
Min 0
Max 9
Event code 53 (ABS_MT_POSITION_X)
Value 0
Min 0
Max 4095
Resolution 12
Event code 54 (ABS_MT_POSITION_Y)
Value 0
Min 0
Max 4095
Resolution 15
Event code 57 (ABS_MT_TRACKING_ID)
Value 0
Min 0
Max 65535
Properties:
Property type 1 (INPUT_PROP_DIRECT)
Testing ... (interrupt to exit)
Event: time 1502282136.626462, -------------- EV_SYN ------------
Event: time 1502282136.703407, type 3 (EV_ABS), code 57 (ABS_MT_TRACKING_ID), value -1
Event: time 1502282136.703407, type 1 (EV_KEY), code 330 (BTN_TOUCH), value 0
Event: time 1502282136.703407, -------------- EV_SYN ------------
|
|
|
|
|
Re: Touchscreen on raspberry: problem with events [message #48632 is a reply to message #48630] |
Thu, 10 August 2017 12:38   |
Giorgio
Messages: 218 Registered: August 2015
|
Experienced Member |
|
|
Hi there,
I was wrong in my previous post. It seems like the logging has some "buffering" system, so events are not printed on console as soon as they happen, but only when the "buffer" is full. Taking that in mind (and waiting for the buffer to be full), I found out that if I makes a single tap, what is logged is something like:
" GDK_BUTTON_PRESSGDK_PROPERTY_NOTIFYGDK_EXPOSEGDK_EXPOSEGDK_P ROPERTY_NOTIFYGDK_MOTION_NOTIFYGDK_BUTTON_RELEASE "
That makes sense, because the couple GDK_BUTTON_PRESS + GDK_BUTTON_RELEASE (and five events between them) is the same that is logged when I tap on a button, and when - for a button - I define a WhenPush callback the button works.
Now let's see what happens when I click the Switch control with the mouse (in this case the switch control works as expected):
GDK_BUTTON_PRESSGDK_PROPERTY_NOTIFYGDK_EXPOSEGDK_BUTTON_RELE ASE
We have the same couple GDK_BUTTON_PRESS + GDK_BUTTON_RELEASE, but between them there are just two other events (GDK_PROPERTY_NOTIFY and GDK_EXPOSE) while with a single tap here are 5 events between them (GDK_PROPERTY_NOTIFY, GDK_EXPOSE, GDK_EXPOSE, GDK_PROPERTY_NOTIFY, GDK_MOTION_NOTIFY).
I think that the problem is that, but I have no idea how to fix it 
Any suggestion from more experienced user is appreciated.
Thanks,
Gio
|
|
|
Re: Touchscreen on raspberry: problem with events [message #48633 is a reply to message #48632] |
Thu, 10 August 2017 14:17   |
 |
Klugier
Messages: 1099 Registered: September 2012 Location: Poland, Kraków
|
Senior Contributor |
|
|
Hello,
For the next time - please write the events separate. Current version is hard to read, because you need to think when event starts and ends.
I do not think the event inside meters so much, we have the correct event order (GDK_BUTTON_PRESS and GDK_BUTTON_RELEASE) when user touch. All we need to know is they are handle correctly. You need to check Switch control code to make sure they are propagate correctly. You could also log the all switch events that happens for example:
void Switch::MouseMove(Point p, dword keyflags) {
Cout() << "Switch::MouseMove(): ";
...
}
// Add log for the rest of events and try to understand the logic...
void Switch::LeftDown(Point p, dword keyflags) { ... }
void Switch::LeftUp(Point p, dword keyflags) { ... }
void Switch::MouseLeave() { ... }
In my opinion " pushindex = -1;" when mouse leave the area could be a problem. This is the highly probably, because tap can move mouse somewhere else and leave the switch area. Please check this clue - you will need to play with Switch event handling a little bit.
For logging you just simply use Cout() and launch your app in terminal. Just like this:
Cout() << "My message!" << "\n";
You could use upp more advenced logging mechanism, however I think Cout() should be sufficient for our needs.
Sincerely,
Klugier
U++ - one framework to rule them all.
[Updated on: Thu, 10 August 2017 14:19] Report message to a moderator
|
|
|
|
|
Re: Touchscreen on raspberry: problem with events [message #48638 is a reply to message #48635] |
Thu, 10 August 2017 16:57   |
rafiwui
Messages: 105 Registered: June 2017 Location: Stuttgart, Germany
|
Experienced Member |
|
|
Quote:I will try to reach out the person who wrote Switch.cpp to see how we can work out things in a way that the touch screen is supported
I guess it would be enough if there would be a very short time after a LeftDown in which MouseMove is ignored.
Greetings
Daniel
|
|
|
Re: Touchscreen on raspberry: problem with events [message #48641 is a reply to message #48637] |
Thu, 10 August 2017 17:13   |
Giorgio
Messages: 218 Registered: August 2015
|
Experienced Member |
|
|
Klugier wrote on Thu, 10 August 2017 16:55
I am happy you found the origin of that problem by yourself. I believe it is not, so hard 
Actually it wasn't so hard, but without your suggestion I would have been lost.
Klugier wrote on Thu, 10 August 2017 16:55
We need more people like you!!!
Well, we need more people like you from the U++ team my little debugging effort is the least I can do to give back to the U++ community.
Klugier wrote on Thu, 10 August 2017 16:55
The patch must be accepted by Mirek - I will raise the ticket on redmine (Our bugtrack) to not forget your issue. Of course, it could take some time, so please be patient. Your current application might work with custom CtrlLib for now.
No problem, the porting is just started...
|
|
|
Re: Touchscreen on raspberry: problem with events [message #48752 is a reply to message #48641] |
Sun, 10 September 2017 16:31   |
 |
mirek
Messages: 14255 Registered: November 2005
|
Ultimate Member |
|
|
Giorgio wrote on Thu, 10 August 2017 17:13Klugier wrote on Thu, 10 August 2017 16:55
I am happy you found the origin of that problem by yourself. I believe it is not, so hard 
Actually it wasn't so hard, but without your suggestion I would have been lost.
Klugier wrote on Thu, 10 August 2017 16:55
We need more people like you!!!
Well, we need more people like you from the U++ team my little debugging effort is the least I can do to give back to the U++ community.
Klugier wrote on Thu, 10 August 2017 16:55
The patch must be accepted by Mirek - I will raise the ticket on redmine (Our bugtrack) to not forget your issue. Of course, it could take some time, so please be patient. Your current application might work with custom CtrlLib for now.
No problem, the porting is just started...
Hi,
I am sorry fot the late reply. It all seems weird - MouseMove there should not cause this kind of problem, unless it has some invalid Point IMO.
(You can check that by simulating with 'real mouse' - click on item, then move the mouse, then release).
Would it be possible to add DUMP(p) to MouseMove / LeftUp / LeftDown ?
|
|
|
|
|
Re: Touchscreen on raspberry: problem with events [message #49508 is a reply to message #48817] |
Wed, 21 February 2018 11:54   |
Giorgio
Messages: 218 Registered: August 2015
|
Experienced Member |
|
|
Hi Mirek,
sorry for the very late answer. The project of porting the application to a touch platform was put in hold by the Company I work for, only during these days I was finally able to resume it. Since the time when I reported the issue, there has been a change of the platform: we used a Raspbian on a Raspberry, now we moved to a Debian (latest, 9.3 stretch) on a standard PC. The touch screen is still an EloTouch. These are some details about drivers and configuration of the video:
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller (rev 09)
drm_kms_helper 155648 1 i915
drm 360448 8 i915,drm_kms_helper
/dev/fb0
/dev/dri/card0
/dev/dri/renderD128
/dev/dri/controlD64
BOOT_IMAGE=/boot/vmlinuz-4.9.0-4-amd64 root=UUID=98a2e6b5-fc24-44c8-92cb-52a8c17f70a9 ro quiet
/etc/modprobe.d/
direct rendering: Yes
server glx vendor string: SGI
client glx vendor string: Mesa Project and SGI
Vendor: Intel Open Source Technology Center (0x8086)
OpenGL vendor string: Intel Open Source Technology Center
[ 58.105] (II) LoadModule: "glx"
[ 58.129] (II) LoadModule: "modesetting"
[ 58.135] (II) LoadModule: "fbdev"
[ 58.151] (II) LoadModule: "vesa"
[ 58.153] (II) LoadModule: "fbdevhw"
[ 58.164] (II) LoadModule: "glamoregl"
[ 58.250] (II) LoadModule: "fb"
[ 58.459] (II) LoadModule: "libinput"
I added the DUMP(p) as per suggestion, and this is the result with a single tap on a switch control (note: this with the standard version of Switch.cpp, not the one I modified as described in this thread):
p = [160, 138]
p = [160, 138]
p = [160, 138]
p = [160, 138]
p = [160, 138]
I hope this could help and tell me if I should run some further tests.
Regards,
Gio
|
|
|
Re: Touchscreen on raspberry: problem with events [message #49515 is a reply to message #49508] |
Thu, 22 February 2018 07:45   |
 |
mirek
Messages: 14255 Registered: November 2005
|
Ultimate Member |
|
|
Giorgio wrote on Wed, 21 February 2018 11:54Hi Mirek,
sorry for the very late answer. The project of porting the application to a touch platform was put in hold by the Company I work for, only during these days I was finally able to resume it. Since the time when I reported the issue, there has been a change of the platform: we used a Raspbian on a Raspberry, now we moved to a Debian (latest, 9.3 stretch) on a standard PC. The touch screen is still an EloTouch. These are some details about drivers and configuration of the video:
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller (rev 09)
drm_kms_helper 155648 1 i915
drm 360448 8 i915,drm_kms_helper
/dev/fb0
/dev/dri/card0
/dev/dri/renderD128
/dev/dri/controlD64
BOOT_IMAGE=/boot/vmlinuz-4.9.0-4-amd64 root=UUID=98a2e6b5-fc24-44c8-92cb-52a8c17f70a9 ro quiet
/etc/modprobe.d/
direct rendering: Yes
server glx vendor string: SGI
client glx vendor string: Mesa Project and SGI
Vendor: Intel Open Source Technology Center (0x8086)
OpenGL vendor string: Intel Open Source Technology Center
[ 58.105] (II) LoadModule: "glx"
[ 58.129] (II) LoadModule: "modesetting"
[ 58.135] (II) LoadModule: "fbdev"
[ 58.151] (II) LoadModule: "vesa"
[ 58.153] (II) LoadModule: "fbdevhw"
[ 58.164] (II) LoadModule: "glamoregl"
[ 58.250] (II) LoadModule: "fb"
[ 58.459] (II) LoadModule: "libinput"
I added the DUMP(p) as per suggestion, and this is the result with a single tap on a switch control (note: this with the standard version of Switch.cpp, not the one I modified as described in this thread):
p = [160, 138]
p = [160, 138]
p = [160, 138]
p = [160, 138]
p = [160, 138]
I hope this could help and tell me if I should run some further tests.
Regards,
Gio
Eh, I hoped to get these DDUMPs together with previous your LOGs
GDK_BUTTON_PRESS
GDK_PROPERTY_NOTIFY
Switch::MouseMove: pushindex: 3 value: 002
GDK_EXPOSE
Switch::MouseMove: pushindex: 3 value: 002
Switch::LeftDown: pushindex: 3 value: 002
GDK_EXPOSE
GDK_MOTION_NOTIFY
GDK_BUTTON_RELEASE
Switch::MouseMove: pushindex: -1 value: 002
GDK_EXPOSE
Switch::LeftUp (before the outer if cycle): pushindex: -1 value: 002
Switch::LeftUp: pushindex: -1 value: 002
|
|
|
|
Re: Touchscreen on raspberry: problem with events [message #49523 is a reply to message #49516] |
Thu, 22 February 2018 18:56   |
Giorgio
Messages: 218 Registered: August 2015
|
Experienced Member |
|
|
Hi Mirek,
this is the output after adding the DDUMP:
p = [158, 142]
keyflags & K_MOUSELEFT = 8388608
p = [158, 142]
keyflags & K_MOUSELEFT = 0
p = [158, 142]
keyflags & K_MOUSELEFT = 0
p = [158, 142]
keyflags & K_MOUSELEFT = 0
p = [158, 142]
keyflags & K_MOUSELEFT = 0
Aside of that, I dug up a little the issue. My first tests (some months ago) was made with a raspbian on a raspberry and, as far as I recalled, most of the controls had problems. Some days ago I started using a plain debian and the only thing that had problems was the Switch control. I made some test using a different window manager (matchbox) and I noticed that using that, also the buttons have problems (once pressed they do not release). So far, the best compatibility I found was with gdm (Gnome Desktop Manager), but may happen that other desktop manager have a total compatibility. By the way, the log attached was got using gdm.
Regards,
Gio
|
|
|
|
|
Re: Touchscreen on raspberry: problem with events [message #49543 is a reply to message #49524] |
Mon, 26 February 2018 14:55   |
Giorgio
Messages: 218 Registered: August 2015
|
Experienced Member |
|
|
Hi Mirek,
this is the log with LOG() of a single tap (using the latest stable version of upp):
MouseMove
p = [155, 136]
keyflags & K_MOUSELEFT = 8388608
MouseMove
p = [155, 136]
keyflags & K_MOUSELEFT = 0
MouseMove
p = [155, 136]
keyflags & K_MOUSELEFT = 0
LeftDown
p = [155, 136]
keyflags & K_MOUSELEFT = 0
LeftUp
p = [155, 136]
keyflags & K_MOUSELEFT = 0
MouseMove
I also tried the nightly build and now the switch control works, although it works only if I keep pressed a little bit the option. I performed also another kind of test. I installed matchbox window manager (all the previous tests were performed with gnome), and with that wm buttons works erratically (e.g. they remain pressed). According to my searches, gnome's wm (mutter) uses OpenGL compatible libraries (I did not find anything about matchbox). So my hypothesis is that upp used with linux box with a touch screen works well with OpenGL libraries, and not with every graphic library.
Regards,
Giorgio
|
|
|
Re: Touchscreen on raspberry: problem with events [message #49545 is a reply to message #49543] |
Mon, 26 February 2018 16:20   |
 |
mirek
Messages: 14255 Registered: November 2005
|
Ultimate Member |
|
|
Giorgio wrote on Mon, 26 February 2018 14:55Hi Mirek,
this is the log with LOG() of a single tap (using the latest stable version of upp):
MouseMove
p = [155, 136]
keyflags & K_MOUSELEFT = 8388608
MouseMove
p = [155, 136]
keyflags & K_MOUSELEFT = 0
MouseMove
p = [155, 136]
keyflags & K_MOUSELEFT = 0
LeftDown
p = [155, 136]
keyflags & K_MOUSELEFT = 0
LeftUp
p = [155, 136]
keyflags & K_MOUSELEFT = 0
MouseMove
That does not make sense at all... 
Would it be possible to do test, with logs in GtkEvent.cpp:
#include <CtrlCore/CtrlCore.h>
#ifdef GUI_GTK
namespace Upp {
#define LLOG(x) // DLOG(rmsecs() << ' ' << x)
#define LOG_EVENTS
void Ctrl::AddEvent(gpointer user_data, int type, const Value& value, GdkEvent *event)
{
if(Events.GetCount() > 50000)
return;
GEvent& e = Events.AddTail();
e.windowid = (uint32)(uintptr_t)user_data;
e.type = type;
e.value = value;
gint x, y;
GdkModifierType mod;
gdk_window_get_pointer(gdk_get_default_root_window(), &x, &y, &mod);
e.mousepos = Point(x, y);
e.state = mod;
e.count = 1;
e.event = NULL;
if(event) {
e.time = gdk_event_get_time(event);
e.event = gdk_event_copy(event);
}
else {
e.time = gtk_get_current_event_time();
e.event = gtk_get_current_event();
}
DUMPHEX(mod);
}
Mirek
|
|
|
Goto Forum:
Current Time: Sat Apr 26 14:18:48 CEST 2025
Total time taken to generate the page: 0.03558 seconds
|
|
|