Disclosure Statement: This site contains affiliate links, which means that I may receive a commission if you make a purchase using these links. As an eBay Partner, I earn from qualifying purchases.
SendMouseEvent
SendMouseEvent
I'm trying to simulate a mouse click to a button on a website. I'm getting the coordinates of the button using HTML getBoundingClientRect() and then sending WVBrowser1.SendMouseEvent with mouse down and then mouse up using a TPoint with the x and y coordinates from getBoundingClientRect(). However, the mouse click is occurring on a different element on the web page. How can I make sure the coordinates I'm sending match the button I'm trying to click?
- salvadordf
- Posts: 4156
- Joined: Thu Feb 02, 2017 12:24 pm
- Location: Spain
- Contact:
Re: SendMouseEvent
Hi.
See this example in the WindowlessBrowser demo to know how to simulate mouse events :
https://github.com/salvadordf/WebView4Delphi/blob/2c7dec03305b1fbb966d81911e3c4179d3e1294b/demos/Delphi_VCL/WindowlessBrowser/uWindowlessBrowser.pas#L117
See this example in the WindowlessBrowser demo to know how to simulate mouse events :
https://github.com/salvadordf/WebView4Delphi/blob/2c7dec03305b1fbb966d81911e3c4179d3e1294b/demos/Delphi_VCL/WindowlessBrowser/uWindowlessBrowser.pas#L117
Re: SendMouseEvent
Here is the code I'm using to send the mouse events:
Seems to me that the x and y values are relative to the webpage and need to be translated to the screen?
Code: Select all
P := Point(x, y);
WVBrowser1.SendMouseInput(COREWEBVIEW2_MOUSE_EVENT_KIND_LEFT_BUTTON_DOWN,
COREWEBVIEW2_MOUSE_EVENT_VIRTUAL_KEYS_NONE, 0, P);
WVBrowser1.SendMouseInput(COREWEBVIEW2_MOUSE_EVENT_KIND_LEFT_BUTTON_UP,
COREWEBVIEW2_MOUSE_EVENT_VIRTUAL_KEYS_NONE, 0, P);
- salvadordf
- Posts: 4156
- Joined: Thu Feb 02, 2017 12:24 pm
- Location: Spain
- Contact:
Re: SendMouseEvent
Read the code comments in the original code sample :
https://learn.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/icorewebview2compositioncontroller?view=webview2-1.0.2478.35#sendmouseinput
TMainForm.HandleMouseMessage is just a Delphi translation but the original has more code comments.
SendMouseInput expects client coordinates for the WebView and the mouse coordinates must be translated for some messages.
https://learn.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/icorewebview2compositioncontroller?view=webview2-1.0.2478.35#sendmouseinput
TMainForm.HandleMouseMessage is just a Delphi translation but the original has more code comments.
SendMouseInput expects client coordinates for the WebView and the mouse coordinates must be translated for some messages.
Re: SendMouseEvent
I've tried several things, all to no avail.
Here's one:
Here's another (based on the code comments in the original code sample):
And adding either ScreenToClient(P) or ClientToScreen(P) doesn't help.
Here's one:
Code: Select all
function TMainForm.OffsetPointToWebView(aPoint: TPoint): TPoint;
begin
Result := ScreenToclient(aPoint);
Result.x := Result.x - FWVDirectCompositionHost.Left;
Result.y := Result.y - FWVDirectCompositionHost.Top;
end;
Code: Select all
P := Point(x, y);
P.x := P.x - WVBrowser1.Bounds.Left;
P.y := P.y - WVBrowser1.Bounds.Top;
- salvadordf
- Posts: 4156
- Joined: Thu Feb 02, 2017 12:24 pm
- Location: Spain
- Contact:
Re: SendMouseEvent
Are you using the WindowlessBrowser demo for the tests?
SendMouseInput only works when the browser uses a "Composition Controller" which is what the WindowlessBrowser demo uses.
I just tested your code in the WindowlessBrowser demo and the simulated click works without any coordinate system transformation.
Just send the getBoundingClientRect().x and getBoundingClientRect().y values to Delphi and populate the point with them.
Edit : Just to be sure, use the middle point of that element to simulate the click.
SendMouseInput only works when the browser uses a "Composition Controller" which is what the WindowlessBrowser demo uses.
I just tested your code in the WindowlessBrowser demo and the simulated click works without any coordinate system transformation.
Just send the getBoundingClientRect().x and getBoundingClientRect().y values to Delphi and populate the point with them.
Code: Select all
P := Point(x, y);
WVBrowser1.SendMouseInput(COREWEBVIEW2_MOUSE_EVENT_KIND_LEFT_BUTTON_DOWN,
COREWEBVIEW2_MOUSE_EVENT_VIRTUAL_KEYS_NONE, 0, P);
WVBrowser1.SendMouseInput(COREWEBVIEW2_MOUSE_EVENT_KIND_LEFT_BUTTON_UP,
COREWEBVIEW2_MOUSE_EVENT_VIRTUAL_KEYS_NONE, 0, P);
- salvadordf
- Posts: 4156
- Joined: Thu Feb 02, 2017 12:24 pm
- Location: Spain
- Contact:
Re: SendMouseEvent
It should also be possible to use the Input.dispatchMouseEvent DevTools method to simulate mouse events :
https://chromedevtools.github.io/devtools-protocol/tot/Input/#method-dispatchMouseEvent
That method should work even without using a Composition Controller.
https://chromedevtools.github.io/devtools-protocol/tot/Input/#method-dispatchMouseEvent
That method should work even without using a Composition Controller.
Re: SendMouseEvent
Okay, that worked. That's what I did originally, so I don't know why I thought it didn't work.
Thank you, Salvador.
Thank you, Salvador.
Re: SendMouseEvent
My apologies, I thought it worked because without realizing it I had left a "click()" on the button in the javascript.
I'm not getting the correct x and y values from the getBoundingClientRect() function. Is there a way in Webview4Delphi similar to Cef4Delphi to read the DOM elements and get the bounds?
I'm not getting the correct x and y values from the getBoundingClientRect() function. Is there a way in Webview4Delphi similar to Cef4Delphi to read the DOM elements and get the bounds?
- salvadordf
- Posts: 4156
- Joined: Thu Feb 02, 2017 12:24 pm
- Location: Spain
- Contact:
Re: SendMouseEvent
Please, provide a simplified demo that I can use to reproduce this issue in my computer.