extract values from webpage

Post Reply
dsgfdsagr
Posts: 6
Joined: Mon Jul 19, 2021 10:51 am

extract values from webpage

Post by dsgfdsagr »

Hey,

How to pick up values from webpage, if i have the XPATH of the nodes? using MiniBrowser .
is there a better Demo to try and do it?

i found that a selector can be used like this

var dates = $$('div',document.querySelectorAll);
for(each in dates){console.log(dates[each].innerText);}

the 'div' you can get with the chrome devTools. highlight the item, and right click on it, copy then copy selector.
however, I don't know how to interface the javascript with the delphi, except call Chrominum1.ExecuteJavaScript()
User avatar
salvadordf
Posts: 3067
Joined: Thu Feb 02, 2017 12:24 pm
Location: Spain
Contact:

Re: extract values from webpage

Post by salvadordf »

Hi,

CEF allows you to send and receive information between the main browser process and the renderer process.

You can use several ways to send that information and most of them are described in the /demos/Delphi_VCL/DOMVIsitor and /demos/Delphi_VCL/JavaScript/JSExtension demos.

Read the code comments in those demos for more information.
dsgfdsagr
Posts: 6
Joined: Mon Jul 19, 2021 10:51 am

Re: extract values from webpage

Post by dsgfdsagr »

Thank you so much. I will read those demos and post my understandings.
dsgfdsagr
Posts: 6
Joined: Mon Jul 19, 2021 10:51 am

Re: extract values from webpage

Post by dsgfdsagr »

a must know is :

https://magpcss.org/ceforum/viewtopic.php?f=10&t=634
The DOM API exposed by Chromium is read-only with the exception of adding listeners. CEF exposes this same API via the CefFrame::VisitDOM method. If you want to manipulate the DOM directly via C++ it will be necessary to use WebKit interfaces directly.
which means you cannot make the javascript change the page like
document.body.innerHTML ='value' ;

to clear the webpage.
dsgfdsagr
Posts: 6
Joined: Mon Jul 19, 2021 10:51 am

Re: extract values from webpage

Post by dsgfdsagr »

this

Code: Select all

import $ from 'jquery';
cause this exception
Debug Output:
[0804/113627.290:INFO:CONSOLE(10)] "Uncaught SyntaxError: Cannot use import statement outside a module", source: about:blank (10)

Process DOMVisitor.exe (4792)

when i try to do this

Code: Select all

function TDOMVisitorFrm.ReadScript: string;
var
  FileStream:TFileStream;
  StringStream:TStringStream;
begin
  FileStream := TFileStream.Create('C:\folder\parse.js',fmOpenRead);
  stringStream := TStringStream.Create;
  stringStream.LoadFromStream(FileStream);
  result := StringStream.DataString;
  fileStream.Free;
  stringstream.Free;
end;


procedure TDOMVisitorFrm.Chromium1ContextMenuCommand(Sender: TObject;
....
      MINIBROWSER_CONTEXTMENU_GETDATA: //new event
      begin
        RawLine := ReadScript;
        frame.ExecuteJavaScript(RawLine,'about:blank',0);
        showmessage(RawLine);
      end;
      
dsgfdsagr
Posts: 6
Joined: Mon Jul 19, 2021 10:51 am

Re: extract values from webpage

Post by dsgfdsagr »

using the JSEval result in
Uncaught ReferenceError: $$ is not defined
which means the jquery is not loaded. here too.

what am I missing?
User avatar
salvadordf
Posts: 3067
Joined: Thu Feb 02, 2017 12:24 pm
Location: Spain
Contact:

Re: extract values from webpage

Post by salvadordf »

I'm sorry but I'm not a JS expert and I don't have an answer but the error message "Uncaught SyntaxError: Cannot use import statement outside a module" suggests that this is a pure JavaScript issue.

Perhaps this could be helpful :
https://exerror.com/uncaught-syntaxerror-cannot-use-import-statement-outside-a-module-when-importing-ecmascript-6/
dsgfdsagr
Posts: 6
Joined: Mon Jul 19, 2021 10:51 am

Re: extract values from webpage

Post by dsgfdsagr »

the easiest way to get a value from the browser is this:

https://www.briskbard.com/forum/viewtopic.php?p=2886&sid=dcfc576df0a494604148f07b74292370#p2886

and use jsRTTIExtention

limited to 1000 charachters.
dsgfdsagr
Posts: 6
Joined: Mon Jul 19, 2021 10:51 am

Re: extract values from webpage

Post by dsgfdsagr »

now how do i get a value from the browser if using the SubProcess?

the procedure
GlobalCEFApp_OnContextCreated
sets the value but i want to get a value.
User avatar
salvadordf
Posts: 3067
Joined: Thu Feb 02, 2017 12:24 pm
Location: Spain
Contact:

Re: extract values from webpage

Post by salvadordf »

You can execute custom JavaScript code from the main browser process and the render process to set values in the DOM.

See the DOMVIsitor demo to know how to send a message to the render process. Then use the "frame" parameter in GlobalCEFApp_OnProcessMessageReceived to call ExecuteJavaScript in the render process like this :

Code: Select all

frame.ExecuteJavaScript(YourJSCode, frame.url, 0);
Post Reply