Page 1 of 1
Recommendations for unattended webspider/screen scraping style application
Posted: Mon Aug 17, 2020 8:09 am
by LachlanG
We're considering using CEF4Delphi in an application that would run unattended, navigating to particular websites, clicking links/buttons and filling in forms on those websites, and scraping data from those websites. It would be nice to be able to run the application as a Windows service.
Would you suggest using the OSR mode for this style of application?
Do you have any other general advice or things to look out for when writing a non-interactive CEF4Delphi application?
Thanks very much,
Lachlan
Re: Recommendations for unattended webspider/screen scraping style application
Posted: Mon Aug 17, 2020 9:26 am
by salvadordf
Hi,
I've never created a Windows service.
I'm sorry but I'm afraid I can't help you with this.
Re: Recommendations for unattended webspider/screen scraping style application
Posted: Tue Aug 18, 2020 4:39 am
by LachlanG
Ignore the Windows service requirement for now then.
What if the application was just a TForm with a single TButton on it. Pressing that TButton would kick off a process that navigated to a website, clicked on some links, entered some data in controls, and scraped some data off the pages.
The user never needs to see what is happening, everything is automated and hidden from the view of the user.
Would you recommend using OSR for such a program or use a different approach?
Re: Recommendations for unattended webspider/screen scraping style application
Posted: Tue Aug 18, 2020 9:03 am
by salvadordf
In that case a browser in OSR mode seems to be the right choice.
I just wrote this that may help you doing those tasks :
viewtopic.php?f=10&p=6067#p6066