Recommendations for unattended webspider/screen scraping style application

[LachlanG]

We're considering using CEF4Delphi in an application that would run unattended, navigating to particular websites, clicking links/buttons and filling in forms on those websites, and scraping data from those websites. It would be nice to be able to run the application as a Windows service.

Would you suggest using the OSR mode for this style of application?

Do you have any other general advice or things to look out for when writing a non-interactive CEF4Delphi application?

Thanks very much,

Lachlan

[salvadordf]

Hi,

I've never created a Windows service.
I'm sorry but I'm afraid I can't help you with this.

[LachlanG]

Ignore the Windows service requirement for now then.

What if the application was just a TForm with a single TButton on it. Pressing that TButton would kick off a process that navigated to a website, clicked on some links, entered some data in controls, and scraped some data off the pages.

The user never needs to see what is happening, everything is automated and hidden from the view of the user.

Would you recommend using OSR for such a program or use a different approach?

[salvadordf]

In that case a browser in OSR mode seems to be the right choice.

I just wrote this that may help you doing those tasks :
viewtopic.php?f=10&p=6067#p6066