Disclosure Statement: This site contains affiliate links, which means that I may receive a commission if you make a purchase using these links. As an eBay Partner, I earn from qualifying purchases.

RetrieveHTML bug ?

Post Reply
thefunkyjoint
Posts: 460
Joined: Thu Aug 10, 2017 12:40 pm

RetrieveHTML bug ?

Post by thefunkyjoint »

Hi,

I'm using the RetrieveHTML() method to get the html source of a page.

But i noticed the following behaviour : when the page is big and have a big scrollbar, this method seems to retrieve only part of the html source, not the entire page. I'm testing this for Instagram ; when i go to the Instagram timeline, no matter how much i roll the scrollbar down to fetch new data, the RetrieveHTML() method always give me the same count of lines on its result, and it seems only the 'visible' part of the page is having its source fetched ; if i want the source from a part of the page below, i need to scroll down and call the method again.

Is this a bug or am i missing something here ?

Thanks
User avatar
salvadordf
Posts: 4056
Joined: Thu Feb 02, 2017 12:24 pm
Location: Spain
Contact:

Re: RetrieveHTML bug ?

Post by salvadordf »

It seems that Instagram generates that page dynamically and only adds content when you scroll.
Open the DevTools to see if the page really has all the HTML you need.
Post Reply