Page 1 of 1

RetrieveHTML bug ?

Posted: Wed Mar 21, 2018 1:45 pm
by thefunkyjoint
Hi,

I'm using the RetrieveHTML() method to get the html source of a page.

But i noticed the following behaviour : when the page is big and have a big scrollbar, this method seems to retrieve only part of the html source, not the entire page. I'm testing this for Instagram ; when i go to the Instagram timeline, no matter how much i roll the scrollbar down to fetch new data, the RetrieveHTML() method always give me the same count of lines on its result, and it seems only the 'visible' part of the page is having its source fetched ; if i want the source from a part of the page below, i need to scroll down and call the method again.

Is this a bug or am i missing something here ?

Thanks

Re: RetrieveHTML bug ?

Posted: Thu Mar 22, 2018 7:52 am
by salvadordf
It seems that Instagram generates that page dynamically and only adds content when you scroll.
Open the DevTools to see if the page really has all the HTML you need.