Skip to content Skip to sidebar Skip to footer

Requests.get Showing Different Html Than Chrome's Developer Tool

I am working on a web scraping tool using python (specifically jupyter notebook) that scrapes a few real estate pages and saves the data like price, adress etc. It is working just

Solution 1:

If eventually data from that page you are after, you can get it very easily using selenium in combination with BeautifulSoup. It gives you all the links of apartments.

from bs4 import BeautifulSoup
from selenium import webdriver

driver = webdriver.Chrome()

driver.get("https://www.sreality.cz/hledani/prodej/byty/brno?stari=mesic")
soup = BeautifulSoup(driver.page_source,"html.parser")
driver.quit()

for title in soup.select(".text-wrap"):
    num = "https://www.sreality.cz" + title.select_one(".title").get('href')
    print(num)

Post a Comment for "Requests.get Showing Different Html Than Chrome's Developer Tool"