This article shows you a few ways to get data from Google reviews. This can help you save Google reviews as an Excel file or other types of files so you can easily use them. There are two ways to scrape Google Reviews data:
- Get Google Maps reviews using Python.
- Use the Google Review Scraper of ScrapeHero Cloud – a tool that doesn’t need coding.
Read my article to know how to scrape reviews from websites quickly and effectively.
What is Google Reviews Scraping?
Google reviews scraping means collecting customer comments about a business or place from Google. This helps businesses understand what people are saying about them online.
Whether scraping is okay or not depends on the laws of the country and place you’re in. Getting public information is usually fine.
How to Scrape Google Reviews with Python?
In this section, I’ll show you how to scrape Google reviews using Python and a tool called Playwright. Playwright acts like a browser in your code, which can help you get around some roadblocks that try to stop scraping. It’s good to know a bit about how Playwright works, though.
You could also use other Python tools like Requests, LXML, or Beautiful Soup to make a scraper for Google Maps without using a browser directly. But getting around the anti-scraping protections they have can be tricky and we won’t cover that here.
Here’s how to scrape Google Maps data with Playwright:
Step 1: Choose Python as your programming language.
Step 2: Install Playwright for Python:
pip install playwright
# to download the browsers you need
playwright installStep 3: Now you’ll write the code to make the browser do what you want and get the data you need from Google Maps using Playwright. Here’s the code you can use:
import asyncio
import json
from playwright.async_api import async_playwright
async def extract_data(page):
review_box_xpath = '//div[@jscontroller="fIQYlf"]'
xpaths = {
'review': '//span[@data-expandable-section] | //span[@class="review-full-text"]',
'author': '//div[@class="TSUbDb"]',
'rating': '//g-review-stars/span',
}
await page.wait_for_selector(review_box_xpath)
review_boxes = page.locator(review_box_xpath)
data = []
for i in range(await review_boxes.count()):
result = review_boxes.nth(i)
data.append({
key: await result.locator(xpath).inner_text() if key != 'rating'
else (await result.locator(xpath).get_attribute('aria-label')).strip(', ')
for key, xpath in xpaths.items()
})
return data
async def run(playwright):
browser = await playwright.chromium.launch(headless=False, proxy={'server': 'proxy url'})
context = await browser.new_context()
page = await context.new_page()
await page.goto("https://www.google.com/")
await page.locator("[aria-label=\"Search\"]").type("burj khalifa")
await page.keyboard.press('Enter')
await page.locator('//a[@data-async-trigger="reviewDialog"]').first.wait_for()
await page.locator('//a[@data-async-trigger="reviewDialog"]').first.click()
for _ in range(3): # pagination
await page.locator('//div[@class="review-dialog-list"]').hover()
await page.mouse.wheel(0, 100000)
await page.wait_for_timeout(2000)
data = await extract_data(page)
with open('google_reviews.json', 'w') as f:
json.dump(data, f, indent=2)
await context.close()
await browser.close()
async def main():
async with async_playwright() as playwright:
await run(playwright)
asyncio.run(main())This code uses the Playwright library in Python to scrape reviews of the Burj Khalifa from Google. It has two main parts:
run function: This function starts a browser, goes to Google, searches for “Burj Khalifa,” and then clicks the button to see reviews. It keeps scrolling down to load more reviews and then saves all the reviews it finds in a file called google_reviews.json.
extract_data function: This function takes information from each review, like who wrote it and what their rating was, and puts it into a neat format.
In the end, this code will make a JSON file with all the reviews you scraped. This file is like a list where each item is a review with its details.
Step 4: Now it’s time to run your code. After it finishes, you’ll have all the Google Maps data you scraped saved in a file.
Note: The specific bits of code used to find information (called Xpaths) might be different depending on where you’re using Google Maps from. Google changes these for different places. The Xpaths used in this example were made while using Google Maps in the United States.
Scrape Google Reviews with ScrapeHero Cloud
ScrapeHero Cloud provides 2 types of Google Reviews Scraper that you can use to scrape Google reviews. Let’s explore and follow these steps to get review data from Google easily and quickly.
No-Code Google Reviews Scraper
The Google Reviews Scraper from ScrapeHero Cloud is a great way to scrape Google reviews without needing to know how to code. It’s easy to use, making it perfect for people who aren’t tech experts.
Here’s how to set up and use the Google Maps scraper:
- Sign up or log in to your ScrapeHero Cloud account.
- Find the Google Reviews scraper in the marketplace.
- Add the scraper to your account (remember to check your email to verify it if you haven’t already).
- You’ll need the Google reviews URL for the business or place you want to scrape. If it’s just one place, put the URL in the box and choose how many pages you want to get.
- You can find the Google review URL from the Google Maps search results or a regular Google search.
- If you have multiple places, switch to Advanced Mode. In the Input tab, add the Google reviews URLs to the SearchQuery field and save your settings.
- Click “Gather Data” to start the scraper.
- The scraper will start getting the data, and you can see how it’s doing in the Jobs tab.
When it’s done, you can look at the data or download it.
You can also save the reviews as an Excel spreadsheet. Just click “Download Data,” pick “Excel,” and then open the file in Microsoft Excel.
Google Reviews Scraper API
The ScrapeHero Cloud Google Reviews API is another way to scrape Google reviews. It’s a simple tool that even people without much tech knowledge can use to easily get customer reviews.
Here’s how to set up and use the Google Reviews API from ScrapeHero Cloud:
- Sign up or log in to your ScrapeHero Cloud account.
- Find the Google Reviews scraper API in the marketplace.
- Click the “subscribe” button.
- You’ll need to choose a plan to use the API because it’s a paid service.
- After subscribing, go to the Documentation tab to learn how to add the API to your app.
Read more:
