How to Build a Google Scraping AI Agent with n8n (Step By Step Tutorial)

How to Build a Google Scraping AI Agent with n8n (Step By Step Tutorial)

by Nate Herk

Applied Sciences & Engineering AI — Models & Research UYQArtificial intelligence (AI)UYQMMachine learning
🎙 Nate Herk 👥 964K 📅 October 6, 2024 ⏱ 20 min 👁 49K 📄 tutorial 🧭 2026-08-28
Available in: English (current) Français

Keywords

n8nAI agentGoogle scrapingLinkedInlead generation

Summary

This tutorial demonstrates how to build an AI agent using n8n that scrapes Google for LinkedIn profile URLs. The creator walks through two workflows: first, a ’tool’ workflow that performs the scraping, and second, an ‘agent’ workflow that interacts with the user and calls the tool. The tool workflow uses a ‘When called by another workflow’ trigger, an OpenAI node to parse the user query into structured parameters, an HTTP Request node to query Google, a Code node to extract LinkedIn URLs from the HTML, a Google Sheets node to append the URLs to a spreadsheet, and a Set node to signal completion. The agent workflow uses a chat trigger, an AI agent node, an OpenAI chat model, window buffer memory, and a workflow tool that calls the scraping workflow. The creator demonstrates the system with examples like ‘CEOs in real estate in Chicago’ and ‘Founders in technology in San Francisco’, showing the results in Google Sheets. He also discusses limitations, such as only returning 10 results and potential CAPTCHA issues, and suggests using a dedicated search API like SerpAPI for more results. The tutorial emphasizes using an n8n assistant on ChatGPT to generate code and headers, making it accessible for non-coders.

202 words

Critical Evaluation

Value of the Information & Strength of the Argument

The video provides a clear, step-by-step practical guide that is easy to follow, with visual demonstrations and explanations of each node. The creator explains the reasoning behind each configuration, such as why a user agent header is needed and how to parse HTML. The argumentation is solid for a tutorial, as it shows real working examples and acknowledges limitations. However, the approach relies on an unofficial scraping method that may violate Google’s terms of service, and the creator does not deeply discuss ethical or legal implications. The use of AI to generate code is pragmatic but not rigorously verified, and the creator admits uncertainty about certain aspects, inviting community feedback.

Scientific Rigor, Source Quality, Title Accuracy

The tutorial is based on the creator’s own experience and uses n8n, OpenAI, and Google Sheets. The only external source mentioned is the n8n assistant on ChatGPT, which is used to generate code and headers. The description provides links to a Skool community and other videos, but no official documentation or academic sources are cited. The title accurately reflects the content, which is a tutorial on building a Google scraping AI agent with n8n. The creator does not provide references to official n8n documentation or Google’s policies, which would strengthen the scientific rigor. The video is more of a practical guide than a scientific study, so the lack of formal sources is not surprising, but it limits the reliability of the information.

247 words

Title / Content Match

The title accurately describes the content: a step-by-step tutorial on building a Google scraping AI agent with n8n.

Quality & Reliability

6/10

The tutorial is practical and reproducible, but relies on an unofficial method (HTTP request to Google) and uses AI assistance for code generation without deep verification. The creator acknowledges limitations and invites community input, showing some scientific caution.

Chapters

Cited Sources

Concurring Sources

  • n8n Documentation — Official documentation for n8n, which supports the workflow setup described in the video.
  • OpenAI API Documentation — Official documentation for OpenAI API, relevant for the chat model and API key setup.

Dissenting Sources

  • Google's Terms of Service — The method of scraping Google via HTTP requests may violate Google's Terms of Service, which prohibit automated access without permission.

Contribution & Novelties

The video offers a practical, accessible tutorial for building an AI agent with n8n that scrapes Google, targeting non-coders. It demonstrates a complete workflow from user query to data storage in Google Sheets, and shows how to use an AI assistant (ChatGPT) to generate code and headers, lowering the barrier to entry. The main novelty is the integration of an AI agent with a custom scraping tool, which is a common pattern in automation but presented in a beginner-friendly way.

Pour aller plus loin :

  • n8n Documentation — Official documentation for n8n, useful for understanding nodes and workflows.
  • OpenAI API Documentation — Official documentation for OpenAI API, relevant for the chat model and API key setup.
  • Google Custom Search JSON API — Official API for programmatic Google search, an alternative to scraping that is more reliable and compliant.
  • SerpAPI — A service for scraping search engine results, mentioned by the creator as a potential solution for more results.
  • Web scraping ethics and legality — Wikipedia article on web scraping, discussing legal and ethical considerations.

174 words

Radar Profile

The radar profile shows moderate scores across all dimensions, with slightly higher scores in quantity of information and technical level, reflecting a practical tutorial that provides a good amount of detail but lacks deep scientific rigor and reliability due to the unofficial scraping method.

Reliability 5/10