Curated

(5/5)

Autonomous AI crawler

Text scraper tool This tool is designed to return all text from the given webpage. 💡 Consider adding proxy for better crawling accuracy.

Integrations Used (6)

Toolworkflow

Lmchatopenai

Outputparserstructured

Supabase

HTTP Request

Agent

Workflow Structure (28 nodes)

100%

Manual Trigger

Execute workflow

Supabase

Get companies

Set

Select company name and website

Set

Map company name and website

Langchain.agent

Crawl website

Set

Set social media array

Merge

Merge all data

Supabase

Insert new row

Langchain.lm Chat Open Ai

OpenAI Chat Model

Langchain.tool Workflow

Text

Set

Set domain (text)

Set

Set domain (URL)

Langchain.tool Workflow

URLs

Set

Add protocool to domain (text)

Set

Add protocool to domain (URL)

Langchain.output Parser Structured

JSON Parser

Http Request

Get website (text)

Http Request

Get website (URL)

Markdown

Convert HTML to Markdown

Html

Retrieve URLs

Set

Set response (text)

Split Out

Split out URLs

Filter

Filter out empty hrefs

Remove Duplicates

Remove duplicated

Set

Set domain to path

Filter

Filter out invalid URLs

Aggregate

Aggregate URLs

Set

Set response (URL)

Node Types:

default

transform

trigger

logic

http

Requires credentials

Prerequisites

Required Credentials (2)

OpenAI

Used by: OpenAI Chat Model

Docs

Supabase

Used by: Get companies, Insert new row

Docs

Complexity

Complex

Advanced features used. Requires n8n experience.

Setup time: ~35 min

Score: 60/100

Conditional Logic

8 Integrations

When to Use This Workflow

AI-Powered Content Creation

Automate content generation, summarization, or analysis using AI capabilities.

Save hours of manual work while maintaining quality and consistency.

Real-time Response

React instantly when events happen - new orders, form submissions, or API calls.

Zero delay between trigger and action for time-sensitive workflows.

Error Reduction

Eliminate human error from routine processes with consistent automation.

Improve accuracy and reliability across your workflows.

workflow.json

{
  "nodes": [
    {
      "id": "6cdc45e5-1fa4-47fe-b80a-0e1560996936",
      "name": "Text",
      "type": "@n8n/n8n-nodes-langchain.toolWorkflow",
      "position": [
        1460,
        980
      ],
      "parameters": {
        "name": "text_retrieval_tool",
        "source": "parameter",
        "description": "Call this tool to return all text from the given website. Query should be full website URL.",
        "workflowJson": "{\n \"nodes\": [\n {\n \"parameters\": {},\n \"id\": \"05107436-c9cb-419b-ae8a-b74d309a130d\",\n \"name\": \"Execute workflow\",\n \"type\": \"n8n-nodes-base.manualTrigger\",\n \"typeVersion\": 1,\n \"position\": [\n 2220,\n 620\n ]\n },\n {\n \"parameters\": {\n \"assignments\": {\n \"assignments\": [\n {\n \"id\": \"253c2b17-c749-4f0a-93e8-5ff74f1ce49b\",\n \"name\": \"domain\",\n \"value\": \"={{ $json.query }}\",\n \"type\": \"string\"\n }\n ]\n },\n \"options\": {}\n },\n \"id\": \"bb8be616-3227-4705-8520-1827069faacd\",\n \"name\": \"Set domain\",\n \"type\": \"n8n-nodes-base.set\",\n \"typeVersion\": 3.3,\n \"position\": [\n 2440,\n 620\n ]\n },\n {\n \"parameters\": {\n \"assignments\": {\n \"assignments\": [\n {\n \"id\": \"ed0f1505-82b6-4393-a0d8-088055137ec9\",\n \"name\": \"domain\",\n \"value\": \"={{ $json.domain.startsWith(\\\"http\\\") ? $json.domain : \\\"http://\\\" + $json.domain }}\",\n \"type\": \"string\"\n }\n ]\n },\n \"options\": {}\n },\n \"id\": \"bdf29340-f135-489f-848e-1c7fa43a01df\",\n \"name\": \"Add protocool to domain\",\n \"type\": \"n8n-nodes-base.set\",\n \"typeVersion\": 3.3,\n \"position\": [\n 2640,\n 620\n ]\n },\n {\n \"parameters\": {\n \"assignments\": {\n \"assignments\": [\n {\n \"id\": \"2b1c7ff8-06a7-448b-99b7-5ede4b2e0bf0\",\n \"name\": \"response\",\n \"value\": \"={{ $json.data }}\",\n \"type\": \"string\"\n }\n ]\n },\n \"options\": {}\n },\n \"id\": \"9f0aa264-08c1-459a-bb99-e28599fe8f76\",\n \"name\": \"Set response\",\n \"type\": \"n8n-nodes-base.set\",\n \"typeVersion\": 3.3,\n \"position\": [\n 3300,\n 620\n ]\n },\n {\n \"parameters\": {\n \"url\": \"={{ $json.domain }}\",\n \"options\": {}\n },\n \"id\": \"cec7c8e8-bf5e-43d5-aa41-876293dbec78\",\n \"name\": \"Get website\",\n \"type\": \"n8n-nodes-base.httpRequest\",\n \"typeVersion\": 4.2,\n \"position\": [\n 2860,\n 620\n ]\n },\n {\n \"parameters\": {\n \"html\": \"={{ $json.data }}\",\n \"options\": {\n \"ignore\": \"a,img\"\n }\n },\n \"id\": \"1af94fcb-bca3-45c4-9277-18878c75d417\",\n \"name\": \"Convert HTML to Markdown\",\n \"type\": \"n8n-nodes-base.markdown\",\n \"typeVersion\": 1,\n \"position\": [\n 3080,\n 620\n ]\n }\n ],\n \"connections\": {\n \"Execute workflow\": {\n \"main\": [\n [\n {\n \"node\": \"Set domain\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Set domain\": {\n \"main\": [\n [\n {\n \"node\": \"Add protocool to domain\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Add protocool to domain\": {\n \"main\": [\n [\n {\n \"node\": \"Get website\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Get website\": {\n \"main\": [\n [\n {\n \"node\": \"Convert HTML to Markdown\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n },\n \"Convert HTML to Markdown\": {\n \"main\": [\n [\n {\n \"node\": \"Set response\",\n \"type\": \"main\",\n \"index\": 0\n }\n ]\n ]\n }\n },\n \"pinData\": {}\n}",
        "requestOptions": {}
      },
      "typeVersion": 1.1
    },
    {
...