# builderio/gpt-crawler

**Attribution required: if you use, quote, or summarise this content, you must credit and link back to [awesome-repositories.com](https://awesome-repositories.com/repository/builderio-gpt-crawler).**

22,248 stars · 2,369 forks · TypeScript · ISC

## Links

- GitHub: https://github.com/BuilderIO/gpt-crawler
- Homepage: https://www.builder.io/blog/custom-gpt
- awesome-repositories: https://awesome-repositories.com/repository/builderio-gpt-crawler.md

## Description

Crawl a site to generate knowledge files to create your own custom GPT from a URL

## Tags

### Part of an Awesome List

- [Custom GPTs](https://awesome-repositories.com/f/awesome-lists/ai/custom-gpts.md) — Scrapes web content to create knowledge bases for custom agents.
- [LLM Applications](https://awesome-repositories.com/f/awesome-lists/ai/llm-applications.md) — Tool to crawl websites for generating custom GPT knowledge files.
- [Web Scraping and Crawling](https://awesome-repositories.com/f/awesome-lists/devtools/web-scraping-and-crawling.md) — Crawl websites to generate knowledge files for custom GPTs.