---
id: gh-firecrawl-knowledge-base
name: "firecrawl-knowledge-base"
url: https://skills.yangsir.net/skill/gh-firecrawl-knowledge-base
author: firecrawl
domain: ai-data-management-analysis
tags: ["knowledge-base", "web-scraping", "rag", "llm-data", "content-extraction"]
install_count: 32500
rating: 4.60 (120 reviews)
github: https://github.com/firecrawl/firecrawl-workflows/tree/main/skills/firecrawl-knowledge-base
---

# firecrawl-knowledge-base

> Firecrawl Knowledge Base 技能可从网页内容构建知识库。它能将 URL 或主题转化为结构化的、LLM 就绪的内容，用于本地参考文档、RAG 块、微调数据集、文档镜像或主题语料库。通过 Firecrawl 抓取网页并转换为 Markdown 格式，同时保留代码示例和表格，支持多种输出模式，确保高质量的知识提取和组织。

**Stats**: 32,500 installs · 4.6/5 (120 reviews)

## Before / After 对比

### LLM 数据准备效率

**Before**:

手动从网页收集、清洗和格式化内容以构建 RAG 或微调数据集，耗时且易出错，需要大量人工干预。

**After**:

自动抓取网页内容，智能转换为 LLM 就绪的 Markdown 格式，大幅缩短数据准备时间，提高数据质量和一致性。

| Metric | Before | After | Change |
|---|---|---|---|
| 数据准备时长 | 8小时 | 0.5小时 | -94% |

## Readme

# Firecrawl Knowledge Base

Use this to turn URLs or topics into organized LLM-ready content.

## Onboarding Interview

Infer the source, goal, depth, and output location from context. If the source and goal are clear, proceed immediately.

Ask at most 1-3 concise questions only if blocked, such as the source URL/topic, whether the output is reference/RAG/training/docs, or training format if training is requested.

## Firecrawl Collection Plan

Use Firecrawl map for documentation sites, search for topic-based corpora, scrape pages into markdown, and preserve code examples and tables.

For files, follow the Firecrawl download-style convention:

```text
.firecrawl/
  <hostname>/
    <path>/
      index.md
```

## Parallel Work

If appropriate, use sub-agents or equivalent parallel task runners:

- one docs section per researcher
- official docs, tutorials, community discussions, and references by source type
- source scraping vs chunk generation vs manifest generation

## Output Modes

- Reference: markdown files, `index.md`, and `sources.json`.
- RAG: markdown files plus chunk files and `manifest.json`.
- Training: scraped source files plus `training-data.jsonl` and `training-metadata.json`.
- Docs mirror: complete markdown mirror with a table of contents.

## Final Deliverable

```markdown
# Knowledge Base: [Source]

## Summary
[What was collected and why]

## Output Structure
[Files/directories created]

## Coverage
[Sections, source types, counts]

## Usage Notes
[How to use in RAG, docs, training, or agent context]

## Sources
[URLs collected]

## Rerun Inputs
workflow: firecrawl-knowledge-base
source: [url/topic]
goal: [reference/rag/train/docs]
depth: [quick/thorough/exhaustive]
output_dir: [.firecrawl/]
```

## Quality Bar

- Preserve code examples and formatting.
- Remove boilerplate navigation where possible.
- Include source URLs in frontmatter or metadata.


---
*Source: https://skills.yangsir.net/skill/gh-firecrawl-knowledge-base*
*Markdown mirror: https://skills.yangsir.net/api/skill/gh-firecrawl-knowledge-base/markdown*