---
id: gh-firecrawl-knowledge-ingest
name: "firecrawl-knowledge-ingest"
url: https://skills.yangsir.net/skill/gh-firecrawl-knowledge-ingest
author: firecrawl
domain: ai-data-management-analysis
tags: ["web-scraping", "data-extraction", "knowledge-management", "browser-automation", "documentation"]
install_count: 31300
rating: 4.60 (120 reviews)
github: https://github.com/firecrawl/firecrawl-workflows/tree/main/skills/firecrawl-knowledge-ingest
---

# firecrawl-knowledge-ingest

> 利用 Firecrawl 浏览器抓取公共或需认证的知识库和文档门户，支持处理 JS 渲染、登录验证、分页内容，并能提取结构化 JSON/Markdown 数据，用于构建和管理知识库。

**Stats**: 31,300 installs · 4.6/5 (120 reviews)

## Before / After 对比

### 知识库内容自动化抓取

**Before**:

人工从包含大量JS、需要登录或多页的文档网站收集知识，耗时费力且容易遗漏关键信息，难以保持数据更新。

**After**:

自动高效地从复杂文档门户抓取结构化知识，确保数据完整性和时效性，大幅减少人工干预和错误。

| Metric | Before | After | Change |
|---|---|---|---|
| 知识库内容抓取时间 | 60分钟 | 5分钟 | -92% |

## Readme

# Firecrawl Knowledge Ingest

Use this when a docs portal needs browser navigation, auth, pagination, or JS rendering.

## Onboarding Interview

Infer the portal URL, output format, auth needs, and page limit from context. If the portal is clear, proceed immediately.

Ask at most 1-3 concise questions only if blocked, such as the portal URL, whether authentication is required, or the desired output format.

## Firecrawl Collection Plan

Use Firecrawl browser to:

- open the portal and inspect navigation
- identify sections, categories, sidebar links, and article URLs
- follow sidebar navigation, next links, pagination, load-more controls, or search
- scrape article content as markdown
- extract metadata such as title, section, last updated date, author, and tags

Try Firecrawl map as a supplement for public URLs, but use browser navigation for auth-gated or JS-heavy content.

## Final Deliverable

```markdown
# Knowledge Ingest: [Portal]

## Summary
[Pages extracted, sections covered, limitations]

## Output
[JSON/markdown/merged file path or content]

## Sections
[Section names and article counts]

## Failed Or Restricted Pages
[Any access/loading issues]

## Sources
[URLs extracted]

## Rerun Inputs
workflow: firecrawl-knowledge-ingest
url: [portal url]
format: [json/markdown/merged]
max_pages: [number]
```

## JSON Shape

Use `source`, `url`, `extractedAt`, `totalArticles`, and `sections[]` with article `title`, `url`, `section`, `content`, and `metadata`.

## Quality Bar

- Preserve code examples, tables, and formatting.
- Strip nav chrome, headers, and footers.
- Track extraction progress and page failures.
- Respect authentication boundaries.


---
*Source: https://skills.yangsir.net/skill/gh-firecrawl-knowledge-ingest*
*Markdown mirror: https://skills.yangsir.net/api/skill/gh-firecrawl-knowledge-ingest/markdown*