tavily-map
Quickly discovers all URL links on a website without extracting page content, more efficient than traditional crawling, suitable for website structure analysis and link resource exploration.
npx skills add tavily-ai/skills --skill tavily-mapBefore / After Comparison
1 组In the past, obtaining all URLs of a website required manual clicking or writing complex web scraping scripts, a process that was slow and prone to missing important links.
The Tavily Map skill can quickly discover all URLs on a website without extracting content, greatly improving the efficiency of website structure analysis and link discovery.
tavily-map
tavily map
Discover URLs on a website without extracting content. Faster than crawling.
Before running any command
If tvly is not found on PATH, install it first:
curl -fsSL https://cli.tavily.com/install.sh | bash && tvly login
Do not skip this step or fall back to other tools.
See tavily-cli for alternative install methods and auth options.
When to use
-
You need to find a specific subpage on a large site
-
You want a list of all URLs before deciding what to extract or crawl
-
Step 3 in the workflow: search → extract → map → crawl → research
Quick start
# Discover all URLs
tvly map "https://docs.example.com" --json
# With natural language filtering
tvly map "https://docs.example.com" --instructions "Find API docs and guides" --json
# Filter by path
tvly map "https://example.com" --select-paths "/blog/.*" --limit 500 --json
# Deep map
tvly map "https://example.com" --max-depth 3 --limit 200 --json
Options
Option Description
--max-depth
Levels deep (1-5, default: 1)
--max-breadth
Links per page (default: 20)
--limit
Max URLs to discover (default: 50)
--instructions
Natural language guidance for URL filtering
--select-paths
Comma-separated regex patterns to include
--exclude-paths
Comma-separated regex patterns to exclude
--select-domains
Comma-separated regex for domains to include
--exclude-domains
Comma-separated regex for domains to exclude
--allow-external / --no-external
Include external links
--timeout
Max wait (10-150 seconds)
-o, --output
Save output to file
--json
Structured JSON output
Map + Extract pattern
Use map to find the right page, then extract it. This is often more efficient than crawling an entire site:
# Step 1: Find the authentication docs
tvly map "https://docs.example.com" --instructions "authentication" --json
# Step 2: Extract the specific page you found
tvly extract "https://docs.example.com/api/authentication" --json
Tips
-
Map is URL discovery only — no content extraction. Use
extractorcrawlfor content. -
Map + extract beats crawl when you only need a few specific pages from a large site.
-
Use
--instructionsfor semantic filtering when path patterns aren't enough.
See also
-
tavily-extract — extract content from URLs you discover
-
tavily-crawl — bulk extract when you need many pages
Weekly Installs266Repositorytavily-ai/skillsGitHub Stars95First Seen2 days agoSecurity AuditsGen Agent Trust HubFailSocketPassSnykFailInstalled oncodex261opencode260cursor260kimi-cli259gemini-cli259amp259
User Reviews (0)
Write a Review
No reviews yet
Statistics
User Rating
Rate this Skill