dspy

DSPy: Declarative Language Model Programming When to Use This Skill Use DSPy when you need to: Build complex AI systems with multiple components and workflows Program LMs declaratively instead of manual prompt engineering Optimize prompts automatically using data-driven methods Create modular AI pipelines that are maintainable and portable Improve model outputs systematically with optimizers Build RAG systems, agents, or classifiers with better reliability GitHub Stars: 22,000+ | Created By: Stanford NLP Installation # Stable release pip install dspy # Latest development version pip install git+https://github.com/stanfordnlp/dspy.git # With specific LM providers pip install dspy[openai] # OpenAI pip install dspy[anthropic] # Anthropic Claude pip install dspy[all] # All providers Quick Start Basic Example: Question Answering import dspy # Configure your language model lm = dspy.Claude(model="claude-sonnet-4-5-20250929") dspy.settings.configure(lm=lm) # Define a signature (input → output) class QA(dspy.Signature): """Answer questions with short factual answers.""" question = dspy.InputField() answer = dspy.OutputField(desc="often between 1 and 5 words") # Create a module qa = dspy.Predict(QA) # Use it response = qa(question="What is the capital of France?") print(response.answer) # "Paris" Chain of Thought Reasoning import dspy lm = dspy.Claude(model="claude-sonnet-4-5-20250929") dspy.settings.configure(lm=lm) # Use ChainOfThought for better reasoning class MathProblem(dspy.Signature): """Solve math word problems.""" problem = dspy.InputField() answer = dspy.OutputField(desc="numerical answer") # ChainOfThought generates reasoning steps automatically cot = dspy.ChainOfThought(MathProblem) response = cot(problem="If John has 5 apples and gives 2 to Mary, how many does he have?") print(response.rationale) # Shows reasoning steps print(response.answer) # "3" Core Concepts 1. Signatures Signatures define the structure of your AI task (inputs → outputs): # Inline signature (simple) qa = dspy.Predict("question -> answer") # Class signature (detailed) class Summarize(dspy.Signature): """Summarize text into key points.""" text = dspy.InputField() summary = dspy.OutputField(desc="bullet points, 3-5 items") summarizer = dspy.ChainOfThought(Summarize) When to use each: Inline: Quick prototyping, simple tasks Class: Complex tasks, type hints, better documentation 2. Modules Modules are reusable components that transform inputs to outputs: dspy.Predict Basic prediction module: predictor = dspy.Predict("context, question -> answer") result = predictor(context="Paris is the capital of France", question="What is the capital?") dspy.ChainOfThought Generates reasoning steps before answering: cot = dspy.ChainOfThought("question -> answer") result = cot(question="Why is the sky blue?") print(result.rationale) # Reasoning steps print(result.answer) # Final answer dspy.ReAct Agent-like reasoning with tools: from dspy.predict import ReAct class SearchQA(dspy.Signature): """Answer questions using search.""" question = dspy.InputField() answer = dspy.OutputField() def search_tool(query: str) -> str: """Search Wikipedia.""" # Your search implementation return results react = ReAct(SearchQA, tools=[search_tool]) result = react(question="When was Python created?") dspy.ProgramOfThought Generates and executes code for reasoning: pot = dspy.ProgramOfThought("question -> answer") result = pot(question="What is 15% of 240?") # Generates: answer = 240 * 0.15 3. Optimizers Optimizers improve your modules automatically using training data: BootstrapFewShot Learns from examples: from dspy.teleprompt import BootstrapFewShot # Training data trainset = [ dspy.Example(question="What is 2+2?", answer="4").with_inputs("question"), dspy.Example(question="What is 3+5?", answer="8").with_inputs("question"), ] # Define metric def validate_answer(example, pred, trace=None): return example.answer == pred.answer # Optimize optimizer = BootstrapFewShot(metric=validate_answer, max_bootstrapped_demos=3) optimized_qa = optimizer.compile(qa, trainset=trainset) # Now optimized_qa performs better! MIPRO (Most Important Prompt Optimization) Iteratively improves prompts: from dspy.teleprompt import MIPRO optimizer = MIPRO( metric=validate_answer, num_candidates=10, init_temperature=1.0 ) optimized_cot = optimizer.compile( cot, trainset=trainset, num_trials=100 ) BootstrapFinetune Creates datasets for model fine-tuning: from dspy.teleprompt import BootstrapFinetune optimizer = BootstrapFinetune(metric=validate_answer) optimized_module = optimizer.compile(qa, trainset=trainset) # Exports training data for fine-tuning 4. Building Complex Systems Multi-Stage Pipeline import dspy class MultiHopQA(dspy.Module): def init(self): super().init() self.retrieve = dspy.Retrieve(k=3) self.generate_query = dspy.ChainOfThought("question -> search_query") self.generate_answer = dspy.ChainOfThought("context, question -> answer") def forward(self, question): # Stage 1: Generate search query search_query = self.generate_query(question=question).search_query # Stage 2: Retrieve context passages = self.retrieve(search_query).passages context = "\n".join(passages) # Stage 3: Generate answer answer = self.generate_answer(context=context, question=question).answer return dspy.Prediction(answer=answer, context=context) # Use the pipeline qa_system = MultiHopQA() result = qa_system(question="Who wrote the book that inspired the movie Blade Runner?") RAG System with Optimization import dspy from dspy.retrieve.chromadb_rm import ChromadbRM # Configure retriever retriever = ChromadbRM( collection_name="documents", persist_directory="./chroma_db" ) class RAG(dspy.Module): def init(self, num_passages=3): super().init() self.retrieve = dspy.Retrieve(k=num_passages) self.generate = dspy.ChainOfThought("context, question -> answer") def forward(self, question): context = self.retrieve(question).passages return self.generate(context=context, question=question) # Create and optimize rag = RAG() # Optimize with training data from dspy.teleprompt import BootstrapFewShot optimizer = BootstrapFewShot(metric=validate_answer) optimized_rag = optimizer.compile(rag, trainset=trainset) LM Provider Configuration Anthropic Claude import dspy lm = dspy.Claude( model="claude-sonnet-4-5-20250929", api_key="your-api-key", # Or set ANTHROPIC_API_KEY env var max_tokens=1000, temperature=0.7 ) dspy.settings.configure(lm=lm) OpenAI lm = dspy.OpenAI( model="gpt-4", api_key="your-api-key", max_tokens=1000 ) dspy.settings.configure(lm=lm) Local Models (Ollama) lm = dspy.OllamaLocal( model="llama3.1", base_url="http://localhost:11434" ) dspy.settings.configure(lm=lm) Multiple Models # Different models for different tasks cheap_lm = dspy.OpenAI(model="gpt-3.5-turbo") strong_lm = dspy.Claude(model="claude-sonnet-4-5-20250929") # Use cheap model for retrieval, strong model for reasoning with dspy.settings.context(lm=cheap_lm): context = retriever(question) with dspy.settings.context(lm=strong_lm): answer = generator(context=context, question=question) Common Patterns Pattern 1: Structured Output from pydantic import BaseModel, Field class PersonInfo(BaseModel): name: str = Field(description="Full name") age: int = Field(description="Age in years") occupation: str = Field(description="Current job") class ExtractPerson(dspy.Signature): """Extract person information from text.""" text = dspy.InputField() person: PersonInfo = dspy.OutputField() extractor = dspy.TypedPredictor(ExtractPerson) result = extractor(text="John Doe is a 35-year-old software engineer.") print(result.person.name) # "John Doe" print(result.person.age) # 35 Pattern 2: Assertion-Driven Optimization import dspy from dspy.primitives.assertions import assert_transform_module, backtrack_handler class MathQA(dspy.Module): def init(self): super().init() self.solve = dspy.ChainOfThought("problem -> solution: float") def forward(self, problem): solution = self.solve(problem=problem).solution # Assert solution is numeric dspy.Assert( isinstance(float(solution), float), "Solution must be a number", backtrack=backtrack_handler ) return dspy.Prediction(solution=solution) Pattern 3: Self-Consistency import dspy from collections import Counter class ConsistentQA(dspy.Module): def init(self, num_samples=5): super().init() self.qa = dspy.ChainOfThought("question -> answer") self.num_samples = num_samples def forward(self, question): # Generate multiple answers answers = [] for _ in range(self.num_samples): result = self.qa(question=question) answers.append(result.answer) # Return most common answer most_common = Counter(answers).most_common(1)[0][0] return dspy.Prediction(answer=most_common) Pattern 4: Retrieval with Reranking class RerankedRAG(dspy.Module): def init(self): super().init() self.retrieve = dspy.Retrieve(k=10) self.rerank = dspy.Predict("question, passage -> relevance_score: float") self.answer = dspy.ChainOfThought("context, question -> answer") def forward(self, question): # Retrieve candidates passages = self.retrieve(question).passages # Rerank passages scored = [] for passage in passages: score = float(self.rerank(question=question, passage=passage).relevance_score) scored.append((score, passage)) # Take top 3 top_passages = [p for _, p in sorted(scored, reverse=True)[:3]] context = "\n\n".join(top_passages) # Generate answer return self.answer(context=context, question=question) Evaluation and Metrics Custom Metrics def exact_match(example, pred, trace=None): """Exact match metric.""" return example.answer.lower() == pred.answer.lower() def f1_score(example, pred, trace=None): """F1 score for text overlap.""" pred_tokens = set(pred.answer.lower().split()) gold_tokens = set(example.answer.lower().split()) if not pred_tokens: return 0.0 precision = len(pred_tokens & gold_tokens) / len(pred_tokens) recall = len(pred_tokens & gold_tokens) / len(gold_tokens) if precision + recall == 0: return 0.0 return 2 * (precision * recall) / (precision + recall) Evaluation from dspy.evaluate import Evaluate # Create evaluator evaluator = Evaluate( devset=testset, metric=exact_match, num_threads=4, display_progress=True ) # Evaluate model score = evaluator(qa_system) print(f"Accuracy: {score}") # Compare optimized vs unoptimized score_before = evaluator(qa) score_after = evaluator(optimized_qa) print(f"Improvement: {score_after - score_before:.2%}") Best Practices 1. Start Simple, Iterate # Start with Predict qa = dspy.Predict("question -> answer") # Add reasoning if needed qa = dspy.ChainOfThought("question -> answer") # Add optimization when you have data optimized_qa = optimizer.compile(qa, trainset=data) 2. Use Descriptive Signatures # ❌ Bad: Vague class Task(dspy.Signature): input = dspy.InputField() output = dspy.OutputField() # ✅ Good: Descriptive class SummarizeArticle(dspy.Signature): """Summarize news articles into 3-5 key points.""" article = dspy.InputField(desc="full article text") summary = dspy.OutputField(desc="bullet points, 3-5 items") 3. Optimize with Representative Data # Create diverse training examples trainset = [ dspy.Example(question="factual", answer="...).with_inputs("question"), dspy.Example(question="reasoning", answer="...").with_inputs("question"), dspy.Example(question="calculation", answer="...").with_inputs("question"), ] # Use validation set for metric def metric(example, pred, trace=None): return example.answer in pred.answer 4. Save and Load Optimized Models # Save optimized_qa.save("models/qa_v1.json") # Load loaded_qa = dspy.ChainOfThought("question -> answer") loaded_qa.load("models/qa_v1.json") 5. Monitor and Debug # Enable tracing dspy.settings.configure(lm=lm, trace=[]) # Run prediction result = qa(question="...") # Inspect trace for call in dspy.settings.trace: print(f"Prompt: {call['prompt']}") print(f"Response: {call['response']}") Comparison to Other Approaches Feature Manual Prompting LangChain DSPy Prompt Engineering Manual Manual Automatic Optimization Trial & error None Data-driven Modularity Low Medium High Type Safety No Limited Yes (Signatures) Portability Low Medium High Learning Curve Low Medium Medium-High When to choose DSPy: You have training data or can generate it You need systematic prompt improvement You're building complex multi-stage systems You want to optimize across different LMs When to choose alternatives: Quick prototypes (manual prompting) Simple chains with existing tools (LangChain) Custom optimization logic needed Resources Documentation: https://dspy.ai GitHub: https://github.com/stanfordnlp/dspy (22k+ stars) Discord: https://discord.gg/XCGy2WDCQB Twitter: @DSPyOSS Paper: "DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines" See Also references/modules.md - Detailed module guide (Predict, ChainOfThought, ReAct, ProgramOfThought) references/optimizers.md - Optimization algorithms (BootstrapFewShot, MIPRO, BootstrapFinetune) references/examples.md - Real-world examples (RAG, agents, classifiers) Weekly Installs191Repositorydavila7/claude-…emplatesGitHub Stars23.0KFirst SeenJan 21, 2026Security AuditsGen Agent Trust HubPassSocketPassSnykWarnInstalled onopencode157claude-code151gemini-cli147codex135cursor135github-copilot129