LangFuse for LLM Observability: Tracing Token Costs, Latency, and Quality Scores Across Production Chains
When your LLM application handles thousands of requests daily, you need production observability — not just console.log. LangFuse is an open-source LLM observability platform that traces every generation, retrieval, and tool call.
1. Instrumenting with LangFuse
from langfuse import Langfuse
from langfuse.decorators import observe, langfuse_context
langfuse = Langfuse()
@observe()
def rag_pipeline(question: str) -> str:
# Retrieval step (automatically traced)
langfuse_context.update_current_observation(name="retrieval")
docs = vector_store.similarity_search(question, k=5)
# Generation step
langfuse_context.update_current_observation(name="generation")
response = llm.invoke(format_prompt(question, docs))
# Score the output
langfuse_context.score_current_trace(
name="relevance",
value=evaluate_relevance(question, response),
comment="Automated relevance evaluation"
)
return response
2. Key Metrics Dashboard
| Metric | What It Tells You |
|---|---|
| P95 Latency | User experience — are responses fast enough? |
| Cost per Conversation | Unit economics — is this profitable? |
| Quality Score Distribution | Are outputs meeting quality thresholds? |
| Token Waste Ratio | Are prompts efficiently designed? |
| Error Rate by Model | Which model needs attention? |
LangFuse transforms LLM applications from black boxes into observable, optimizable systems with clear cost and quality metrics.
