Chapter 5: Production Deployment

Learning Objectives

• Use LangSmith for monitoring and debugging
• Understand when to migrate from LCEL to LangGraph
• Integrate MCP (Model Context Protocol) tools
• Apply production best practices and optimizations
• Plan next steps for advanced LangChain development

5.1 LangSmith: Observability for LLM Applications

LangSmith is LangChain's platform for tracing, debugging, and monitoring LLM applications in production.

Setting Up LangSmith

import os

# Set environment variables (or use .env file)
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = "your-langsmith-api-key"
os.environ["LANGCHAIN_PROJECT"] = "my-project"  # Optional: organize traces

# Now all LangChain operations are automatically traced!
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

llm = ChatOpenAI(model="gpt-4")
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful assistant."),
    ("human", "{question}")
])

chain = prompt | llm

# This invocation is automatically logged to LangSmith
response = chain.invoke({"question": "What is the capital of France?"})
print(response.content)

# View traces at: https://smith.langchain.com

What LangSmith Shows You

• Full trace hierarchy: See every step in your chain
• Latency breakdown: Identify slow components
• Token usage: Track costs per request
• Error details: Debug failures with full context
• Input/Output pairs: Review what went in and came out

5.2 When to Use LangGraph

LangGraph is LangChain's framework for building complex, stateful agent workflows. Use it when you outgrow simple LCEL chains.

LCEL vs LangGraph Decision Matrix

# LangGraph enables complex workflows with state
from langgraph.graph import StateGraph, END
from typing import TypedDict, Annotated
import operator

# Define state schema
class AgentState(TypedDict):
    messages: Annotated[list, operator.add]
    next_step: str

# Create graph
workflow = StateGraph(AgentState)

# Add nodes (each node is a function)
workflow.add_node("research", research_node)
workflow.add_node("write", write_node)
workflow.add_node("review", review_node)

# Add edges (define flow)
workflow.add_edge("research", "write")
workflow.add_conditional_edges(
    "write",
    should_continue,  # Function that returns next node
    {"review": "review", "end": END}
)

# Compile and run
app = workflow.compile()
result = app.invoke({"messages": ["Write a blog post about AI"]})

5.3 MCP (Model Context Protocol) Integration

MCP is an open protocol for standardizing how applications provide tools to LLMs. LangChain supports MCP through adapters:

from langchain_mcp_adapters import MultiServerMCPClient
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI

# Connect to MCP servers
async with MultiServerMCPClient(
    {
        "filesystem": {
            "command": "npx",
            "args": ["-y", "@anthropic/mcp-filesystem", "/path/to/allowed/dir"]
        },
        "web": {
            "url": "http://localhost:8080/mcp"  # HTTP-based MCP server
        }
    }
) as client:
    # Get tools from all connected servers
    tools = client.get_tools()

    # Create agent with MCP tools
    agent = create_agent(
        model=ChatOpenAI(model="gpt-4"),
        tools=tools,
        system_prompt="You can read files and browse the web."
    )

    result = await agent.ainvoke({
        "messages": [{"role": "user", "content": "Read the README.md file"}]
    })

5.4 Production Best Practices

Performance Optimization

import asyncio
from langchain_openai import ChatOpenAI
from langchain_core.globals import set_llm_cache
from langchain_community.cache import SQLiteCache

# Enable caching to avoid redundant API calls
set_llm_cache(SQLiteCache(database_path=".langchain.db"))

# Use streaming for better UX
llm = ChatOpenAI(model="gpt-4", streaming=True)

# Async for better throughput
async def process_many(questions):
    tasks = [llm.ainvoke(q) for q in questions]
    return await asyncio.gather(*tasks)

Error Handling

from langchain_openai import ChatOpenAI
from langchain_core.runnables import RunnableWithFallbacks

# Create fallback chain
primary = ChatOpenAI(model="gpt-4")
fallback = ChatOpenAI(model="gpt-3.5-turbo")

robust_llm = primary.with_fallbacks([fallback])

# Will try gpt-4, then gpt-3.5-turbo if it fails
response = robust_llm.invoke("Hello")

Security Checklist

• ✅ Store API keys in environment variables or secrets manager
• ✅ Validate and sanitize user inputs before passing to LLM
• ✅ Set token limits to prevent runaway costs
• ✅ Use rate limiting for public-facing APIs
• ✅ Log and monitor for unusual patterns
• ✅ Review tool permissions (especially for agents with shell/file access)

5.5 Next Steps

Congratulations on completing this introduction! Here's where to go next:

Deepen Your Knowledge

• LangGraph: Build complex multi-agent systems with cycles and state
• Advanced RAG: Explore reranking, query expansion, and hybrid search
• Evaluation: Use LangSmith to systematically evaluate your chains
• Custom Models: Integrate local LLMs with Ollama or vLLM

Resources

• LangChain Documentation
• LangChain GitHub
• LangSmith Platform
• LangGraph Documentation

Exercise

Summary

• LangSmith provides tracing, debugging, and monitoring
• Use LangGraph for complex workflows with branching, cycles, or state
• MCP is the emerging standard for tool integration
• Production apps need caching, fallbacks, and error handling
• Always consider security: input validation, rate limiting, monitoring