LangChain: Building Production LLM Applications
LangChain emerged in late 2022 as a response to a growing problem: building LLM applications required repeating the same boilerplate — prompt templating, provider abstraction, memory management, tool integration — across projects. Founder Harrison Chase created LangChain to provide composable abstractions that let developers focus on application logic rather than infrastructure.
The framework has grown into the most widely adopted LLM development toolkit, downloaded over 100 million times and used by companies including Salesforce, Ray, and Elastic. Its ecosystem has expanded to include LangSmith (observability and evaluation), LangGraph (agent orchestration), and LangServe (deployment). This guide covers the core abstractions and production patterns for building with LangChain.
Core Abstractions: Models, Prompts, and Output Parsers
LangChain’s foundation is the model abstraction layer. ChatOpenAI, ChatAnthropic, ChatOllama, ChatGoogleGenerativeAI, and dozens of others provide a consistent interface regardless of the underlying provider. Switching from GPT-4o to Claude 3.5 requires changing one line of code — the rest of your application remains unchanged.
Prompt templates parameterize prompts using ChatPromptTemplate. Instead of string formatting, LangChain uses message-based templates where each message has a role (system, human, AI) and content. The template handles variable interpolation, message formatting, and type conversion automatically. LangChain also supports partial formatting (pre-filling some variables) and few-shot example selection with FewShotPromptTemplate.
Output parsers convert raw model responses into structured data. StrOutputParser returns plain text. CommaSeparatedListOutputParser parses comma-delimited output. JsonOutputParser and PydanticOutputParser extract structured data with validation. DatetimeOutputParser parses date strings. Custom parsers implement the BaseOutputParser interface for application-specific formats.
The LCEL (LangChain Expression Language) composes these components into runnable pipelines using the pipe operator: chain = prompt | model | parser. LCEL provides automatic streaming, batching, async support, and observability. It’s more than syntactic sugar — the runtime traces execution, handles errors, and optimizes parallel execution of independent steps.
Chains: Composing Multi-Step Workflows
Chains combine multiple steps into a single pipeline. LLMChain is the simplest — prompt + model + parser. SequentialChain passes the output of one step as input to the next. RouterChain selects between multiple downstream chains based on the input. TransformChain applies arbitrary Python functions within a chain.
LangChain’s RunnablePassthrough and RunnableLambda let you inject Python code into LCEL pipelines. For example, extract relevant fields from intermediate outputs, call external APIs, or conditionally branch between chains. This makes LCEL Turing-complete — any workflow expressible in Python can be represented as a runnable graph.
Production chains should include: retry logic (LangChain supports exponential backoff and fallback models), timeout per step, and structured error messages that propagate up the chain for debugging. Use RunnableBranch for conditional routing — if one model fails, route to a fallback model. These patterns prevent cascading failures in multi-step pipelines.
RAG Integration
LangChain’s RAG support is the most comprehensive among LLM frameworks. The document pipeline handles ingestion: document loaders (PDF, HTML, CSV, Notion, Confluence, S3, YouTube transcripts, and 100+ others), text splitters (RecursiveCharacterTextSplitter, SemanticChunker, MarkdownHeaderTextSplitter, CodeSplitter), and embedding integration (OpenAI, Cohere, Hugging Face, Ollama).
Text splitting is critical for retrieval quality. RecursiveCharacterTextSplitter with chunk_size=1000 and chunk_overlap=200 is the safe default. SemanticChunker uses embedding similarity to split at natural semantic boundaries — better quality but slower. MarkdownHeaderTextSplitter preserves document structure by splitting on headers, maintaining section hierarchy as metadata.
Vector store integration supports 30+ databases through a unified VectorStore interface. The from_documents class method handles chunk embedding and indexing in one call. similarity_search returns chunks with relevance scores. The as_retriever() method wraps the store with configurable search parameters — k (number of results), score_threshold (minimum relevance), and fetch_k (candidates for MMR reranking).
The standard RAG chain: load → split → embed → index → retrieve → format → generate. LangChain’s create_retrieval_chain and create_stuff_documents_chain combine retrieval and generation into a ready-to-use pipeline. The “stuff” method inserts all retrieved documents into a single prompt — simple but limited by context window. For longer contexts, use create_map_reduce_chain (summarize each document, then combine summaries) or create_refine_chain (iterate through documents, refining the answer with each).
Agents in LangChain
LangChain’s agent framework, built on LangGraph in recent versions, provides stateful, multi-step execution with tool use. The create_react_agent function creates a ReAct agent with configurable system prompt, tools, and message history. LangGraph adds: persistent state across steps, cyclic execution graphs (agents can revisit previous steps), human-in-the-loop checkpoints, and conditional routing.
LangGraph’s StateGraph is the most flexible agent framework available. You define nodes (functions that process state) and edges (conditional or fixed transitions between nodes). The graph executes as a state machine — each node reads and writes to a shared state dictionary. This enables complex patterns: supervisor agents that delegate to worker agents, reflection loops (generate → critique → revise), and hierarchical planning (generate plan → execute steps → verify results).
Agent tools in LangChain include: TavilySearchResults (web search), DuckDuckGoSearchRun, WikipediaQueryRun, ArxivQueryRun, PythonREPLTool (sandboxed code execution), ShellTool, FileReadTool, DatabaseTool (SQL query execution), and custom tools via the @tool decorator. Each tool has a name, description, and parameter schema — the agent’s LLM uses these to decide when and how to call tools.
Memory and Conversation Management
LangChain’s memory module provides conversation history management. ConversationBufferMemory stores the full history. ConversationSummaryMemory periodically summarizes older turns. ConversationBufferWindowMemory keeps only the last k exchanges. VectorStoreRetrieverMemory stores embeddings of past conversations in a vector database for semantic retrieval.
Memory is integrated into chains via the memory parameter. For multi-turn conversations, use ConversationSummaryMemory with return_messages=True. The summary is regenerated every N turns using a separate LLM call. For personalized experiences, use VectorStoreRetrieverMemory — the agent retrieves relevant past conversations, enabling cross-session continuity.
LangChain also supports entity memory (ConversationEntityMemory) that extracts and remembers named entities (people, places, products, preferences) across sessions. This is useful for CRM applications where the agent must remember customer-specific information.
Production with LangSmith
LangSmith is LangChain’s observability platform for LLM applications. It traces every step of chain and agent execution — prompt templates, model calls, tool invocations, and intermediate outputs. Each trace includes latency, token usage, cost, and error information. Traces are organized by project, session, and run.
LangSmith’s evaluation framework allows you to define custom metrics and run evaluation datasets against chain variants. It supports: pairwise comparison (which output is better?), scored evaluation (rate output on 1-5 scale), and automated evaluation (LLM-as-judge, exact match, JSON validity). Evaluation datasets can be created from production traces, enabling continuous improvement pipelines.
The feedback loop closes when you export evaluation results, improve prompts or chains, deploy the new version, and measure the improvement. LangSmith supports A/B testing by routing different user sessions to different chain versions and comparing outcome metrics.
LangChain vs Other Frameworks
LangChain competes with several other LLM development frameworks. LlamaIndex specializes in RAG and data indexing with a more opinionated API for document processing. Haystack by deepset focuses on NLP pipelines with strong support for traditional NLP tasks alongside LLMs. Semantic Kernel by Microsoft integrates deeply with the Azure ecosystem and C# applications.
LangChain’s advantage is its breadth — it covers more integrations, more tools, and more deployment options than any alternative. Its disadvantage is the learning curve — the framework has many abstractions and the API has changed significantly across major versions. For teams already in the Python ecosystem, LangChain is usually the best starting point.
Best Practices
Version control prompts alongside code. LangChain’s Hub (a centralized prompt registry) and PromptTemplate serialization make this straightforward. Test prompt changes against a curated evaluation dataset before deploying to production. Monitor for regressions — a prompt that improves average quality might degrade edge cases.
Cache expensive operations. LangChain provides BaseCache implementations for Redis, in-memory, and SQLite. Cache embedding results (same text produces same embedding), LLM responses for identical queries (semantic caching with embedding similarity), and document load results. Cache invalidation should be time-based or event-driven.
Handle provider failures gracefully. LangChain’s fallback mechanism lets you specify a list of models: ChatOpenAI(model="gpt-4o").with_fallbacks([ChatAnthropic(model="claude-3-5-sonnet")]). If the primary provider fails (rate limit, outage, timeout), the fallback is invoked automatically. This is critical for production systems where downtime is unacceptable.
FAQ
Should I use LangChain or build on raw API calls? Use raw API calls for simple applications (single model call, no chaining). Use LangChain when you need: multi-step chains, agent loops, integration with multiple providers, RAG pipelines, or production observability. LangChain adds abstraction overhead — about 10-20% latency for simple calls — but saves much more in development time for complex applications.
What is LangGraph and how is it different from LangChain? LangGraph extends LangChain with stateful, cyclic execution graphs suitable for agent systems. LangChain chains are DAGs (directed acyclic graphs) — data flows in one direction. LangGraph supports cycles (agents that revisit previous states), persistent state, and human-in-the-loop checkpoints. Use LangChain for simple chains and RAG; use LangGraph for complex agents.
How do I handle token limits in LangChain? Use add_start_index=True in document loaders to track token positions. Use RecursiveCharacterTextSplitter with appropriate chunk size. For LLM calls, set max_tokens in the model configuration. LangChain’s Runnable interface supports automatic truncation — set truncation=True in the prompt template to silently truncate inputs that exceed the model’s context window.
Is LangChain production-ready? Yes, with careful engineering. LangChain 0.3+ (as of late 2024) has a stable, well-tested core. LangSmith provides production observability. However, avoid experimental modules (LangChain 0.0.x-era agents, experimental chains) in production. Stick to LCEL, LangGraph, and well-maintained integrations.
How do I test LangChain chains? Unit test each component independently (prompt templates, output parsers, tools) and integration test the full chain with representative inputs. Use LangSmith to create evaluation datasets and run automated evaluations against chain versions. Test with edge cases: empty inputs, extremely long inputs, missing context, ambiguous queries, and known adversarial inputs.