OpenAI API: Complete Developer's Guide

AI & LLMs AI & LLMs 8 min read 1609 words Beginner ExcellentWiki Editorial Team

The OpenAI API is the most widely used interface for LLM integration, powering applications from ChatGPT (200M+ users) to thousands of enterprise products. Its combination of model quality, API design, tool support, and reliability has made it the default choice for developers building AI applications.

Since the API’s launch in 2020, OpenAI has iterated rapidly. The current API (as of mid-2026) offers chat completions, embeddings, image generation (DALL-E 3), audio transcription (Whisper), text-to-speech (TTS), fine-tuning, and the Assistants API (stateful agent execution). This guide covers the full API surface with production-tested patterns.

Authentication and Client Setup

Authentication uses API keys, created and managed in the OpenAI dashboard. Best practices: set the key as an environment variable (OPENAI_API_KEY), never hardcode it, and use different keys for development and production. The official Python client (openai package) automatically reads the environment variable.

For production, implement key rotation — generate a new key monthly, run both keys in parallel during transition, then revoke the old key. Use organization-level keys for cost tracking across teams. OpenAI supports granular API key permissions — restrict keys to specific models or endpoints to limit blast radius from key leaks.

The client provides async (AsyncOpenAI) and synchronous (OpenAI) variants. Use async in web frameworks (FastAPI, Django) for non-blocking I/O. The client handles retry logic with exponential backoff (default: 2 retries), connection pooling, and timeout configuration. Configure timeouts: timeout=30.0 for chat, timeout=60.0 for streaming, timeout=120.0 for batch operations.

Chat Completions: The Core API

The chat completions endpoint (/v1/chat/completions) accepts a messages array and returns a model-generated response. Each message has a role — system (instructions, persona, constraints), user (the end user’s input), assistant (previous model responses, useful for few-shot examples or conversation history), and tool (results from tool/function calls).

The model parameter specifies which model to use. The messages array contains the full conversation history. Additional parameters control generation: temperature (0-2, default 1), top_p (0-1, default 1), max_tokens (maximum output length), stop (sequences that stop generation), presence_penalty (-2 to 2, discourages topic repetition), and frequency_penalty (-2 to 2, discourages token repetition).

Response format: the response object contains choices (array of generated completions), usage (prompt_tokens, completion_tokens, total_tokens), model (the model used), and system_fingerprint (a hash representing the model configuration). For n>1, multiple choices are returned with independent sampling.

Production optimization: set max_tokens to the minimum reasonable value for your use case — this reduces both latency and cost. For classification, 10-50 tokens is sufficient. For generation, estimate based on your typical output length and add 20% headroom. Never leave max_tokens unlimited unless you have cost protection in place.

Streaming for Real-Time Applications

Streaming is essential for responsive user experiences. Set stream=True in the request. The response becomes a generator yielding ChatCompletionChunk objects. Each chunk contains a choices array with a delta object (the incremental content). Accumulate deltas to reconstruct the full response.

The streaming API supports most features of non-streaming mode, including function calling (the final chunk includes the assembled tool_calls) and usage information (final chunk includes usage stats when stream_options={"include_usage": true}). The streaming response also supports stream_options={"include_usage": true} for token usage metadata.

Edge delivery optimizes streaming latency. Deploy your application on edge platforms (Cloudflare Workers, Vercel Edge) that terminate the streaming connection geographically close to the user. OpenAI’s API points are globally distributed — connect to the nearest endpoint (api.openai.com resolves to different IPs based on location). For users in Asia-Pacific, latency to US-west endpoints is 150-300ms; OpenAI’s regional endpoints (currently US-centric) are a recognized limitation.

Function Calling and Tool Use

Function calling lets the model request external data or actions. Define functions as JSON objects with name, description, and parameters (JSON Schema). The model can request multiple function calls in a single response (parallel function calling, supported in GPT-4o and later models).

The function calling flow: (1) Send messages with tools parameter containing function definitions. (2) Model responds with assistant_message and optional tool_calls. (3) Execute each tool call in your application. (4) Send tool messages with the results. (5) Model generates a final text response incorporating tool results.

Best practices: function descriptions should be detailed and specific — the model uses them to decide when to call each function. Parameter descriptions should include valid ranges and formats. Use additionalProperties: false to strictly enforce parameter schemas. Test tool calls with edge cases — what happens when a search returns no results? The model needs to handle this gracefully.

The Assistants API provides stateful agent execution with built-in tool management, persistent threads, and file retrieval. An assistant has a model, instructions (system prompt), tools (code_interpreter, file_search, and custom functions), and optional tool_resources. Threads store conversation history. Runs manage execution with automatic function calling loops, error handling, and status tracking.

Structured Outputs

Structured outputs, released in mid-2024, enforce JSON Schema compliance on model outputs. Set response_format: {"type": "json_schema", "json_schema": {"name": "...", "schema": {...}}} in the request. The model’s output is constrained at the token level to produce valid JSON matching the schema.

Implementation uses constrained decoding at inference time. Only tokens that maintain JSON validity are sampled. This eliminates parsing failures that plague applications relying on free-text responses returned as JSON. In production, structured outputs reduced parsing errors from 5-8% to near zero for companies like Stripe and Replicate.

The response_format also supports {"type": "text"} (default, any text) and {"type": "json_object"} (valid JSON, any structure — less strict than json_schema). Use json_schema for production — the strict enforcement catches more edge cases. Define the schema using Python dicts or Pydantic models and serialize to JSON.

Embeddings API

The embeddings endpoint (/v1/embeddings) generates vector representations of text. text-embedding-3-small (1536 dimensions) is the default recommendation — best quality-to-cost ratio. text-embedding-3-large (3072 dimensions) offers higher accuracy for retrieval-sensitive applications. Both support dimensionality reduction via the dimensions parameter — you can request 256-dimensional vectors from the large model, often outperforming the small model’s native dimensions.

Batch embedding requests for efficiency. Send up to 100 texts in a single API call. Batched requests are priced per token (not per request), so batching reduces overhead without affecting cost. The response includes embedding vectors ordered to match the input order.

Embeddings are used in RAG pipelines, semantic search, clustering, classification (embed + linear classifier), and recommendation systems. Cache embedding results for static content. Cosine similarity is the standard distance metric for OpenAI embeddings.

Token Management and Cost Control

Token usage is the primary cost driver. Monitor with response.usage from each API call. The tiktoken library counts tokens locally before sending requests, letting you estimate costs and manage context budgets. Each model has a specific encoding — o200k_base for GPT-4o, cl100k_base for GPT-4/GPT-3.5.

Cost optimization strategies: use GPT-4o mini for 80% of requests, GPT-4o for 20%. Cache identical requests. Implement semantic caching for similar queries. Set per-user and per-session token budgets. Use max_tokens aggressively — restrictive limits reduce cost and encourage conciseness. Monitor for anomalous usage patterns (spikes, loops) with automated alerts.

OpenAI’s dashboard provides cost tracking by API key, model, and date range. Set spending limits and notification thresholds. For high-volume applications, negotiate custom pricing — OpenAI offers volume discounts for commitments above $10,000/month.

Production Best Practices

Implement retry logic with exponential backoff for transient failures (rate limits, 5xx errors). The Python client handles 2 retries by default; increase to 4-5 for production. Use max_retries in client configuration. On 429 rate limit errors, respect the Retry-After header.

Error handling matrix: 400 errors (bad request) — fix your code. 401 (authentication) — check API key. 429 (rate limit) — back off and retry. 500 (server error) — retry with backoff. Timeout errors — retry once, then fail. Log all errors with context (model, endpoint, request ID) for debugging.

Monitor production with OpenAI’s usage dashboard plus custom instrumentation. Track: request volume, latency (p50, p95, p99), error rate, token usage per request, cost per user, and cache hit rate. Set up automated alerts for error rate > 1%, p95 latency > 5 seconds, or anomalous cost spikes.

FAQ

How do I handle rate limits? Implement per-user rate limiting in your application layer. Use exponential backoff for retries. OpenAI’s rate limits are documented per model and tier — GPT-4o generally allows 500-10,000 RPM depending on your usage tier. Request higher limits through the OpenAI dashboard. For high-volume applications, distribute requests across multiple API keys.

What is the difference between the Chat API and the Completions API? The Chat API (messages array) is the recommended interface for all current models. The legacy Completions API (single prompt string) is deprecated for new applications. The Chat API supports system prompts, multi-turn conversations, tool use, and structured outputs — all essential for production applications.

How do I get deterministic outputs? Set temperature=0 and seed=<integer> for reproducible results (OpenAI guarantees deterministic outputs for identical inputs with the same seed). Note that temperature=0 uses greedy decoding (always selects the most likely token), which is deterministic regardless of seed. Use seeds for regression testing — generate test cases and compare outputs across model versions.

Can I fine-tune GPT-4o? Yes, OpenAI offers fine-tuning for GPT-4o and GPT-4o mini. Fine-tuning is available through the API with dataset upload. Pricing is $25/M training tokens for GPT-4o mini and $75/M for GPT-4o. Fine-tuned models are charged at standard inference rates. Fine-tuning improves performance on domain-specific tasks but requires careful data preparation to avoid overfitting.

How do I manage API keys across a team? Use separate API keys for each developer, plus separate keys for staging and production. Set usage limits per key. Use organization-level keys for billing overview. Rotate keys quarterly. Never commit keys to version control — use environment variables or a secrets manager (Vault, AWS Secrets Manager, 1Password).