Go JSON Encoding and Decoding Guide
JSON in Go
JSON is the universal interchange format for web services, and Go’s encoding/json package provides integrated support for marshaling (Go values to JSON) and unmarshaling (JSON to Go values). The package uses reflection at runtime to map Go struct fields to JSON properties, controlled by struct tags. While convenient, the reflection overhead makes encoding/json slower than specialized libraries like jsoniter (drop-in replacement) or ffjson (code generation). For most applications, the standard library’s balance of safety, convenience, and performance is appropriate. The package handles all standard Go types and can be extended through the json.Marshaler and json.Unmarshaler interfaces (Go Documentation, “Package encoding/json,” pkg.go.dev/encoding/json). Understanding the performance characteristics is important for high-throughput services: marshaling typically takes 200-500 ns for small structs, while unmarshaling is 2-3x slower due to allocation and type reflection. For applications handling large payloads, pay careful attention to allocation patterns and consider using stream processing.
Struct Tags for JSON Mapping
Struct tags define the bidirectional mapping between Go struct fields and JSON keys:
type ServerConfig struct {
Host string `json:"host"`
Port int `json:"port,omitempty"`
Debug bool `json:"-"`
Timeout time.Duration `json:"timeout"`
AllowedOrigins []string `json:"allowed_origins,omitempty"`
MaxConnections int `json:"max_connections,string"`
---Key tag options: json:"name" specifies the JSON field name; omitempty omits zero-value fields; string encodes the field as a JSON string (useful for JavaScript number precision); - skips the field entirely. For enums, implement json.Marshaler and json.Unmarshaler to control how enum values appear in JSON output:
type Role int
const (
RoleAdmin Role = iota
RoleEditor
RoleViewer
)
func (r Role) MarshalJSON() ([]byte, error) {
names := map[Role]string{RoleAdmin: "admin", RoleEditor: "editor", RoleViewer: "viewer"}
return json.Marshal(names[r])
---
func (r *Role) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil { return err }
values := map[string]Role{"admin": RoleAdmin, "editor": RoleEditor, "viewer": RoleViewer}
if v, ok := values[s]; ok { *r = v; return nil }
return fmt.Errorf("invalid role: %s", s)
---Custom Marshaling and Unmarshaling
Implementing json.Marshaler and json.Unmarshaler provides full control over the JSON representation of a type:
type Color struct{ R, G, B uint8 }
func (c Color) MarshalJSON() ([]byte, error) {
hex := fmt.Sprintf("#%02x%02x%02x", c.R, c.G, c.B)
return json.Marshal(hex)
---
func (c *Color) UnmarshalJSON(data []byte) error {
var hex string
if err := json.Unmarshal(data, &hex); err != nil { return err }
if len(hex) != 7 || hex[0] != '#' {
return fmt.Errorf("invalid color format: %s", hex)
}
_, err := fmt.Sscanf(hex[1:], "%02x%02x%02x", &c.R, &c.G, &c.B)
return err
---RawMessage for Dynamic Fields
json.RawMessage defers JSON decoding, allowing a struct to hold arbitrary sub-objects that can be conditionally decoded based on other fields. This pattern is essential for handling polymorphic JSON APIs where the payload structure depends on a discriminator field.
Streaming JSON with Decoder and Encoder
For large JSON payloads, streaming avoids allocating the entire payload in memory. Use json.NewDecoder for reading streams and json.NewEncoder for writing. The Decoder.Token() method provides token-level access for processing very large arrays without building the entire data structure. The Encoder.SetIndent() method controls output formatting, useful for debugging but avoided in production for bandwidth efficiency.
Time Handling
time.Time marshals to RFC 3339 format by default. For custom time formats, implement json.Marshaler and json.Unmarshaler or use a wrapper type. The time.RFC3339Nano layout is recommended for API interoperability. For legacy systems that require specific date formats, consider using *string fields as intermediaries.
Performance Benchmarking JSON Libraries
For JSON-intensive workloads, understanding performance characteristics guides library selection. encoding/json uses reflection with runtime type dispatch, typically achieving 200-500 MB/s marshaling throughput for small structs. jsoniter (json-iterator/go) provides a drop-in replacement with 2-5x faster parsing through code-generated field decoders and iterator-based streaming. ffjson generates static MarshalJSON/UnmarshalJSON methods at build time, eliminating reflection overhead entirely but requiring a code generation step in CI. simdjson/go leverages SIMD instructions for JSON parsing, achieving up to 4 GB/s on modern CPUs but with limited Go integration. For most web services, encoding/json is adequate: CPU time in JSON parsing is rarely the bottleneck compared to database queries and network I/O. Profile before optimizing: if JSON parsing exceeds 10% of CPU time, consider jsoniter as the simplest upgrade path. For API gateways and data pipelines processing gigabytes of JSON, simdjson or hand-written streaming parsers may be warranted. The go-marshal-test benchmark suite provides reproducible comparisons across libraries.
Security Considerations for JSON Parsing
JSON deserialization introduces security risks in production systems. Unbounded memory allocation from deeply nested JSON objects can be mitigated with json.Decoder and json.Decoder.DisallowUnknownFields() combined with Decoder.UseNumber(). The json.Decoder token-by-token API allows limiting nesting depth: implement a custom decoder that tracks depth and returns an error when exceeding a configurable limit. For untrusted input, set json.Decoder buffer limits and read timeouts to prevent slow-loris attacks. The json.RawMessage deferral pattern should not be used with untrusted input without size limits, as it stores raw bytes in memory without validation. Disable duplicate field detection with json.Decoder for performance but consider it for security-critical parsing. The interface{} and any types in decode targets are dangerous for untrusted input because they can deserialize to attacker-controlled types. Always decode into strongly typed structs with json tags when processing external data. The json.Valid() function provides a lightweight syntax check before full deserialization, useful for request validation middleware.
Streaming JSON for Large Payloads
Processing multi-gigabyte JSON files requires streaming to avoid exhausting memory. Go’s json.Decoder reads tokens sequentially from an io.Reader, enabling processing of arbitrary-sized arrays and objects. For a JSON array of objects, wrap the decoder in a loop calling Decode for each element: for dec.More() { var item Item; dec.Decode(&item) }. The dec.Token() method reads raw JSON tokens (Delim, String, Number, Bool, nil) for fully custom parsing without predefined types. The json.RawMessage type defers deserialization: decode a field as RawMessage and conditionally unmarshal based on its content, useful for polymorphic JSON structures.
Streaming also benefits output: json.Encoder writes directly to an io.Writer with enc.Encode(value). For arrays too large to hold in memory, iterate over data and call enc.Encode for each element, wrapping the output in [ and ] manually. The json.Encoder.SetIndent method pretty-prints output but adds significant CPU overhead for large payloads - use it only for debugging. The json.Encoder.SetEscapeHTML(false) prevents escaping <, >, and &, improving throughput when encoding non-HTML output. Benchmarking against encoding/json alternatives like json-iterator/go or segmentio/encoding/json may reveal 2-10x speedups for specific workloads. For compression, pipe encoder output through gzip.NewWriter to reduce network transfer size at modest CPU cost.
JSON Schema Validation
Validating JSON against a schema before deserialization catches structural errors early. The jsonschema crate validates JSON documents against JSON Schema draft-07 or 2020-12 specifications. Integration with Go’s encoding/json involves parsing the JSON with json.RawMessage or interface{} first, validating against the schema, then deserializing into the target type. For high-throughput APIs, schema validation at the HTTP middleware level rejects malformed payloads before they reach business logic handlers.
The kin-openapi library validates request and response bodies against OpenAPI specifications, combining schema validation with path and parameter checks. For gRPC transcode endpoints, proto schema validation via protoc-gen-validate provides similar guarantees. Performance-sensitive APIs should validate against compiled schemas: jsonschema.NewCompiler().Compile(schema) produces an optimized validator. Detailed validation errors including JSON path are available through the validator’s error interface. Schema validation combined with json.Decoder.DisallowUnknownFields() provides defense-in-depth against API misuse. For public APIs, publish the schema as a downloadable JSON Schema file and reference it in the Content-Type header via the $schema link header convention.
JSON Performance Benchmarks
Go 1.22+ includes encoding/json improvements that close the gap with third-party alternatives. Benchmarks on representative payloads should inform format and library decisions. For small messages (<1KB), the standard library’s json.Decoder performs comparably to json-iterator/go. For large arrays (>10K elements), json-iterator/go and segmentio/encoding/json show 2-5x speed improvements. The simdjson library (via CGo or pure Go bindings) provides the fastest parsing for extremely large JSON payloads, using SIMD instructions for parallel parsing up to 4GB/s throughput on modern CPUs.
Serialization benchmarks follow similar patterns: json-iterator excels at struct serialization with field caching, while segmentio/encoding/json optimizes string escaping for HTML-safe output. Memory allocation patterns differ: encoding/json allocates more per operation, while segmentio reuses buffers internally. For latency-sensitive services, pre-allocating bytes.Buffer and reusing json.Encoder instances reduces GC pressure. Always benchmark with production-representative payload sizes and structures: the optimal library depends on the specific data shape, nesting depth, and field types in your workload.
Frequently Asked Questions
What is the difference between json.Marshal and json.MarshalIndent? Marshal produces compact JSON. MarshalIndent adds indentation. Use Marshal in production.
How do I handle time.Time in Go JSON? time.Time marshals to RFC 3339 format by default. Use custom marshalers for different formats.
What is json.RawMessage used for? RawMessage stores raw JSON bytes without decoding, enabling deferred or conditional parsing.
How can I improve JSON performance? Use Decoder/Encoder for streaming, preallocate slices, and consider jsoniter for CPU-bound parsing.
How do I handle case-insensitive JSON field names? encoding/json is case-insensitive by default when unmarshaling.
Related: Go Generics Guide | Go Testing Frameworks | Go Database Guide
FAQ
Why is Go popular for backend development?
Go offers fast compilation, excellent concurrency (goroutines, channels), a comprehensive standard library, built-in testing and profiling, cross-compilation, and deployment as a single binary. These features make it ideal for microservices, CLI tools, and network services.
How does Go concurrency work?
Goroutines are lightweight threads multiplexed onto OS threads. Channels provide typed communication between goroutines. The select statement waits on multiple channel operations. Go’s concurrency model (CSP) is “do not communicate by sharing memory; share memory by communicating.”
What is the best way to structure a Go project?
Follow the standard layout: cmd/ for main packages, internal/ for private packages, pkg/ for shared libraries. Keep packages small and focused. Avoid deep nesting. Use domain-driven package naming. The standard library’s approach is a good model.
How do I handle errors in Go?
Return errors explicitly. Check errors immediately. Use sentinel errors for expected conditions, custom error types for additional context, and error wrapping (fmt.Errorf("...%w...", err)) for stack-like traces. Use errors.Is() and errors.As() for unwrapping.
Is Go good for web development?
Yes. Go’s net/http package provides a production-quality HTTP server. Frameworks like Gin, Echo, and Chi add routing and middleware. Go compiles to fast, single-binary deployments with small container images, ideal for REST APIs and microservices.