Skip to main content

Request & Response Design: Payloads, Headers, and Conventions

You've designed the URLs. You've chosen the right HTTP methods. You know exactly which status codes to return.

But none of that tells you what goes inside the request. What does the JSON body look like? How do you name fields — camelCase or snake_case? What do you wrap the response in? How does a client ask for page 3 of a task list? What does an error response look like when two fields fail validation at once?

These questions feel like details. They're not. Inconsistent payload design is one of the leading causes of client bugs — because clients have to special-case every endpoint that behaves differently from every other endpoint. A well-designed payload format is a contract: clients can write one piece of code that handles all your responses, not a different handler for each endpoint.

This article closes that gap. By the end, you'll have designed the complete wire format for TaskFlow — request bodies, response envelopes, pagination, filtering, headers, and a consistent error structure that clients can always rely on.

Quick Reference​

Naming convention: camelCase for JSON fields in request and response bodies.

Response envelope pattern:

{
  "data": { ... },
  "meta": { ... }
}

Pagination (cursor-based):

{
  "data": [...],
  "meta": {
    "cursor": "dXNlcjox",
    "has_more": true,
    "limit": 20
  }
}

Error body format:

{
  "error": {
    "code": "validation_error",
    "message": "The request contains invalid fields.",
    "details": [{ "field": "due_date", "message": "Must be a future date." }]
  }
}

Essential headers:

HeaderDirectionPurpose
Content-TypeBothFormat of the body being sent
AcceptRequestFormat the client wants back
AuthorizationRequestCredentials
ETagResponseResource version fingerprint
Cache-ControlResponseCaching instructions
X-Request-IDBothTracing across services

Gotchas:

  • ⚠️ Always return the full updated resource in PATCH/PUT responses — don't make clients re-fetch
  • ⚠️ Offset pagination breaks when rows are inserted mid-page — prefer cursors for large collections
  • ⚠️ Never include stack traces or internal error messages in responses
  • ⚠️ An empty collection is 200 OK with "data": [] — not 404

See also:

Version Information​

Relevant specifications:

Last verified: June 2025

What You Need to Know First​

Required reading (in order):

  1. REST APIs: What They Are and How They Work — foundational mental model, especially the request/response anatomy
  2. Resource Design: URLs, Nouns, and Hierarchies — TaskFlow's resource structure is the basis for all examples here
  3. HTTP Methods and Status Codes: The Full Picture — understanding which status codes to return is a prerequisite for understanding what to return alongside them

Helpful background:

  • Basic familiarity with JSON syntax — what objects, arrays, strings, and numbers look like

What We'll Cover in This Article​

By the end of this guide, you'll understand:

  • JSON naming conventions and why consistency matters
  • Response envelope patterns — when to wrap and when to send naked payloads
  • The three pagination strategies and when to use each
  • Filtering, sorting, and field selection via query parameters
  • Which HTTP headers matter for production APIs and what each one does
  • A consistent error response format that clients can always parse
  • The complete TaskFlow request and response design

What We'll Explain Along the Way​

Don't worry if you're unfamiliar with these — we'll define them as we encounter them:

  • Cursor-based pagination (what a cursor is and how it works)
  • ETags and conditional requests
  • Content negotiation (Accept and Content-Type headers)
  • Idempotency keys

JSON Naming Conventions​

The first question to settle before any other payload design: how do you name your fields?

JSON has no enforced convention. You'll find all of these in production APIs:

{ "dueDate": "2025-08-01" }      // camelCase
{ "due_date": "2025-08-01" }     // snake_case
{ "DueDate": "2025-08-01" }      // PascalCase
{ "due-date": "2025-08-01" }     // kebab-case

The "right" answer is the one your team picks and applies without exception. But the industry has settled on two dominant conventions, and the choice has real consequences:

camelCase is the JavaScript native format. When a browser or Node.js client deserializes JSON, dueDate maps directly to a JavaScript variable with no transformation. It's the convention used by GitHub's API, Stripe, Twilio, and most modern consumer-facing APIs.

snake_case is the Python and Ruby native format. It's also easier to read for multi-word field names at a glance. It's used by Slack's API, Twitter's API (now X), and many older APIs.

TaskFlow uses camelCase — our primary clients are web and mobile apps built in JavaScript and Swift/Kotlin, both of which expect camelCase. The convention applies to every field name in every request and response body.

// ✅ TaskFlow convention — camelCase throughout
{
  "id": "task_99",
  "title": "Review pull request",
  "assigneeId": "user_42",
  "projectId": "proj_5",
  "dueDate": "2025-08-01",
  "createdAt": "2025-06-15T10:30:00Z",
  "updatedAt": "2025-06-15T10:30:00Z"
}

// ❌ Inconsistent — mixing conventions breaks client code generators
{
  "id": "task_99",
  "title": "Review pull request",
  "assignee_id": "user_42",   // snake_case
  "ProjectId": "proj_5",      // PascalCase
  "due-date": "2025-08-01"    // kebab-case
}

Date and Time Format​

Always use ISO 8601 for dates and timestamps. Always include timezone information for timestamps.

{
  "dueDate": "2025-08-01", // Date only — ISO 8601 date
  "createdAt": "2025-06-15T10:30:00Z", // Timestamp — ISO 8601 with UTC
  "updatedAt": "2025-06-15T14:22:17Z"
}

Never use Unix timestamps in your primary API (though you may include them as supplementary fields). Never use locale-formatted dates ("August 1, 2025" or "01/08/2025") — they're ambiguous and require client-side parsing logic that varies by region.

Null vs Absent Fields​

Be explicit about whether you omit null fields or include them:

// Option A: Include null fields explicitly
{
  "title": "Review PR",
  "dueDate": null,      // Field exists, has no value
  "assigneeId": null
}

// Option B: Omit null fields entirely
{
  "title": "Review PR"
  // dueDate and assigneeId don't appear
}

TaskFlow uses Option A — all fields are always present in responses, with null for unset optional values. This means clients never need to check "does this field exist?" — they only check "is this field null?"

Pick one and apply it consistently. Mixing both in the same API breaks client deserialization code.

Response Envelope Patterns​

When your server returns a task, does it send the task object directly — or wrap it in a container?

// Naked payload — no envelope
{
  "id": "task_99",
  "title": "Review pull request",
  "status": "open"
}

// Enveloped payload
{
  "data": {
    "id": "task_99",
    "title": "Review pull request",
    "status": "open"
  }
}

Both approaches are valid REST. The tradeoffs are real.

Naked payloads are simpler and more direct. The client gets exactly the resource, nothing extra. This works well for simple APIs and is the default style for many well-regarded APIs (including parts of GitHub's API).

Enveloped payloads add a consistent wrapper around every response. The envelope provides a predictable home for metadata — pagination cursors, request IDs, warnings, and links — without polluting the resource object itself. This is the approach taken by APIs like Stripe, which wraps every response in { "object": "...", "data": ... }.

The core problem with naked payloads emerges when you need to add metadata to a response that's currently returning a plain object. If you're returning { "id": "task_99", ... } directly, and you later need to add a cursor field for pagination, you have two bad options: add cursor as a field on the task (wrong — it's not part of the task) or break the response shape (not backward compatible).

An envelope solves this from the start. Metadata always lives in meta. Resources always live in data. Adding new metadata never disturbs the data shape.

TaskFlow uses enveloped responses for all endpoints:

// Single resource
{
  "data": {
    "id": "task_99",
    "title": "Review pull request",
    "assigneeId": "user_42",
    "status": "open",
    "dueDate": "2025-08-01",
    "createdAt": "2025-06-15T10:30:00Z",
    "updatedAt": "2025-06-15T10:30:00Z"
  }
}

// Collection
{
  "data": [
    { "id": "task_99", "title": "Review pull request", ... },
    { "id": "task_100", "title": "Fix login bug", ... }
  ],
  "meta": {
    "cursor": "dXNlcjox",
    "hasMore": true,
    "limit": 20
  }
}

The data field always holds the resource or array of resources. The meta field holds pagination state and any other response-level metadata. Error responses use a different top-level key (error) — covered in the error format section.

Pagination​

Most collection endpoints can't return everything at once. A TaskFlow project might contain 10,000 tasks. Returning all of them in one response would be slow, memory-intensive, and useless for most clients. Pagination lets clients request data in manageable chunks.

There are three main pagination strategies. Each has a distinct use case.

Offset Pagination​

The simplest approach: skip the first N items, return the next M.

GET /tasks?offset=40&limit=20

"Skip 40, give me the next 20."

{
  "data": [...],
  "meta": {
    "total": 312,
    "offset": 40,
    "limit": 20
  }
}

Pros: Simple to implement. Clients can jump to any page directly ("go to page 5"). The total count is easy to return.

Cons: It breaks when data changes between pages. If someone inserts a task between page 1 and page 2 being fetched, every item shifts by one — the client sees a duplicate (an item appears on both page 1 and page 2) or a gap (an item is skipped entirely). On large datasets, OFFSET 50000 forces the database to scan and discard 50,000 rows before returning anything.

Use when: The dataset is small, static, or where jumping to arbitrary pages matters (admin dashboards, reporting interfaces).

Cursor-Based Pagination​

Instead of skipping N rows, the server returns an opaque cursor — a pointer to where the next page begins. The client sends the cursor back to get the next page.

GET /tasks?limit=20

Response:

{
  "data": [...20 tasks...],
  "meta": {
    "cursor": "dXNlcjox",
    "hasMore": true,
    "limit": 20
  }
}

Next page:

GET /tasks?limit=20&cursor=dXNlcjox

The cursor is opaque to the client — it's a base64-encoded pointer (often the ID or timestamp of the last item returned) that the server decodes internally. Clients must treat it as a black box and not attempt to construct or decode it.

Pros: Stable even as data changes — no duplicates or gaps. Efficient for large datasets (the database uses an index seek, not a scan). Works well for infinite scroll UIs.

Cons: Clients can't jump to arbitrary pages. No easy "total count" (you'd need a separate query). Harder to implement than offset.

Use when: The dataset is large or frequently updated. Mobile apps with infinite scroll. Any feed-style interface.

Page-Based Pagination​

A variation of offset pagination that expresses position as a page number rather than an item count.

GET /tasks?page=3&perPage=20
{
  "data": [...],
  "meta": {
    "page": 3,
    "perPage": 20,
    "totalPages": 16,
    "total": 312
  }
}

Pros: Familiar for users ("Page 3 of 16"). Easy to render page controls in a UI.

Cons: The same underlying problems as offset pagination — data shifts between fetches.

Use when: The UI explicitly shows page numbers and a total page count. Admin interfaces, search results.

TaskFlow's Pagination Decision​

TaskFlow uses cursor-based pagination for all collection endpoints. The primary clients are mobile apps with infinite scroll and web apps where "load more" is the dominant pattern. Stability matters more than random access.

// GET /tasks?limit=20&cursor=dXNlcjox
// TypeScript response type

interface PaginatedResponse<T> {
  data: T[];
  meta: {
    cursor: string | null; // null when there are no more pages
    hasMore: boolean;
    limit: number;
  };
}

// Example response
const response: PaginatedResponse<Task> = {
  data: [
    {
      id: "task_99",
      title: "Review pull request",
      assigneeId: "user_42",
      status: "open",
      dueDate: "2025-08-01",
      createdAt: "2025-06-15T10:30:00Z",
      updatedAt: "2025-06-15T10:30:00Z",
    },
    // ...19 more tasks
  ],
  meta: {
    cursor: "dXNlcjox", // opaque — client passes this back as-is
    hasMore: true,
    limit: 20,
  },
};

Default limit is 20. Maximum limit is 100. Requests above 100 return a 400 with a clear error message.

Filtering, Sorting, and Field Selection​

Query parameters handle everything that shapes a collection response without changing the URL identity of the resource.

Filtering​

Filters narrow the collection to items matching specified criteria. Use simple field=value query parameters for basic filtering:

GET /tasks?status=open
GET /tasks?assigneeId=user_42
GET /tasks?status=open&assigneeId=user_42   — AND logic
GET /tasks?projectId=proj_5&status=open

For range filters, use suffixed parameter names:

GET /tasks?dueDateFrom=2025-07-01&dueDateTo=2025-08-01
GET /tasks?createdAfter=2025-06-01

Avoid inventing a query mini-language (like /tasks?filter=status:open,due_date>2025-07-01) unless you have a compelling reason — it adds parsing complexity on both server and client without meaningful benefit for most APIs.

Sorting​

Use a sort parameter with a field name. Prefix with - to indicate descending order:

GET /tasks?sort=dueDate          — Ascending by due date
GET /tasks?sort=-dueDate         — Descending by due date
GET /tasks?sort=-createdAt       — Newest first (most common default)
GET /tasks?sort=status,-dueDate  — By status ascending, then due date descending

The - prefix convention is widely used and immediately readable. Always document which fields are sortable — sorting by non-indexed fields can cause serious performance problems on large tables.

Always define a default sort order. TaskFlow defaults to sort=-createdAt — newest tasks first.

Field Selection (Sparse Fieldsets)​

Sometimes clients only need a subset of a resource's fields. A task list in the mobile app might only need id, title, and status — not the full task object with all its fields.

Field selection lets clients specify exactly what they want:

GET /tasks?fields=id,title,status

Response:

{
  "data": [
    { "id": "task_99", "title": "Review pull request", "status": "open" },
    { "id": "task_100", "title": "Fix login bug", "status": "open" }
  ],
  "meta": { ... }
}

Field selection is a performance optimization — smaller payloads, faster responses, less bandwidth. It's most valuable for mobile clients on slow connections.

TaskFlow implements field selection for all collection endpoints. The fields parameter accepts a comma-separated list of field names. Requesting a non-existent field returns a 400.

HTTP Headers That Matter​

Headers are the metadata layer of HTTP — they carry information about the request and response that doesn't belong in the URL or the body. Here are the ones every production API should handle correctly.

Content-Type​

Tells the receiver what format the body is in. Always set this on requests with a body, and always set it on responses with a body.

Content-Type: application/json; charset=utf-8

If a client sends a request without Content-Type: application/json, your server should return 415 Unsupported Media Type rather than trying to guess the format.

Accept​

The client's way of saying what format it wants back. For a JSON API, you'll typically only support application/json, but handle the header correctly:

Accept: application/json

If a client requests a format you don't support (e.g., Accept: application/xml), return 406 Not Acceptable.

Authorization​

Carries the client's credentials with every request (statelessness in action). The most common format for modern REST APIs is Bearer token:

Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

We'll design TaskFlow's full authentication scheme in Article 7. For now: the Authorization header is how identity travels with every request.

ETag​

An ETag (Entity Tag) is a fingerprint of a resource's current state — typically a hash of the resource content or a version number. The server includes it in responses:

GET /tasks/task_99
Response:
  ETag: "a3f5b2c8"
  Body: { "id": "task_99", ... }

On subsequent requests, the client sends the ETag back in an If-None-Match header:

GET /tasks/task_99
If-None-Match: "a3f5b2c8"

If the resource hasn't changed, the server returns 304 Not Modified with no body — saving the bandwidth of re-transmitting the full resource. If it has changed, the server returns 200 OK with the new resource and a new ETag.

ETags also enable optimistic concurrency — preventing lost updates when two clients edit the same resource simultaneously:

PATCH /tasks/task_99
If-Match: "a3f5b2c8"
Body: { "title": "Updated title" }

If the ETag doesn't match (someone else modified the task since you last fetched it), the server returns 412 Precondition Failed instead of applying your update on top of stale data.

Cache-Control​

Tells clients and intermediaries how long a response can be cached and under what conditions:

Cache-Control: max-age=60           — Cache for 60 seconds
Cache-Control: no-store             — Never cache (auth responses, sensitive data)
Cache-Control: no-cache             — Cache but always revalidate before using
Cache-Control: private, max-age=60  — Cache only in the browser, not CDNs

For TaskFlow:

GET /tasks          → Cache-Control: private, max-age=30
GET /tasks/task_99  → Cache-Control: private, max-age=60, must-revalidate
POST /tasks         → Cache-Control: no-store
DELETE /tasks/99    → Cache-Control: no-store

Task lists are user-specific (private) and change frequently (short max-age). Individual tasks change less often (longer max-age). Write operations should never be cached.

X-Request-ID​

A tracing header that assigns a unique ID to each request, making it possible to trace a single request across multiple services and log entries:

Request:
  X-Request-ID: req_8kJd92nAm4p

Response:
  X-Request-ID: req_8kJd92nAm4p   ← Echo it back in the response

If the client sends an X-Request-ID, echo it in the response. If they don't, generate one server-side. Log it with every log line related to that request. When a client reports a bug, they can give you a request ID and you can find the exact log trail.

The X- prefix indicates a custom, non-standard header. While RFC 6648 deprecated the convention of using X- for custom headers in 2012, X-Request-ID is so widely established in practice that it's effectively a standard.

Idempotency-Key​

For POST operations where duplicate execution would cause harm (payments, order creation, email sending), clients can send an idempotency key — a client-generated unique ID the server uses to deduplicate:

POST /tasks
Idempotency-Key: idem_9fA3kLm2Xp
Body: { "title": "Review pull request" }

If the server has already processed a request with this key, it returns the original response without creating a duplicate. If the client never got the response (network timeout), it can safely retry with the same key.

TaskFlow implements idempotency keys for task creation. The key is stored server-side for 24 hours. This is the pattern made famous by Stripe's idempotent requests design.

Error Response Format​

Inconsistent error responses are one of the most common API pain points. If every endpoint returns errors in a different format, clients have to write special-case parsing for each one.

The goal is a single, predictable error format that every client can handle with one piece of code.

TaskFlow's Error Format​

Every error response uses the same top-level structure:

{
  "error": {
    "code": "validation_error",
    "message": "The request contains invalid fields.",
    "details": [
      {
        "field": "dueDate",
        "message": "Must be a future date.",
        "value": "2020-01-01"
      },
      {
        "field": "title",
        "message": "Cannot be empty."
      }
    ]
  }
}

Each field has a specific purpose:

  • code — A machine-readable string identifying the error type. Clients switch on this. Values are stable across API versions.
  • message — A human-readable description. Suitable for logging. Not guaranteed to be stable (wording may change). Do not display directly to end users.
  • details — An array of field-level errors (for validation failures). Always an array, even for single-field errors — this keeps the format consistent and avoids clients having to handle both object and array shapes.

Error Codes for TaskFlow​

Defining stable error codes upfront prevents the chaos of every developer inventing their own error strings:

CodeStatusMeaning
validation_error400/422One or more fields failed validation
invalid_json400Request body is not valid JSON
authentication_required401No credentials provided
invalid_credentials401Credentials provided but invalid
insufficient_permissions403Authenticated but not authorized
not_found404Resource does not exist
method_not_allowed405HTTP method not supported for this endpoint
conflict409State conflict (e.g., duplicate email)
rate_limited429Too many requests
internal_error500Unexpected server error
service_unavailable503Server temporarily unavailable

Real Error Responses​

Let's see the format applied to real TaskFlow scenarios:

// 400 — Missing required field
{
  "error": {
    "code": "validation_error",
    "message": "The request contains invalid fields.",
    "details": [
      { "field": "title", "message": "title is required and cannot be empty." }
    ]
  }
}

// 401 — No auth token provided
{
  "error": {
    "code": "authentication_required",
    "message": "This endpoint requires authentication. Provide a valid Bearer token.",
    "details": []
  }
}

// 403 — Trying to edit someone else's comment
{
  "error": {
    "code": "insufficient_permissions",
    "message": "You do not have permission to edit this comment.",
    "details": []
  }
}

// 409 — Email already registered
{
  "error": {
    "code": "conflict",
    "message": "An account with this email address already exists.",
    "details": [
      { "field": "email", "message": "Email is already in use." }
    ]
  }
}

// 500 — Unexpected server error
{
  "error": {
    "code": "internal_error",
    "message": "An unexpected error occurred. Please try again or contact support.",
    "details": []
  }
}

Notice that the 500 response contains no internal details — no stack trace, no database error message, no file paths. All of that is logged server-side. Sending internal details to clients is a security risk and provides no value to the client, who can't act on them.

Validation Error Implementation​

Here's what producing a validation error looks like in practice on a TypeScript/Express server:

import { Request, Response } from "express";

interface ErrorDetail {
  field?: string;
  message: string;
  value?: unknown;
}

interface ApiError {
  code: string;
  message: string;
  details: ErrorDetail[];
}

// Helper to build a consistent error response
function createErrorResponse(error: ApiError) {
  return { error };
}

// POST /tasks handler — validation example
async function createTask(req: Request, res: Response) {
  const errors: ErrorDetail[] = [];

  // Validate required fields
  if (!req.body.title || req.body.title.trim() === "") {
    errors.push({
      field: "title",
      message: "title is required and cannot be empty.",
    });
  }

  // Validate date format and future constraint
  if (req.body.dueDate) {
    const due = new Date(req.body.dueDate);
    if (isNaN(due.getTime())) {
      errors.push({
        field: "dueDate",
        message: "dueDate must be a valid ISO 8601 date.",
        value: req.body.dueDate,
      });
    } else if (due < new Date()) {
      errors.push({
        field: "dueDate",
        message: "dueDate must be in the future.",
        value: req.body.dueDate,
      });
    }
  }

  // Return all validation errors at once — not one at a time
  if (errors.length > 0) {
    return res.status(422).json(
      createErrorResponse({
        code: "validation_error",
        message: "The request contains invalid fields.",
        details: errors,
      }),
    );
  }

  // Proceed with task creation...
}

The key design decision: collect all validation errors before responding. Returning one error at a time forces clients into a frustrating loop — fix one field, submit, get another error, fix that field, submit again. Return everything at once.

Putting It All Together: TaskFlow's Complete Wire Format​

Let's trace through three complete interactions with all conventions applied.

Creating a Task​

POST /tasks
Content-Type: application/json
Authorization: Bearer eyJhbGci...
Idempotency-Key: idem_9fA3kLm2Xp

{
  "title": "Review pull request #204",
  "assigneeId": "user_42",
  "projectId": "proj_5",
  "dueDate": "2025-08-01"
}

Success response:

HTTP/1.1 201 Created
Content-Type: application/json
Location: /tasks/task_99
ETag: "a3f5b2c8"
Cache-Control: no-store
X-Request-ID: req_8kJd92nAm4p

{
  "data": {
    "id": "task_99",
    "title": "Review pull request #204",
    "assigneeId": "user_42",
    "projectId": "proj_5",
    "status": "open",
    "dueDate": "2025-08-01",
    "createdAt": "2025-06-15T10:30:00Z",
    "updatedAt": "2025-06-15T10:30:00Z"
  }
}

Listing Tasks with Filters and Pagination​

GET /tasks?status=open&assigneeId=user_42&sort=-dueDate&limit=20
Authorization: Bearer eyJhbGci...

Response:

HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: private, max-age=30
X-Request-ID: req_2mPq4rNt7s

{
  "data": [
    {
      "id": "task_99",
      "title": "Review pull request #204",
      "assigneeId": "user_42",
      "projectId": "proj_5",
      "status": "open",
      "dueDate": "2025-08-01",
      "createdAt": "2025-06-15T10:30:00Z",
      "updatedAt": "2025-06-15T10:30:00Z"
    }
    // ...19 more tasks
  ],
  "meta": {
    "cursor": "dXNlcjox",
    "hasMore": true,
    "limit": 20
  }
}

Updating a Task with Optimistic Concurrency​

PATCH /tasks/task_99
Content-Type: application/json
Authorization: Bearer eyJhbGci...
If-Match: "a3f5b2c8"

{
  "status": "complete"
}

Success (ETag matched — no concurrent modification):

HTTP/1.1 200 OK
Content-Type: application/json
ETag: "b7d9e1f4"
Cache-Control: private, max-age=60, must-revalidate
X-Request-ID: req_5nRv8wXy1k

{
  "data": {
    "id": "task_99",
    "title": "Review pull request #204",
    "assigneeId": "user_42",
    "projectId": "proj_5",
    "status": "complete",
    "dueDate": "2025-08-01",
    "createdAt": "2025-06-15T10:30:00Z",
    "updatedAt": "2025-06-15T14:22:17Z"
  }
}

Failure (ETag didn't match — someone else modified the task first):

HTTP/1.1 412 Precondition Failed
Content-Type: application/json

{
  "error": {
    "code": "conflict",
    "message": "The resource has been modified since you last fetched it. Fetch the latest version and retry.",
    "details": []
  }
}

Common Misconceptions​

❌ Misconception: An empty list should return 404​

Reality: 404 Not Found means a resource doesn't exist at that URL. The /tasks collection always exists — it may simply have zero items. An empty collection is a 200 OK with "data": [].

// ✅ Correct — empty collection
HTTP/1.1 200 OK

{
  "data": [],
  "meta": {
    "cursor": null,
    "hasMore": false,
    "limit": 20
  }
}

❌ Misconception: You should return partial success for batch operations​

Reality: Partial success (some items succeeded, some failed in a batch request) is genuinely tricky. Avoid it unless you have a strong reason — it forces clients to parse a hybrid success/failure response and implement split recovery logic. Design batch operations to either fully succeed or fully fail. If partial success is unavoidable, use 207 Multi-Status and return per-item results.

❌ Misconception: More specific error messages are always better​

Reality: Error messages should give the client enough information to fix the request — nothing more. Internal details (database error messages, stack traces, table names, file paths) are security risks and irrelevant to the client. The message field is for operators reading logs, not for displaying to end users verbatim.

❌ Misconception: Offset pagination is fine for everything​

Reality: Offset pagination works until your data changes or your dataset grows large. At 100,000 rows, OFFSET 50000 scans half the table. At high write rates, items shift between pages and clients see duplicates or gaps. Start with cursors if your data will grow or change — retrofitting cursor pagination later requires a breaking API change.

Troubleshooting Common Issues​

Problem: Clients are seeing duplicate items when paginating​

Symptoms: The same task appears on both page 1 and page 2 of a paginated list.

Cause: Offset-based pagination combined with inserts between page fetches. New rows shift all subsequent rows down, causing overlap.

Fix: Switch to cursor-based pagination. The cursor is anchored to a specific row position (usually the last ID or timestamp seen), so inserts don't affect it.

Problem: Validation errors are returning one field at a time​

Symptoms: Client submits a form, fixes one error, submits again, gets a different error. Multiple round trips for a single form.

Cause: The server validates fields sequentially and returns on the first failure.

Fix: Collect all validation errors before returning. Use a details array in the error response. Return all failures in a single 422 response.

// ❌ Stop-on-first-error — multiple round trips
if (!req.body.title) {
  return res.status(422).json({ error: "title is required" });
}
if (!req.body.projectId) {
  return res.status(422).json({ error: "projectId is required" });
}

// ✅ Collect all errors first
const errors: ErrorDetail[] = [];
if (!req.body.title)
  errors.push({ field: "title", message: "title is required" });
if (!req.body.projectId)
  errors.push({ field: "projectId", message: "projectId is required" });
if (errors.length > 0) {
  return res.status(422).json(
    createErrorResponse({
      code: "validation_error",
      message: "The request contains invalid fields.",
      details: errors,
    }),
  );
}

Problem: The response format is different across endpoints​

Symptoms: Some endpoints return naked objects, others return enveloped responses. Some return arrays directly, others wrap them. Clients need special-case code for each endpoint.

Fix: Enforce the envelope pattern at the framework level — a response serializer that always wraps data in { data: ... } — rather than relying on each handler to remember. Add an integration test suite that validates the response shape of every endpoint.

Check Your Understanding​

Quick Quiz​

  1. A client sends GET /tasks?cursor=dXNlcjox. The cursor points to the last task in the collection — there are no more pages. What should the response look like?

    Show Answer
    {
      "data": [],
      "meta": {
        "cursor": null,
        "hasMore": false,
        "limit": 20
      }
    }

    cursor is null (no next page to point to), hasMore is false. Status code is 200 OK — an empty final page is not an error.

  2. What's wrong with this error response?

    HTTP/1.1 422 Unprocessable Entity

    {
      "error": "dueDate: invalid date format at line 1, column 42 (PostgreSQL error: invalid input syntax for type date)"
    }
    Show Answer

    Three problems:

    • The error leaks internal implementation details (PostgreSQL error message, line/column references to internal parsing)
    • The format is a plain string, not a structured object with code, message, and details
    • Clients can't reliably parse or switch on a free-form string

    Correct response:

    {
      "error": {
        "code": "validation_error",
        "message": "The request contains invalid fields.",
        "details": [
          {
            "field": "dueDate",
            "message": "dueDate must be a valid ISO 8601 date."
          }
        ]
      }
    }

  3. A client wants to list tasks sorted by due date (ascending), filtered to open tasks only, with 10 results per page. Write the request.

    Show Answer
    GET /tasks?status=open&sort=dueDate&limit=10
    Authorization: Bearer <token>
    Accept: application/json

  4. When would you use ETag + If-Match on a PATCH request? What problem does it solve?

    Show Answer

    Use If-Match when you want to prevent lost updates — the "last write wins" problem that occurs when two clients edit the same resource concurrently.

    Scenario: User A and User B both fetch task_99 (ETag: "a3f5b2c8"). User A updates the title. The server saves the change and issues a new ETag ("b7d9e1f4"). User B now tries to update the assignee — but their PATCH carries If-Match: "a3f5b2c8". The server detects the mismatch and returns 412 Precondition Failed instead of silently overwriting User A's title change.

    Without If-Match, User B's update would succeed but lose User A's title change.

Hands-On Exercise​

Challenge: Design the complete request and response for the following TaskFlow operation:

A mobile client wants to load the first page of open tasks assigned to the current user, showing only the id, title, dueDate, and status fields. The client last fetched this list 30 seconds ago and has the ETag from that response.

Write the full HTTP request (including all relevant headers) and both possible responses: one where the data has changed, one where it hasn't.

Show Answer

Request:

GET /tasks?status=open&assigneeId=user_42&fields=id,title,dueDate,status&limit=20
Authorization: Bearer eyJhbGci...
Accept: application/json
If-None-Match: "c4e8a1f9"

Response A — data hasn't changed:

HTTP/1.1 304 Not Modified
ETag: "c4e8a1f9"
Cache-Control: private, max-age=30
X-Request-ID: req_3pQr6sUv9w

(No body — client uses its cached copy)

Response B — data has changed:

HTTP/1.1 200 OK
Content-Type: application/json
ETag: "d5f9b2g0"
Cache-Control: private, max-age=30
X-Request-ID: req_3pQr6sUv9w

{
  "data": [
    {
      "id": "task_99",
      "title": "Review pull request #204",
      "dueDate": "2025-08-01",
      "status": "open"
    }
    // ...more tasks
  ],
  "meta": {
    "cursor": "dXNlcjox",
    "hasMore": true,
    "limit": 20
  }
}

Only the four requested fields appear in each task object — field selection is applied. The response includes a new ETag that the client stores for the next conditional request.

Summary: Key Takeaways​

  • Pick a JSON naming convention and apply it universally. TaskFlow uses camelCase throughout. Dates use ISO 8601. Null fields are included explicitly — never omitted.

  • Enveloped responses pay for themselves. Wrapping data in { "data": ..., "meta": ... } gives you a stable home for metadata without ever polluting the resource shape. Build it in from day one.

  • Cursor pagination is the right default for growing datasets. Offset pagination is simple but breaks under inserts and doesn't scale. Use cursors for any collection that will grow or change frequently.

  • Query parameters shape collections without changing resource identity. Filtering (?status=open), sorting (?sort=-dueDate), and field selection (?fields=id,title) are all query parameter conventions. Define defaults for everything — especially sort order.

  • Headers are the metadata layer — use them. Content-Type, Authorization, ETag, Cache-Control, and X-Request-ID each carry information that belongs in headers, not in the body.

  • One consistent error format handles everything. The code / message / details structure is machine-readable, human-readable, and extensible. Clients write one error handler and it works everywhere.

  • Return all validation errors at once. Never make clients fix one field, resubmit, and discover the next error. Collect everything and return it in a single response.

What's Next?​

You now have the complete design for what travels over the wire in TaskFlow — bodies, headers, pagination, and errors.

The natural next step is API Versioning: Strategies and Tradeoffs (coming soon) — where we confront an uncomfortable truth: the API you design today will need to change. The versioning strategy you choose determines whether those changes can be made without breaking your existing clients. It's a decision that's nearly impossible to retrofit once clients are in production.

References​