Solving N+1 with Bidirectional Hints

The N+1 Problem in 30 Seconds

You’ve seen this pattern:

query {
  authors {
    name
    books {
      title
    }
  }
}

The resolver fetches 10 authors. Then it fetches books for each author. That’s 1 + 10 = 11 database queries. Scale to 100 authors and you’re making 101 queries for what should be 2.

This is the N+1 problem, and it’s been plaguing GraphQL (and ORMs) since their inception.

DataLoader: The Standard Solution

Facebook’s DataLoader solves N+1 through batching. Instead of fetching books for each author individually, it collects all author IDs within a single execution tick and fetches all books in one query.

const bookLoader = new DataLoader(authorIds => {
  return db.query('SELECT * FROM books WHERE author_id IN (?)', [authorIds]);
});

// In resolver
resolve(author) {
  return bookLoader.load(author.id);
}

This transforms 101 queries into 2. Problem solved?

Not entirely.

The Limits of Batching

DataLoader batches requests within a single execution context. It doesn’t help with:

Cross-Request Patterns: If 90% of clients who fetch an author immediately fetch their books, DataLoader doesn’t prefetch books proactively. It waits for the second request.

Personalized Prefetch: Different users have different access patterns. Power users might always want reviews. Casual users might only want titles. DataLoader treats all clients identically.

Expensive Resolver Avoidance: Some resolvers are computationally expensive (LLM calls, complex aggregations). If you knew a field wouldn’t be accessed, you could skip the computation entirely.

DataLoader is reactive. It optimizes what you ask for. It doesn’t predict what you’ll ask for next.

HINT: Prediction with Feedback

The HINT protocol extends the batching paradigm with two additions:

1. Hints: Clients declare anticipated fields before query execution
2. Feedback: Clients report which predictions were actually used

This transforms caching from open-loop (guess and forget) to closed-loop (guess, measure, adapt).

HINT for GraphQL

GraphQL’s extensions field provides a natural home for HINT messages. Here’s how it works:

Client Sends Hint

The client includes anticipated fields in the request:

{
  "query": "query { author(id: 1) { name } }",
  "extensions": {
    "pce": {
      "version": "1.0",
      "request_id": "req-abc-123",
      "scope": { "tenant": "app-123" },
      "ttl": 30000,
      "anticipated_fields": [
        "author.books",
        "author.books.title",
        "author.books.publishedDate"
      ]
    }
  }
}

The client is saying: “I’m asking for the author’s name, but I anticipate needing their books next.”

Server Returns Proactive Payload

The server’s Predictive Caching Engine (PCE) receives the hint and decides whether to prefetch. If it does, it returns the proactive data tagged with a prediction identifier:

{
  "data": {
    "author": {
      "name": "Jane Doe"
    }
  },
  "extensions": {
    "pce": {
      "prediction_id": "pred-xyz-789",
      "ttl": 30000,
      "proactive": {
        "author.books": [
          { "title": "First Book", "publishedDate": "2020-01-15" },
          { "title": "Second Book", "publishedDate": "2022-06-01" }
        ]
      }
    }
  }
}

The client caches this proactive payload locally, keyed by the prediction_id.

Client Uses Cached Data

When the client subsequently queries for books:

query {
  author(id: 1) {
    books {
      title
      publishedDate
    }
  }
}

It checks its local cache first. Cache hit. No network request needed.

Client Reports Feedback

Periodically, the client batches feedback and sends it to the server:

{
  "extensions": {
    "pce": {
      "feedback": {
        "version": "1.0",
        "outcomes": [
          {
            "prediction_id": "pred-xyz-789",
            "outcome": "HIT",
            "timestamp": 1704153600,
            "details": {
              "latency_saved_ms": 45,
              "fields_used": ["author.books.title", "author.books.publishedDate"]
            }
          }
        ]
      }
    }
  }
}

Now the server knows. The prediction worked. This pattern should be reinforced.

The Feedback That Matters

The outcome taxonomy enables nuanced adaptation:

The EVICTED_UNUSED outcome is particularly valuable. It identifies pure waste—bandwidth consumed for data that was never accessed. A PCE that sees repeated EVICTED_UNUSED outcomes for a pattern should stop predicting it.

When Hints Beat DataLoader

HINT and DataLoader are complementary, not competing. DataLoader optimizes within a request. HINT optimizes across requests. Together, they’re more powerful than either alone.

HINT excels when:

1. Patterns span requests: User fetches profile, then always fetches orders. DataLoader can’t help because they’re separate requests.

2. Patterns are personalized: Power users want full data. Casual users want summaries. HINT allows per-client prediction adaptation.

3. Resolvers are expensive: If you can predict a field won’t be accessed, you can skip its resolver entirely. This is impossible with DataLoader.

4. Latency is critical: In high-latency environments (mobile, satellite), avoiding even one round-trip matters. Proactive prefetch pays for itself.

5. You want to learn: DataLoader doesn’t provide usage analytics. HINT feedback gives you rich data about access patterns.

Implementation Sketch

A minimal HINT-enabled GraphQL server might work like this:

// Middleware to extract hints
app.use('/graphql', (req, res, next) => {
  req.hint = req.body.extensions?.pce;
  next();
});

// After primary query execution
async function executeWithHints(query, hint) {
  // Execute primary query
  const result = await execute(query);

  // If hint provided, consider prefetch
  if (hint?.anticipated_fields) {
    const plan = pce.synthesizePlan(hint);

    if (plan && pce.checkBudgets(plan)) {
      const proactive = await pce.executePlan(plan);
      result.extensions = {
        pce: {
          prediction_id: plan.id,
          ttl: hint.ttl,
          proactive
        }
      };
    }
  }

  return result;
}

// Endpoint for feedback
app.post('/graphql/feedback', (req, res) => {
  const outcomes = req.body.extensions?.pce?.feedback?.outcomes;
  pce.recordFeedback(outcomes);
  res.sendStatus(204);
});

The PCE maintains a model that learns from feedback. Simple implementations might use frequency counting. Sophisticated ones might train lightweight ML models on per-tenant patterns.

Beyond N+1

While N+1 is the motivating example, HINT’s value extends further:

Schema Stitching: In federated GraphQL, hints can propagate across service boundaries, enabling coordinated prefetch.

Subscription Optimization: Hints can inform what data to include in push updates, reducing over-fetching in real-time scenarios.

Query Cost Prediction: Hints provide signal for query cost analysis before execution, enabling early rejection of expensive patterns.

Cache Warming: Hints from high-value clients can drive background cache warming, ensuring fast responses for critical paths.

Try It

HINT is an open standard. The full specification is available at /hint/.

For GraphQL specifically:

• Hints go in extensions.pce in requests
• Proactive payloads return in extensions.pce in responses
• Feedback is batched and sent to a separate endpoint or inline

Voxell products including ARC are designed around HINT for GPU-accelerated caching. But the protocol is implementation-independent—build your own PCE if you prefer.

The N+1 problem has been unsolved for too long. DataLoader was a good start. HINT closes the loop.

For the full HINT specification and transport bindings, see /hint/.