Pagination at scale — why can `skip/limit` become slow and what’s a better pattern?

Deep dive

Expanding on the short answer — what usually matters in practice:

• Context (tags): pagination, skip-limit, performance
• Data model and access patterns: dominant queries (read/write ratio, sorting, pagination).
• Indexes: when they help vs hurt (write amplification, memory).
• Consistency & transactions: what’s guaranteed and what can bite you.
• Explain the "why", not just the "what" (intuition + consequences).
• Trade-offs: what you gain/lose (time, memory, complexity, risk).
• Edge cases: empty inputs, large inputs, invalid inputs, concurrency.

Examples

Here’s an additional example (building on the short answer):

db.posts.find({ _id: { $gt: lastId } })
  .sort({ _id: 1 })
  .limit(20)

Common pitfalls

• Too generic: no concrete trade-offs or examples.
• Mixing average-case and worst-case (e.g., complexity).
• Ignoring constraints: memory, concurrency, network/disk costs.

Interview follow-ups

• When would you choose an alternative and why?
• What production issues show up and how do you diagnose them?
• How would you test edge cases?