concept · concepts/Data Gravity.md
Data Gravity
Large data is expensive to move, so it exerts a "gravity" that should pull compute toward it: move the small thing (the query, the spec) to the data — not the data to the app.
Grounded 2026-06-26 by McCrory — Data Gravity in the Clouds (2010), the post that coined the term — fetched under the auto-ground convention. The source confirms the heuristic and adds the originating analogy (data as a mass) and the mechanism (latency + throughput).
Explanation
The source (Working Heuristics — Data, Matching, Product, Org, Tooling) frames it as a placement rule: put compute where the data has gravity. Between the two things you could move — the data or the logic that operates on it — move whichever is smaller. The data is almost always the heavy object: pulling a large dataset into an application to process it there incurs transfer cost, staleness, duplication, and a second place for it to drift. Pushing the query or the transformation spec down to where the data already lives avoids all of that.
Dave McCrory's coinage (McCrory — Data Gravity in the Clouds, 2010) supplies the underlying physics: treat data as a mass that, as it accumulates, attracts applications and services. The "accelerators" of that pull are latency and throughput — the closer compute sits to the data, the faster it runs and the more reliant it becomes. And the decisive asymmetry: "data if large enough can be virtually impossible to move," so the compute gravitates to the data, not the reverse.
Why it matters / where it shows up
- The cost asymmetry is large and one-directional. A query is bytes; a dataset is gigabytes-to-petabytes. Moving the small thing is cheaper by orders of magnitude and stays correct (no copy to keep in sync).
- It's the placement sibling of Data Contract. Both say do the work at the right place: the contract derives once at the boundary; data gravity computes where the data lives. Re-locating or re-deriving is the shared waste.
- It generalizes to platform choice. Where the data has gravity often is the hardest, most-recurring part of the workflow — which is why it connects to Choose the Platform by the Hardest Recurring Part.
Open questions
- When the "small thing" (a model, a heavy UDF) stops being small, where does the gravity argument flip — and is moving a data subset out ever the right call?
Connections
- Do It Once, at the Right Place — data gravity is the compute-placement face of that meta-principle (compute where the data lives, don't copy it out and back).
- Choose the Platform by the Hardest Recurring Part — data gravity is often what makes a given part of the workflow the hardest/most-recurring, and thus decisive.
- Data Contract — sibling "do it once, at the right place" heuristic.
- Build It as an Extension — both push capability toward where things already are (the data; the user's mental model) rather than standing up something separate.
Sources
- Working Heuristics — Data, Matching, Product, Org, Tooling — "put compute where the data has gravity; move the small thing to the data, not the data to the app."
- McCrory — Data Gravity in the Clouds — the 2010 post that coined "data gravity"; the mass analogy and the latency/throughput mechanism.