Data archiving

The most reliable way to keep a large object fast is to stop it from growing without bound. Data archiving is the discipline of moving cold data — records that are old, closed, or rarely accessed — out of the hot transactional object so that day-to-day queries, reports, and sharing recalculations operate on a smaller, faster dataset. An architect defines the archiving strategy up front, because retrofitting one onto a bloated object is far harder.

Start by classifying data by temperature. Hot data is current and frequently touched; warm data is occasionally referenced; cold data is retained mostly for compliance or the rare lookup. Your retention policy — driven by legal, regulatory, and business needs — decides when a record transitions from hot to cold and is eligible to move.

There are several destinations, chosen by how the archived data must be accessed. Big Objects keep massive volumes on-platform in a separate, append-optimised store queryable with async SOQL — ideal when you still need occasional programmatic access to billions of rows without impacting your standard objects. Off-platform archives (a data warehouse, cloud storage, or a partner archiving product) suit data that only needs to be produced on request. A tiered approach combines both.

Archiving decision:
  Need frequent, low-latency access?      -> keep hot (index / skinny table)
  Need occasional on-platform queries?    -> Big Object (async SOQL)
  Need rare, compliance-only retrieval?   -> off-platform (warehouse / storage)
  Then: delete migrated records from the hot object on a schedule

Implement archiving as a repeatable, scheduled process — typically a Batch Apex job or an ETL pipeline that copies eligible records to the archive, verifies the copy, then deletes them from the source. Preserve the ability to reconstruct or reference archived records (store the original Id), and communicate clearly to users where older data now lives.

As an exercise, define a retention policy for a high-volume object, pick an archive destination for its cold tier, and sketch the batch job that would move and then purge eligible records. With data architecture covered end to end, the next section turns to the second great architect domain: sharing and visibility at scale.