Google Cloud has recently unveiled a preview of Bigtable Tiered Storage, a new capability that enables developers to manage both hot and cold data within a single Bigtable instance—optimizing costs while maintaining seamless data access.
With this enhancement, developers can define age-based tiering policies directly on tables, with a minimum age threshold of 30 days. The service automatically migrates data between SSD and infrequent-access storage tiers without requiring manual export of rarely accessed records.
Anton Gething, Senior Product Manager at Google, and Derek Lee, Software Engineer at Google, detailed how tiered storage reduces operational overhead and eliminates the need for manual data migration. They noted:
This feature works in tandem with Bigtable’s autoscaling capabilities to optimize resource utilization. Moreover, data stored in the infrequent-access tier remains fully accessible through the same Bigtable API used for SSD-hosted data.
Data is shifted to the infrequent-access tier based solely on cell timestamps relative to the configured age threshold—regardless of how often the data is read. Once a cell’s timestamp exceeds the defined age, it is automatically moved from the SSD layer to the infrequent-access layer.
Bigtable is a managed, low-latency NoSQL database on Google Cloud, supporting key-value and wide-column data models with compatibility for Cassandra and HBase. It is engineered for rapid access to structured, semi-structured, or unstructured data. Common use cases highlighted by Google include time-series data from sensors, devices, and operational systems in manufacturing and automotive industries.
The documentation emphasizes that to maximize SSD performance and fully leverage tiered storage, developers should apply timestamp range filters when querying data exclusively stored on SSDs.
Given Bigtable’s design to support both operational and analytical workloads within a single database, Gething and Lee explained how tiered storage simplifies analytics and reporting access:
You can query infrequently accessed data using Bigtable SQL and create logical views to present this data in a query-friendly format when needed. This is particularly valuable for granting specific users historical data access for reporting—without exposing full table permissions.
Additionally, the new tiered storage option significantly boosts per-node capacity: nodes with tiered storage offer up to 540% more storage than standard SSD-only nodes. Florin Lungu, Chief DevOps Engineer and Vice President at Deutsche Bank, commented:
Bigtable Tiered Storage delivers a solution for managing data costs without compromising data availability. (...) This could substantially influence how organizations refine their data storage strategies.
To move data back to SSD storage, developers must either raise the tiering policy’s age threshold to reclassify older data as “hot,” disable tiered storage entirely, or overwrite the data with a newer timestamp and delete the old version.
Pricing for Bigtable is based on compute capacity, database storage, backup storage, and network usage—with cold storage costing up to 85% less than SSD storage. It’s important to note that tiered storage is not available for Bigtable HDD instances and does not support Bigtable Data Boost or hot backups.
Earlier this year, Google introduced tiered storage for Spanner, its managed distributed SQL database, as previously reported by InfoQ.
