AWS leverages a segment that was not exactly born overnight, but until recently, it was mostly populated by niche open source platforms or relational databases with SQL extensions for functions such as time-period definitions, temporal primary keys, and syntax for time-cut tables. The explosion in data volumes has driven the emergence of databases specifically designed for the purpose.
It’s the utility cases, stupid, not to mention the preponderance of utility cases involving data that lives in the cloud and often comes in unpredictable torrents that have aroused interest in custom-built time series databases. Consumer issues such as measuring real-time product demand, analyzing clickstream data, managing intelligent supply networks, monitoring IT infrastructure, tracking commodity prices and capital markets, and real-time supply chain optimization are among those that have created demand for fit-to-purpose tag serial platforms. designed for the cloud. It is spurred open source and quasi open source platforms like InfluxDB Cloud and Time scale Cloud, and in the mix, Amazon Timestream is now jumping.
Time series data emphasizes the design parameters of most SQL and NoSQL platforms. Among the key points are how to handle and partition sliding time windows, handling both numeric (eg Meter reading) and alphanumeric text (eg “STATUS: OK”) as first-class devices and then automation of data lifecycle management so as not to clog high-performance levels designed for landing real-time data feeds.
As mentioned, Timestream is a database platform that AWS designed from the ground up. The SQL interface and multi-AZ auto-replication can evoke similarities Amazon Aurora, while the serverless architecture may make it look like a clone of DynamoDB. But Timestream is its own being. It is server-free, with the ability to automatically scale for trillions of events. It can automatically sort data from a stored memory in the memory for magnetic storage. Unlike DynamoDB, Timestream is not only an operational database, but instead also designed to handle complex analytical queries that, with SQL support, can include complex table or time division partition merges. Timestream also has SQL support for time series functions for approximation and interpolation.
Not surprisingly, the first plugs that come to Timestream at launch are focused on consuming streaming and IoT data. It includes another plug Amazon Kinesis Data Analytics (KDA) to Apache Flink. That KDA Flink adapter can be used with Amazon Kinesis, Amazon MSK and Apache Kafka. In addition, Timestream has connections to data coming from IT infrastructure and similar sources for DevOps monitoring Telegraph open source agent, and soon for Prometheus time series database for system data (both are often used together). Specifically, Timestream has a connector for pulling in data from the Telegraph; after release, AWS promises a two-way (read and write) connector for Prometheus.
Since the data is exposed as relational, there is not surprisingly a JDBC interface for SQL clients, which in turn can connect popular BI tools for visualization such as. Amazon QuickSight. There is also support for open source Grafana. On the machine learning side, there is an interface to Amazon SageMaker for developing predictable models. Amazon Timestream customers will be able to interact with the console, analytics apps and via the SDKs that AWS supports, such as Python, Go, Node and others. In addition, SDKs and connectors can be used to develop analytics and models in Jupyter notebooks.
Compared to InfluxDB, both are serverless, but unlike InfluxDB Cloud, Timestream does not use a special query language. Like TimescaleDB Cloud, Timestream is queried via SQL, but it is not a customization of PostgreSQL’s relational database.
On our wish list, we would like to see federated query from other AWS data platforms, e.g. Amazon Redshift for analysis, and Neptuneto provide a graphical display that could provide the interrelationships with insights that would be useful in related matters, such as intelligent supply networks and supply chain optimization.