A healthy Amazon Redshift cluster must not have more than 80% of storage utilization, but this threshold had been crossed. This caused high disk usage per node whenever disk-intensive queries were run (and these queries were already optimized). High disk usage and high storage utilization – The storage utilization of the Amazon Redshift cluster crossed 90% because each dc2.8xlarge node had less SSD space (2.56 TB).In addition to the pain of purchasing extra compute, the team faced some challenges: This led to an increase in the operating costs for supporting mission-critical applications. As a result, more compute was purchased whenever additional storage was required. The previous generation of Amazon Redshift instances didn’t allow you to scale compute and storage resources separately because compute and storage were tightly coupled in a node. However, Customer Service has been using dense-compute nodes over dense-storage nodes to support real-time analytical dashboards that require high-speed retrievals using NVMe-based SSDs offered exclusively by the dense-compute nodes. In fact, Customer Service’s storage capacity needs have grown faster than computing power needs. General challengesĪs the Amazon Customer Service data warehouse continues to grow in volume, additional storage is required to store the new information. The new nodes enabled the team to plan and scale storage resources flexibly without having to purchase additional compute, and they can continuously innovate with the highest standards for their customers. Upgrading to 3 RA3 nodes allowed the team to not only save up to 55% on Amazon Redshift operating costs per year, but also improve the load times of most dashboards up to 47% and the data retrieval performance of most queries up to 25%. The team operated 8 DC2 nodes, but they required 6 nodes of computing power and 10–12 nodes of storage unit capacity. The RA project was a major strategic move for the Amazon Customer Service Technology team. Moving to the most advanced Amazon Redshift architecture enabled the team to reduce its infrastructure costs, improve the performance of queries, optimize its compute and storage capacity planning processes, and accelerate the performance of analytical dashboards for making business-critical decisions. In 2021, the Amazon Customer Service Technology team upgraded its dense-compute nodes (dc2.8xlarge) to the Amazon Redshift RA3 instance family (ra3.16xlarge). The additional features Automatic Vacuum Delete, Automatic Table Sort, and Automatic Analyze eliminate the need for manual maintenance and tuning of Redshift clusters to get the best performance for new clusters and production workloads.Amazon Customer Service solves exciting and challenging customer care problems for, the world’s largest online retailer. If Amazon Redshift determines that applying a key will improve cluster performance, tables will be automatically altered without requiring administrator intervention. Automatic Table Optimization selects the best sort and distribution keys to optimize performance for the cluster’s workload. For dynamic workloads where query patterns are not predictable, Automated Materialized Views improve throughput of queries, lower query latency, shorten execution time through automatic refresh, auto query rewrite, incremental refresh, and continuous monitoring of Amazon Redshift clusters. ![]() Redshift Advisor makes recommendations when an explicit user action is needed to further turbocharge Redshift performance. In addition, you can now easily set the priority of your most important queries, even when hundreds of queries are being submitted. Automatic workload management (WLM) uses ML to dynamically manage memory and concurrency, helping maximize query throughput. Short query acceleration (SQA) sends short queries from applications such as dashboards to an express queue for immediate processing rather than being starved behind large queries. ![]() ![]() Sophisticated algorithms to predict and classify incoming queries based on their run times and resource requirements to dynamically manage performance and concurrency while also helping you prioritize your business-critical workloads.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |