Experiences with Amazon Redshift

In my current role as a Principal Architect, we were given a new project to create a data warehouse system. We would be storing huge amount of data in the warehouse and would be making a lot queries on the database.

Since, the data was critical in nature we had to introduce encryption aswell in the database. We did a comparative study of Cassandra vs ElasticSearch vs Amazon Redshift.

Following were the outcomes of our study-

1. Amazon Redshift is a Postgres based columnar database. It’s an OLAP db in nature. It’s mainly used in analytics to be made over big data.
2. In terms of Search, the first time query execution is slow(ran over records of 2.5cr running in seconds). However, further executions of the same query are way too fast(running in millisecs).It appears that the query compliation time is a time consuming process .
3. Bulk insertions are very fast, however single inserts are time consuming.
4. Setup and configuring the DB is a cakewalk and AWS provides it’s own sample data to test your use-cases.

Overall, it’s more of a data warehouse on which one can run analytics/aggregation. For search related operations, ElasticSearch by far looks the best.