There can be an assortment of reasons why, and some of them may not be related to the database. Investigating bottlenecks in the query performanceīefore looking for a solution, it is important to first understand why the queries are slow. Let’s walk through some of the techniques we did to discover and fix this. In particular, we discovered in our investigation that our configurations for our database could have been modified for improved performance. After some investigation, we discovered that the main bottleneck was the performance of retrieving data from our PostgreSQL database replica.Īlthough introducing external caching stores like Redis or Memcached to our applications would have been beneficial, we wanted to make sure that we were properly utilizing our database. They couldn’t process the events quickly enough during times of peak traffic, so notifications were delayed to our customers. In fact, that was exactly what happened to our email and push sending pipelines earlier last year. A difference of several tens of milliseconds in process time per update event is enough to disrupt our pipelines and prevent us from delivering updates to our users in a timely manner. For the Notifications team at Redfin (the team I work on), it ensures that we can capture the updates in the US and Canadian real estate markets and inform our users within minutes of them occurring. The ability to fetch data quickly is crucial for any high throughput application and service. I, however, want to discuss a less commonly acknowledged cache that also helps with improving data retrieval time: a cache within a database. When thinking about caching data for incredibly fast retrieval, technologies like Redis and Memcached often come to a developer’s mind.

Cache too high