KeyDB: the multithreaded fork of Redis

At Hosted Power we set ourselves apart by delivering environments with the very fastest performance. We constantly strive to remain the highest-performing player on the market.  
That is why innovation is a major focus at Hosted Power. We continuously improve our processes and tooling, which underpin the high performance of the mission-critical environments we provide for your websites and applications. 

This is where a technology such as Redis quickly comes into the spotlight. Redis is an in-memory datastore used for caching in our environments. It lets us access and retrieve data quickly without having to access the hard drive, resulting in minimal response times. 

The open source community has long been putting pressure on Redis to make its system truly multithreaded. However, for reasons such as ease of programming and cost-effective implementation, Redis decided to retain its single-threaded architecture. 

Of course, this eventually led to the emergence of multithreaded alternatives to the caching system, prompting us at Hosted Power to investigate these alternative technologies. Notably, some benchmarks showed performance results 5 to as much as 25 times higher than those of Redis. In other words, our curiosity was piqued, so we started testing.

One alternative emerged as the most stable and compatible option for a professional environment: KeyDB.

KeyDB is a multithreaded fork of Redis. This means it is fully compatible with the Redis protocol, changes and modules, while promising to keep up with changes Redis makes in the future.  

KeyDB is not only multithreaded, it also offers several unique advantages: 

• higher performance with less hardware 
• a smaller cluster thanks to active replication (that is, two master nodes accept both read and write operations) 
• the ability to create your own command using ModJS  
• promising upcoming features such as FLASH storage, JSON support and multi-tenant support.  

All this sounds promising, but we still rely most on our own test results to substantiate these expectations, and we are happy to share them with you. 

Testing KeyDB 

To generate the best possible results, we used a benchmarking tool developed by RedisLabs , called memtier_benchmark. It can generate different traffic patterns against Redis instances. This allowed us to simulate various scenarios, varying the number of clients and the number of requests they generated across different configurations, from 2xCPU/4GB RAM to 8xCPU/32GB RAM. 

The results did not disappoint! We started testing in an environment with just 2 cores and 4GB of memory. Here, we tested 100 clients, each generating 100, 1000 and 10 000 requests. 

As the results show, Redis performs much better than KeyDB, while keeping average latency lower. This is what we might expect, since KeyDB should improve significantly once more threads are assigned. 

 Testing with 4 cores therefore supports our hypothesis: 

From 4 cores onwards, KeyDB starts outperforming Redis across all metrics. KeyDB performs 66% more operations per second and achieves higher data throughput than Redis. As expected, we got even better results when we increased the number of threads to six. 

KeyDB doubles the data throughput and operations per second that Redis can deliver. A clear pattern is emerging. To confirm the significant speed gains once more, we increased the number of threads to 8 cores.

Time to draw a conclusion: KeyDB starts to show its advantage with at least 4 cores, while the highest throughput was achieved with a setup of 6 threads.  

Latency is not a metric shown in the graphs above, but it was included in the test results, which showed that KeyDB maintains a lower average latency.  

The future looks promising for this Redis alternative. Since KeyDB already outperforms an optimised Redis setup, we will also look into implementing KeyDB in our own TurboStack® setup. Every feature that boosts performance motivates us to guarantee more speed for you, our customer, on your mission-critical application, shop or website.