What's happening with Bunny Storage S3 compatibility?

We initially announced that we were working on S3 support for Bunny Storage all the way back in 2022. Today, as 2023 is slowly coming to an end, many of our customers continue to follow our blog, hoping for good news about the release.

At bunny.net, we value transparency. As such, we wanted to share some updates and insights into the delay, and also share a rough roadmap as we work to get the fastest cloud storage in the world compatible with the S3 protocol.

The delay

First, we wanted to share the current state of the project and the reasons for the long delay. Initially, when we announced S3 support for Bunny Storage, the project was in the final stages of development and almost ready to go live.

As we entered the stress and scalability testing stages, however, we realized that we would be unable to adapt the existing system to work efficiently with an S3-compatible API.

Due to early architectural decisions for Bunny Storage, we used a Cassandra database to store and query file information. Cassandra allowed us to scale practically infinitely. Today, we store tens of billions of files and many petabytes of data.

However, selecting Cassandra also meant we had to design a very specific data querying and storage model to achieve good performance. Unfortunately, it turned out that this design differs significantly between the current Bunny Storage architecture and the S3 requirements.

After careful consideration, we decided that to meet our long-term scalability, performance, and innovation goals, would require completely reworking Bunny Storage. This includes how we store data, with the system being rebuilt from the ground up to be S3-first. With years of experience handling various edge-case scenarios, we're using everything we've learned to build a system that supports our long-term goals.

The reasons

Rebuilding Bunny Storage, a globally distributed storage platform that automatically replicates files to over 15 regions around the world in near real-time, is a massive undertaking. However, we've already made strong progress. We've taken everything we've learned over the years and designed a much more scalable, robust, and future-proof system.

At bunny.net, we believe in shipping fast, but due to the scale of the challenges, we chose to take the time needed to focus on building a system that will scale with our long-term goals and provide an exceptional experience for our users.

We understand that this may be very frustrating for everyone patiently waiting for the fastest globally distributed storage to gain S3 compatibility. So much so, in fact, that we've developed an internal joke around it.

The roadmap

Today, we wanted to share some updates on our progress so far, along with a revised timeline for the target S3 release. Our team is working diligently to build an incredible product that fits with the rest of the bunny.net ecosystem.

Unfortunately, due to the complexity of Bunny Storage, it's difficult to give an exact date. But we are working aggressively toward shipping S3 compatibility in early 2024. We're also continuing to update and improve the existing Bunny Storage APIs and add new features for current use cases.

We'll do our best to keep everyone updated along the way.

Why are you launching X and not S3?

We also wanted to take this opportunity to answer a question we get asked a lot: Why are you spending your time developing feature X when you promised S3 compatibility years ago?

The answer is simple: bunny.net operates multiple independent engineering teams, each focused on its own project. While S3 support is one of the key priorities, we also have many different teams working on exciting new projects as we continue to pursue our mission of making the internet hop faster.

That said, we have some exciting new updates lined up to announce over the next months. But rest assured, our storage team is actively working on the next generation of Bunny Storage at the same time!

Help us build S3 faster!

If you excel at developing complex distributed systems on a massive scale, are passionate about networking and storage, and really want to see S3 happen, check out our careers page. bunny.net is growing quickly, and we're looking for exceptional talent to help us ship S3 even faster!