What's happening with Bunny Storage S3 compatibility?

We initially announced that we were working on S3 support for Bunny Storage all the way back in 2022. Today, 2023 is slowly coming to an end, and many of our customers continue to follow our blog, hoping for good news about the release.

At bunny.net, we value transparency. As such, we wanted to share some updates and insights into the delay and also share a rough roadmap as we work to get the fastest cloud storage in the world compatible with the S3 protocol.

The Delay

First, we wanted to share the current state of the project and the reasons for the long delay. Initially, when we announced S3 support for Bunny Storage, the project was in the final stages of development and almost ready to go live.

As we entered the stress and scalability testing stages, however, we realized that we would be unable to adapt the existing system to work efficiently with an S3-compatible API.

Due to early architectural decisions for Bunny Storage, we used a Cassandra database to store and query the file information. Cassandra allowed us to scale practically infinitely. Today, we store tens of billions of files and many petabytes of data.

However, selecting Cassandra also meant we had to design a very specific data querying and storage model to achieve good performance. Unfortunately, it turned out that this design varies significantly between the current Bunny Storage architecture and the S3 requirements.

After careful consideration, we decided that to achieve long-term scalability, performance, and innovation goals, we wanted to completely rework Bunny Storage, how we store data, and built it with the idea of being S3-first. With years of experience handling various edge case scenarios, we decided to take all the experience we've gained and build the system to support the long-term goals.

The Reasons

Rebuilding Bunny Storage, a globally distributed storage platform that automatically replicates files to over 15 regions around the world in close to real-time, is a massive undertaking. However, we've already made some very good progress. We were able to take everything we've learned through the years and design a much more scalable, robust, and future-proof system.

At bunny.net, we believe in shipping fast, but due to significant challenges, we decided to take the time necessary to focus on building a system that will scale with our long-term goals and provide an exceptional experience for our users.

We understand that this could be very frustrating to everyone patiently waiting for the fastest globally distributed storage to get S3 compatibility. So much so, in fact, that we have developed an internal joke around it.

The Roadmap

Today, we wanted to share some updates on our progress so far together with a readjusted timeline of the target release dates for S3. Our team is working diligently to build an incredible product to fit with the rest of bunny.net ecosystem.

Unfortunately, due to the complexity of Bunny Storage, it's difficult to give an exact date, but we are working aggressively toward shipping S3 compatibility in early 2024. We are continuing to also update and improve the existing Bunny Storage APIs and add new features for existing use cases.

We will do our best to keep everyone updated in the process.

Why are you launching X and not S3?

We also wanted to take this opportunity to answer a question we get asked a lot: why are you spending your time developing feature X, when you promised S3 compatibility years ago?

The answer is simple: bunny.net operates multiple independent engineering teams, each dedicated to its own project. While S3 support is one of the key priorities for bunny.net, we also have many different teams working on exciting new projects as we continue to pursue our mission of making the internet hop faster.

That being said, we have some very exciting other new updates lined up to announce over the next months, but rest assured, our storage team is busy working on the next generation of Bunny Storage at the same time!

Help us build S3 faster!

If you excel at developing complex distributed systems on a massive scale, are passionate about networking and storage, and really want to see S3 happen, make sure to check out our careers page. bunny.net is growing very quickly, and we're looking for exceptional talent to join our team and help us ship S3 even faster!