Skip to main content

Unlimited cloud storage provider used ancient technique to boost upload speeds — Son of Intel's Robson makes a timid return with SSD cache but on a much, much bigger scale

Blackblaze claims to have achieved upload speeds on its B2 Cloud Storage platform that are up to 30% faster than AWS S3 by tapping into a “shard stash” based on solid-state drives (SSDs).

When customers previously uploaded small files to Blackblaze B2, the data was written to several hard drives, and these had to be completed before returning a response to the client. But data is now written to HDDs as well as some SSDs – known as the ‘shard stash’ at the same time. The SSD-held data is stored only until HDDs have received all the data, after which point these copies are deleted. Since uploading to the SSD is much faster than the HDD, the result is faster uploads.

The firm benchmarked its new storage technology by uploading a 256kb file and a 1MB file to the Blackblaze B2 US East servers as well as the AWS S3 equivalents – and found the file was 30% faster than S3 for the former file and 10% faster for the latter.

Seeking inspiration from Intel’s Robson 

When a client application uploads files, Blackblaze normally deploys a coordinator pod to split them into 16 data shards alongside four parity shards. These 20 shares are then written to 20 different HDDs. 

When using HDDs, incidentally, most of the time it takes to write the file is dedicated to waiting for the drive platter to spin to the right location. This data is first written to in-memory cache, and then written to the physical disk in the future. Until this point, users won’t receive a “success” response. 

But even the best HDDs are much slower than SSDs, and the firm’s engineers have managed to find a way to incorporate them into the upload process and boost performance without raising costs too much. The new and improved B2 Cloud Storage lets the coordinator split files under 1MB into 20 shards as normal, which are sent to HDDs, but there are additional shards sent to servers fitted with ten Micron SSDs.

Because writing to the SSDs is much quicker, these “shard stash” servers can serve as a temporary but secure repository for the data, and send a “success” response to the user far quicker than using the previous model. Once the data is safely uploaded to the HDDs, the data is purged from the SSDs and can be reused.

It’s similar to Intel’s TurboMemory system that it developed in the 2000s, codenamed Robson, which was embedded in the best laptops of the era to boot systems much quicker.

More from TechRadar Pro



Comments

Post a Comment

Popular posts from this blog

TechRadar's Year in Review: AI revolution, smartphone evolution, and visions of the future

If I had to sum up 2023 in one word it would be AI. Okay, technically that’s two initials, but let’s not quibble – because there’s no doubt that it provided the year in tech with its main story arc. Artificial intelligence was everywhere this year, as every company in the world gawped at the massive impact of ChatGPT, and decided they wanted to jump aboard that bandwagon. You could barely move for new chatbots, new image generators, and new AI-enhanced features in every device released in 2023. And there’s no sign of that changing in 2024; in fact, we expect it to be an even bigger deal. Elsewhere, phones and laptops did much the same as they had the previous year – well, except where they had new AI-enhanced features – so it was in the virtual reality space where bigger changes were afoot. If the Meta Quest 3 was impressive then the Apple Vision Pro looks to be genuinely game-changing, and we can’t wait to see what impact it has in the next 12 months. Of course we also saw plenty ...
Post a Comment
Read more

Microsoft and Sony enter "binding agreement" to keep Call of Duty on PlayStation

Microsoft has announced that it has reached a "binding agreement" with Sony, pledging to keep Call of Duty on PlayStation following the acquisition of Activision Blizzard.  With Microsoft having won its anti-trust case with the US Federal Trade Commission , its planned acquisition of Call of Duty developers Activision Blizzard has taken a big step closer to becoming a reality. In order to alleviate the concerns of monopoly-building that got Microsoft in legal hot water to begin with, the head of Xbox, Phil Spencer, announced yesterday via Tweet that it has entered into an "agreement to keep Call of Duty on PlayStation."  This agreement will no doubt come as a relief to fans concerned that Microsoft's acquisition plans might result in Call of Duty becoming an Xbox exclusive, following in the footsteps of future Bethesda releases. Since Microsoft's acquisition of Bethesda caused the studio to break its tradition of offering titles cross-platform for upcomi...
Post a Comment
Read more

This Stardew Valley-like farming sim plays at the bottom of your screen so you work and play at the same time

Stardew Valley and Animal Crossing are among some of the best choices out there if you're looking for a game to get swept up in for hours at a time. But, if you're a fan of management sims and a multitasker as well, this upcoming farming sim will likely pique your interest.  Developed by Mister Morris Games, Rusty's Retirement is an idle-farming simulation game that runs at the bottom of your gaming monitor, allowing you to focus on other tasks like working while you tend to your crops. Set to release in Q1 2024 on Steam , the idle sim puts you in the shoes of a wholesome robot as you grow crops and use them to produce clean and energy-efficient biofuel. Having gathered this "vital robot power source," you'll then need to sell your harvest to the other machine inhabitants of your settlement. This, in turn, will let you reinvest the money into expanding your farm. If you're a terrible procrastinator (like me) this might not sound like your cup of tea...
Post a Comment
Read more