Skip to main content

Unlimited cloud storage provider used ancient technique to boost upload speeds — Son of Intel's Robson makes a timid return with SSD cache but on a much, much bigger scale

Blackblaze claims to have achieved upload speeds on its B2 Cloud Storage platform that are up to 30% faster than AWS S3 by tapping into a “shard stash” based on solid-state drives (SSDs).

When customers previously uploaded small files to Blackblaze B2, the data was written to several hard drives, and these had to be completed before returning a response to the client. But data is now written to HDDs as well as some SSDs – known as the ‘shard stash’ at the same time. The SSD-held data is stored only until HDDs have received all the data, after which point these copies are deleted. Since uploading to the SSD is much faster than the HDD, the result is faster uploads.

The firm benchmarked its new storage technology by uploading a 256kb file and a 1MB file to the Blackblaze B2 US East servers as well as the AWS S3 equivalents – and found the file was 30% faster than S3 for the former file and 10% faster for the latter.

Seeking inspiration from Intel’s Robson 

When a client application uploads files, Blackblaze normally deploys a coordinator pod to split them into 16 data shards alongside four parity shards. These 20 shares are then written to 20 different HDDs. 

When using HDDs, incidentally, most of the time it takes to write the file is dedicated to waiting for the drive platter to spin to the right location. This data is first written to in-memory cache, and then written to the physical disk in the future. Until this point, users won’t receive a “success” response. 

But even the best HDDs are much slower than SSDs, and the firm’s engineers have managed to find a way to incorporate them into the upload process and boost performance without raising costs too much. The new and improved B2 Cloud Storage lets the coordinator split files under 1MB into 20 shards as normal, which are sent to HDDs, but there are additional shards sent to servers fitted with ten Micron SSDs.

Because writing to the SSDs is much quicker, these “shard stash” servers can serve as a temporary but secure repository for the data, and send a “success” response to the user far quicker than using the previous model. Once the data is safely uploaded to the HDDs, the data is purged from the SSDs and can be reused.

It’s similar to Intel’s TurboMemory system that it developed in the 2000s, codenamed Robson, which was embedded in the best laptops of the era to boot systems much quicker.

More from TechRadar Pro



Comments

Post a Comment

Popular posts from this blog

The latest Apple TV 4K test lets you watch four sports streams at once

Apple is trying something new with the latest beta version of tvOS 16.5: the option to watch up to four simultaneous streams at once. Right now it's limited to live sports streamed through the Apple TV app on the Apple TV 4K , specifically MLB Friday Night Baseball and the MLS Season Pass. A multi-view option was spotted in the tvOS software last month, but the code was hidden and not enabled. MacRumors reported that the feature would be enabled this weekend, and beta testers have since been able to use it. As yet multi-view hasn't been officially announced by Apple, but it's expected that tvOS 16.5 is going to be pushed out in its final form within the next month or so. WWDC 2023 is around the corner as well, when we should be hearing about the next major updates for Apple's various operating systems – including tvOS 17. How it works Over at 9to5Mac there's a hands-on demonstrating how the multi-view feature works, and it's pretty much as you would expe...
Post a Comment
Read more

Quantum computers are fast becoming cheaper and smaller — and they could be coming to a data center near you very soon

IonQ claims we’re closer to widespread enterprise quantum computing deployment as it lifted the lid on two rack-mounted models that can be deployed on-premises.   The startup has built the fourth-generation #AQ35 IonQ Forte Enterprise and fifth-generation #AQ64 IonQ Tempo, both of which are designed to be deployed in enterprise and government data centers. It’s also said it is deploying two quantum computers to the US Air Force.  While revealing these two models, IonQ co-founder and CTO Jungsang Kim said quantum computers are already in use by enterprises to churn through machine learning workloads. This, he added, suggests we’re much closer to readily available and affordable machines. Priming enterprises for a quantum future “We believe in the enterprise-grade quantum computing, which is where it can be something of value for enterprises, can happen in the next few years as we build powerful enough quantum computers that can actually do things that classical computers w...
Post a Comment
Read more

Nvidia RTX 4080 GPU could get cheaper with a new version – but don’t get your hopes up

Nvidia’s RTX 4080 is purportedly getting a new spin on the GPU which could reduce the cost, but any price reduction will likely be very minor, sadly, if it happens at all. Tom’s Hardware flagged up this rumor – and treat it with caution, as with anything from the ever-spinning mill – that originated from HKEPC (a tech site in Hong Kong), claiming that while the current RTX 4080 graphics card is built on the AD103-300 chip, Nvidia is going to use a slightly different GPU in the future, namely AD103-301. There’s now more evidence this is actually happening, Tom’s points out, courtesy of a graphics card maker, Galax, which under its RTX 4080 product details lists the GPU as ‘AD103-300/301’. Furthermore, VideoCardz , which also picked up on this, informs us that Gainward, another card maker, has also listed the updated GPU variant AD103-301 in its product specs. With two separate third-party graphics card makers mentioning this new spin on the GPU in their specs, it seems pret...
Post a Comment
Read more