Skip to main content

Unlimited cloud storage provider used ancient technique to boost upload speeds — Son of Intel's Robson makes a timid return with SSD cache but on a much, much bigger scale

Blackblaze claims to have achieved upload speeds on its B2 Cloud Storage platform that are up to 30% faster than AWS S3 by tapping into a “shard stash” based on solid-state drives (SSDs).

When customers previously uploaded small files to Blackblaze B2, the data was written to several hard drives, and these had to be completed before returning a response to the client. But data is now written to HDDs as well as some SSDs – known as the ‘shard stash’ at the same time. The SSD-held data is stored only until HDDs have received all the data, after which point these copies are deleted. Since uploading to the SSD is much faster than the HDD, the result is faster uploads.

The firm benchmarked its new storage technology by uploading a 256kb file and a 1MB file to the Blackblaze B2 US East servers as well as the AWS S3 equivalents – and found the file was 30% faster than S3 for the former file and 10% faster for the latter.

Seeking inspiration from Intel’s Robson 

When a client application uploads files, Blackblaze normally deploys a coordinator pod to split them into 16 data shards alongside four parity shards. These 20 shares are then written to 20 different HDDs. 

When using HDDs, incidentally, most of the time it takes to write the file is dedicated to waiting for the drive platter to spin to the right location. This data is first written to in-memory cache, and then written to the physical disk in the future. Until this point, users won’t receive a “success” response. 

But even the best HDDs are much slower than SSDs, and the firm’s engineers have managed to find a way to incorporate them into the upload process and boost performance without raising costs too much. The new and improved B2 Cloud Storage lets the coordinator split files under 1MB into 20 shards as normal, which are sent to HDDs, but there are additional shards sent to servers fitted with ten Micron SSDs.

Because writing to the SSDs is much quicker, these “shard stash” servers can serve as a temporary but secure repository for the data, and send a “success” response to the user far quicker than using the previous model. Once the data is safely uploaded to the HDDs, the data is purged from the SSDs and can be reused.

It’s similar to Intel’s TurboMemory system that it developed in the 2000s, codenamed Robson, which was embedded in the best laptops of the era to boot systems much quicker.

More from TechRadar Pro



Comments

Post a Comment

Popular posts from this blog

Garmin's new radar-equipped tail light will keep you safe on your e-bike

Garmin's Varia bike radars are some of the most popular pieces of cycling tech around – and now the company has delivered its first rearview radar to have been specially designed for some of the best e-Bikes .   Garmin's Varia range mounts to the back of your bike and broadcasts a radar signal behind you, so you can get visual and audible alerts when something's overtaking you. Even better, the new Varia eRTL615 plugs directly into most e-bikes, with no battery required. Because the catchily-named Varia eRTL615 is also a tail light, it'll also make sure you're visible to other vehicles too, promising to emit a flashing or solid light that's visible from up to a mile away in daylight. To connect Garmin's new radar tail light to your e-bike, you'll need to pick the right Garmin adapter cable (which isn't included). You can buy power cables compatible with Bosch, Shimano, or USB-A terminals or connections, with more info on those available on Garmin...
Post a Comment
Read more

Revolution Software is using their own AI technology to remake Broken Sword

TechRadar Gaming is reporting live from Gamescom 2023 on the latest and greatest developments in gaming and hardware. Revolution Software announced at Gamescom 2023 that Broken Sword would be coming back, with Broken Sword - The Shadow of the Templars getting a full remake while a sixth title in the series is coming in the future too, under the title Broken Sword - Parzival’s Stone .  Speaking to TRG ahead of the announcement, Cecil talked about the studio’s plans for a Broken Sword remake and the sixth title in the series. Cecil is a larger-than-life character, who is able to talk about the studio’s plans with enthusiasm. It even carries a pocketful of stones to illustrate the plans for Parzival’s Stone , but he also talks about how Broken Sword - The Shadow of the Templars would be using AI to upscale.  Cecil wasn’t shy about the studio’s use of AI technology, but he gave a fairly robust explanation of why the game was using it. The AI technology will be used to upda...
Post a Comment
Read more

Hackers steal passwords, emails from hookup websites

Two gay hookup websites have been breached with sensitive and personal user data stolen and sold online, new reports have claimed. The databases, which are now being sold on dark web forums, were taken from platforms called TruckerSucker, and CityJerks. They contain enough personally identifiable information to engage in identity theft , such as usernames and passwords, email addresses, profile pictures, sexual preferences, birth dates, postal addresses, IP addresses, and bios. The passwords are encrypted, but according to TechCrunch, the algorithm is “weak” and could be broken by a more persistent hacker. The silent treatment HaveIBeenPwned founder Troy Hunt, who was tipped off on the leak, described the incident as a “typical forum breach, albeit with super sensitive content.”  However the content includes more than just identity data, as there are also messages users exchanged, including arranging meetings and describing their sexual preferences.  In total, more than...
Post a Comment
Read more