SanDisk have just created an SD card with 512 GB of memory, and it is expected that 2 TB will be achieved from the format. As someone who also has over 1TB in both my Microsoft OneDrive and Dropbox, but only 8GB on my university account, it seems that my corporate storage systems are not keeping up with the latest trend in the scale-up in cloud-based storage. It should be remember that security in the Cloud is not really an issue, as it is possible to storage into a cloud-based data bucket, and encrypt the data. So as data storage capacity has increased 1000-fold over the last 10 years, my corporate storage has increased by a factor of eight.
Intel created the first DRAM (dynamic random-access memory) chip in 1971. It was named the 1103 and could hold 1kB of data. DRAM chips are made up of small capacitors which are charged up with electrical charge (for a binary 1), or discharged (for a binary 0). As they use the charging up and discharging of capacitors, they tended to be slower than the static version – SRAM (dynamic random-access memory), which toggle the state of a pair of transistors. A SRAM needs a larger space than DRAM, DRAM has often been used to create larger memory storage chips than the equivalent SRAM ones. Both SRAM and DRAM lose their contents when the power is taken away (volatile memory), so storage systems use nonvolatile memory to preserve the data when the power is taken away.
Life on a postage stamp
The IBM PC, released in 1981, only had around 1MB of memory and a 30MB hard disk. Now we are looking at 2TB on a card the size of a postage stamp (where the area is mainly taken up with the physical layout of the card and connector). One of the key growth areas of computing is likely to be in-memory computing, where it is possible to store all the data you need locally on memory, and have no real need for connections to the Internet. With a 2TB data storage, you could probably hold all the data you are ever going to need.
So let’s look at Bob’s footprint over his 80 years on the planet:
- Email. Bob sends 300 emails a day, and receives 400, each have 1000 characters (1B), so that’s 700KB each day, and 255MB a year. Then over 80 years this generates 20GB of emails. Footprint: 0.1%.
- Photos. I take five photos every day, each are 1.5MB. This 547MB each year, and over 80 years it creates 43GB. Footprint: 2.15%.
- Documents. Bob creates 10 documents each day, with an average of 1MB for document, which creates 3.6GB over a year, and 292GB over 80 years. Footprint: 14.6%.
- Bob loves Wikipedia, which takes around 10GB of space, and this would take Footprint: 0.05%.
- Social media. Bob wants to save ever post that he has made to Facebook, Twitter, and other social media sites. This is an average 30MB of data each day. This gives 10.9GB of data each year, and 876GB of data over 80 years. Footprint: 43.8%.
- Video. Bob takes a 5 videos each week, with a size of 100MB. That’s 5.2GB each year, and 416GB for 80 years. Footprint: 20.8%.
So that is 81% of the SD card used and we have stored the whole of Bob’s life … every email, photo, media post … in fact everything on the size of a postage stage.
Why don’t corporate systems keep up with the Cloud?
Corporations are still struggling with the Cloud, and knowing how they should use public and private cloud, and how to create a single entity which keeps some things locally, but can burst into public cloud spaces (Figure 1). The problems with the scalability have continually been around performance, resilience, and security.
- Security. Security can be solved by creating an encryption layer for all the data which leaves the corporate infrastructure and is then stored in a public cloud.
- Resilience. With resilience, the main cloud providers such as Amazon AWS and Microsoft Azure have shown a near 100% up-time over the past few years, with very few problem with outages. The last major outage for Amazon Web Services (AWS) happened in August 2013 for nearly an hour and was due to issues in their North Virginia datacenter. It mainly affected Amazon.com, but it’s effect on many companies who had built their own business in the Cloud, such as Vine and Instagram.It is estimated that Amazon lost as much as $1,100 in net sales per second (to put into context a five-minute outage in August 2013 cost Google $545,000).
- Performance. Performance has always been an issue, especially where network connections are slow, or become busy over certain time periods. This as an issue is reducing as high-speed network connections provide fast response rates, especially where the content is placed at the edge of the public cloud.
Corporations are playing catch-up with the Cloud, and many are in the process of understanding how they can create a single Cloud infastructure, which is integrates their private cloud infastructure with a public one. In order to do this effectively they need to understand issues around security, performance and resiliance. When this is done, then perhaps I can get more than 8GB for my storage.
We now have the opportunity to store all the data that we need in-memory, and have no need to store in remote databases. Increasing applications may store all the data they need in-memory, and avoid fetching is from relatively slow network connections.