No, data isn't the new oil...

by Simon Crosby, on May 10, 2021 8:15:00 AM

It’s a catchy phrase, I guess, for the twitterati.  But apart from the obvious - there's lots of oil and we're drowning in data - the image tells you everything about why this analogy is profoundly wrong: My car runs (sadly, till the next upgrade) on gas.  Not oil.  And when I upgrade it, I’ll leap beyond the oil-based economy to something sustainable and electronic.  And floods of data from sensors in my car and the roads will be refined in real-time to guide me.  Maybe a better phrase would be: “Real-time insights help Teslas pick optimal routes”?  Notice: no value in raw data (oil).

The quip is useful in one respect:  Orgs that can quickly transform masses of raw data into insights (viz: oil refined into gasoline) are in a more powerful position: Instead of owning a voluminous commodity (oil), they can derive competitive advantage from the high value and lower volume of their insights (gas).  Sure (yawn), “big oil = big data”.  To that I’d add “= big shipping, storage, processing headache”.  Saving data for all time leads to huge, undifferentiated, valueless data-lakes.  Expensive shuffling of undifferentiated bits for no organizational win.  And you pay your cloud provider a lot, or buy tons of disk.

“Store-then-analyze” - batch-based big-data processing - is too slow for today’s dynamic economy, too expensive to operate, and results in a finger-pointing, backward looking organizational attitude toward data in which everyone hopes for the best (“the engineers said save all the data, so we did! - but nobody analyzed it!”). Organizations need to make sense of data fast to drive decisions and endable automation.  They need continuous intelligence, not continuous storage bills.

Swim is one of the companies leading the charge toward continuous, real-time insights - no matter how much data you have.  In the terms of the analogy, we deliver electricity directly from the well-head, in real time.  It’s a million times faster.  Still want to ship, store, and batch-process your oil at 10x the cost?  Thought so.