For Start-Ups, Sorting the Data Cloud Is the Next Big Thing

The following excerpt is from the Business Section of the December 26 New York Times.

For Start-Ups, Sorting the Data Cloud Is the Next Big Thing

By MALIA WOLLAN

SAN FRANCISCO — The idea of big data goes something like this: In a world of ever-increasing digital connectivity, ever larger mountains of data are produced by our cellphones, computers, digital cameras, RFID readers, smart meters and GPS devices. The huge quantity of data becomes unwieldy and difficult for companies and governments to manage and understand.

“My smartphone produces a huge amount of data, my car produces ridiculous amounts of really valuable data, my house is throwing off data, everything is making data,” said Erik Swan, 47, co-founder of Splunk, a San Francisco-based start-up whose software indexes vast quantities of machine-generated data into searchable links.

Companies search those links, as one searches Google, to analyze customer behavior in real time.

Splunk is among a crop of enterprise software start-up companies that analyze big data and are establishing themselves in territory long controlled by giant business-technology vendors like Oracle and I.B.M.

Founded in 2004, before the term “big data” had worked its way into the vocabulary of Silicon Valley, Splunk now has some 3,200 customers in more than 75 countries, including more than half the Fortune 100 companies.

The amount of data being generated globally increases by 40 percent a year, according to the McKinsey Global Institute, the consulting firm’s research arm. And while Splunk has a lead in selling software to analyze machine data, big data is big enough to create new opportunities for a multitude of start-ups, many of them using the open-source software Hadoop.

==========