Big Data

Excerpt from an article in The New York Times
Monday, February 20, 2012

I.B.M.: Big Data, Bigger Patterns 

By QUENTIN HARDY It's not just about Big Data. For the big players in enterprise technology algorithms, it's about finding big patterns beyond the data itself.

The explosion of online life and cheap computer hardware have made it possible to store immense amounts of unstructured information, like e-mails or Internet clickstreams, then search the stored information to find some trend that can be exploited. The real trick is to do this cost-effectively. Companies doing this at a large scale look for similarities between one field and another, hoping for a common means of analysis.

When it comes to algorithms, "if I can do a power grid, I can do water supply," said Steve Mills, I.B.M.'s senior vice president for software and systems. Even traffic, which like water and electricity has value when it flows effectively, can reuse some of the same algorithms. Mr. Mills, speaking at a Goldman Sachs technology conference in San Francisco on Wednesday, called it "leveraging the cost structure of new mathematics."

That kind of cross-pollination is reminiscent of the way Wall Street, starting in the 1990s, hired astrophysicists and theoretical mathematicians to design arcane financial products. Now the cost of computing has come down so much that it is useful to bring such talent to other industries. I.B.M., Mr. Mills said, is now the largest employer of Ph.D. mathematicians in the world, bringing their talents to things like oil exploration and medicine. "On the side we're doing astrophysics, genomics, proteomics," he said.