Big Data Touches Everything from Distant Planets to Music Royalties

Muhammad Ali, the legendary boxer who had both the grace to float like a butterfly and the power and precision to sting like a bee, died yesterday in Phoenix at age 74. Ali dominated his sport in an era when athletes relied on instinct, raw ability and the limited capacity for observation of the human eye. In 2016, however, sports performance has been revolutionized by data analysis.

Ali will be remembered by many, if not most, as the greatest boxer of all time. We'll never know whether he could have been even better in an era when cameras pinpoint every move a boxer makes and feed that information into computers. It's certainly tempting to speculate, however, that even The Champ could have gone to another level with benefit of Big Data.

In some way or another, both our dramatically increased capacity for capturing data — making and cataloging thousands of observations in a matter of minutes, or sometimes even seconds — as well as our exploding ability to study and interpret all of that information is changing nearly everything. Big Data is having a big impact everywhere you look.

On Thursday, a commentator at Forbes reported about how Big Data has both helped and hindered the digital music industry. With computers instantly and accurately logging the distribution and penetration of digital music, the information needed to streamline and rationalize payment of royalties has been piling up for years. Yet it's only in recent months that tech wizards have cracked the tough nut of communcation and coordination between music tracking software and accounting software.

Sometimes the benefit of Big Data is more direct, running in a relatively straight line that revises our prior understanding of something by shedding new light. A few years back, as detailed in an engrossing Thursday post to the ISACA Now blog, astronomers used hyperaccurate surveys of the night sky to arrive at a new stellar classification of Pluto, now a mere dwarf planet instead of a full member of the local solar system.

On a more terrestrial scale, Big Data is assisting farmers in Africa, where soil samples are analyzed to assist in planting, fertilization and crop rotation. A brief feature in Tech Republic explains how a Dutch company has employed a combination of Big Data and mobile technology to analyze soil samples at the source, instead of relying on remote labs.

Big Data also has big potential to improve the healthcare industry — doctors and patients alike are intrigued by such possibilities as near-instantaneous and comprehensive analysis of individual medical histories. Here, as in the music industry, a major problem is interoperability. There are deep troves of data in various formats, but there's still work to be done to move it all into the same computational conversation.

There are other challenges. It's much cheaper than it used to be to more or less perpetually store massive data sets, but there are costs involved — and the data just keeps piling up. Beyond that, data frequently has multiple stakeholders, and it's far from cut-and-dried that data belongs to, and can be used at the sole discretion of, whoever happens to have collected it.

That doesn't mean that Big Data won't continue to be a superhero of the IT realm. On Thursday, a story in the Christian Science Monitor mused that Big Data may already be capable of doing something that Batman, Spider-Man and Superman all handle: fighting crime. Or, you know, at least contributing strongly to what one Chicago police chief calls "evidence-based policing."

Crime, on the other hand, may be small potatoes. A feature that appeared Friday at HealthData Management suggests that Big Data has a role to play in curing cancer.

Ultimately, whether the legacy of Big Data includes stunning advances in medicine, safer streets, better farming, or even just more exciting basketball games, it's clear that there is a vast need for professionals trained in all aspects of data management and data analysis. If you want to make a difference, then get certified and join the field. There are plenty of employers who'd be thrilled to bring more Big Data acumen aboard.