What’s Next for Big Data
When I attended the Strata Conference last month, there were 2 themes that struck me as being repeated throughout the conference and across various speakers: machine-learning and visualization tools.
For me, in light of the evolution of Big Data and machine-scale data creation, these ideas seem unavoidable.
The notion that as data volume and velocity grows exponentially, so too must our capacity to discover, classify, and manipulate that data; which leads to the inevitable conclusion that soon, our human-scale artisanal approach to data management will simply not be able to keep pace. The recent emphasis on expanding machine-learning capacity is a strong indicator that our processes are attempting to accelerate alongside our technology and that we are moving into an age of industrial-scale data management practices.
Similarly, the advances embedded in the current generation of visualization tools coming to market mean that vast quantities of information can now be displayed in ways more in-tune with our highly evolved pattern-matching brains. Out of the box, tools like Trifacta and Zoomdata simplify the visualization interface without giving up the power of analytics. Not surprisingly, the simplification of the visualization UX democratizes the data and opens the door to non-technical, business-driven data visualization.
As I mentioned in an earlier post, I think that we are nearing the end of the line for human-scale data, and by extension, human-scale data processing. Highlighting the role of machine-learning and advanced visualization techniques make sense to me and leave me excited about the future of data management practices.