Tuesday, February 18, 2014

Marginally Interesting: Data Analysis: The Hard Parts

Excerpt from a timely data analysis reality check

"The general message is: Data analysis has become super easy. But has it? I think people want it to be, because they have understood what data analysis can do for them, but there is a real shortage in people who are good at it. So the usual technological solution is to write tools which empower more people do it. And for many problems, I agree that this is how it works. You don’t need to know TCP/IP to fetch some data from the Internet because there are libraries for that, right?
For a number of reasons, I don’t think that you cannot “toolify” data analysis that easily. I wished it would be, but from my hard-won experience with my own work and teaching people this stuff, I’d say it takes a lot of experience to be done properly and you need to know what you’re doing. Otherwise you will do stuff which breaks horribly once put into action on real data."
Marginally Interesting: Data Analysis: The Hard Parts

No comments: