So what should be in the toolkit of people who call themselves a data scientist?
A fundamental skill is the ability to manipulate data. A data scientist should be familiar and comfortable with a number of platforms and scripting tools to get the job done. What is difficult in Excel might be trivial in R. And when R struggles, you should switch to Unix (or use a programming language such as Python) get that portion of the data munging done. Along the way, you pick up a lot of tips and tricks. For example: how to read a big datafile in R?
The goal is to get the job done. Familiarity with a wide variety of tools, and expertise in some is the hallmark of any good would-be data scientist.
Showing posts with label dataset manipulation. Show all posts
Showing posts with label dataset manipulation. Show all posts
Wednesday, December 7, 2011
Monday, October 17, 2011
Get the Basics right - Suggestion for R Beginners
I am always looking for suggestions on how to get better at R, esp. for beginners. So when I see someone who's gotten adept at it, I ask them how they got there.
This weekend, at the Bay Area ACM Data Mining Camp, one person gave me what seemed like a good suggestion. Just wanted to share it here, for anyone else who's just getting started.
He told me that there are tons and tons of libraries, and if you start going down that path, you might know how to use a library or two, but you may not learn the basics of data manipulation, which is one of R's main strengths.
His suggestion:
Get some data and learn to manipulate it - understand the differences between vectors, data-frames, arrays and matrices. Once you have this down, only then should you start exploring the different libraries. Don't rush in to try every new library that someone praises.
This weekend, at the Bay Area ACM Data Mining Camp, one person gave me what seemed like a good suggestion. Just wanted to share it here, for anyone else who's just getting started.
He told me that there are tons and tons of libraries, and if you start going down that path, you might know how to use a library or two, but you may not learn the basics of data manipulation, which is one of R's main strengths.
His suggestion:
Get some data and learn to manipulate it - understand the differences between vectors, data-frames, arrays and matrices. Once you have this down, only then should you start exploring the different libraries. Don't rush in to try every new library that someone praises.
Wednesday, October 12, 2011
A true data-doodler - Christophe Ladroue (R ddly and plyr on Triathlon Results)
To me, this post by Christophe Ladroue personifies what data doodlers do.
They take a dataset that is of interest to them (In his case, his triathlon results) and then they manipulate the numbers to see what insights can be drawn. Most bloggers only show their final results which look great, but for our purposes (for wannabe data doodlers) it is much more fun to see the process. It is often messy, but that's the way we learn.
In Christophe's example, you will see some data cleanup, then he plots averages and medians across categories. Then he starts to try to squeeze insights out of the day. That is true data analysis.
Check out his full post.
They take a dataset that is of interest to them (In his case, his triathlon results) and then they manipulate the numbers to see what insights can be drawn. Most bloggers only show their final results which look great, but for our purposes (for wannabe data doodlers) it is much more fun to see the process. It is often messy, but that's the way we learn.
In Christophe's example, you will see some data cleanup, then he plots averages and medians across categories. Then he starts to try to squeeze insights out of the day. That is true data analysis.
Check out his full post.
Subscribe to:
Posts (Atom)