diff --git a/.Rhistory b/.Rhistory new file mode 100644 index 0000000..f7c61b9 --- /dev/null +++ b/.Rhistory @@ -0,0 +1,15 @@ +list.files("diet_data") +files_full <- list.files("diet_data", full.names=TRUE) +summary(files_full) +tmp <- vector(mode = "list", length = length(files_full)) +summary(tmp) +for (i in seq_along(files_full)) { +tmp[[i]] <- read.csv(files_full[[i]]) +} +str(tmp) +str(tmp[[1]]) +head(tmp[[1]][,"Day"]) +output <- do.call(rbind, tmp) +str(output) +output <- rbind(tmp[[1]], tmp[[2]], tmp[[3]], tmp[[4]], tmp[[5]]) +str(output) diff --git a/FAQ.md b/FAQ.md new file mode 100644 index 0000000..017c758 --- /dev/null +++ b/FAQ.md @@ -0,0 +1,90 @@ +##FAQs +I thought I would pull together some frequently asked questions regarding Programming Assignment 1 that regularly show up on the forums. + +####1) My code runs fine and my answers match the sample output, but whenever I try to submit, I get a message telling me that my answer is incorrect. + +You're not submitting via the Coursera website are you? You need to re-read the Assignment 1 instructions (all of them). + +Instead of submitting via the website, you need to use the `submit()` script. A link and more detailed instructions are included in the "Grading" section of the assignment 1 instructions. + +If you're using the `submit()` script and still getting this error, double check to make sure you're not printing the results rather than returning them. In other words, the final line of your code should not contain `print()`. Use `return()` instead (you may not have to do this as R automagically returns the final line of a function). + +####2) Do I need to round my answers to match the sample output? + +No. You don't need to do any rounding of your results to pass the submission tests. + + + +####3) I only see 3 parts to the assignment. What are these 10 parts listed on the assignment page? + +You're correct that there are only 3 parts to assignment 1. The 10 parts could probably be more accurately described as tests. The `submit()` script will run your code with a variety of different parameters to test it. If there are issues with your function, it may only pass some of the tests. + + + +####4) My `pollutantmean()` passes the first 3 tests, but fails the 4th with the error message: "Error in pollutantmean("specdata", "nitrate") : argument "id" is missing, with no default" + +You didn't assign a default value to `id`. The first line of your function should look exactly like the one in the instructions: `pollutantmean <- function(directory, pollutant, id = 1:332) {` + + + +####5) I get an error stating "unexpected '>' " or "unexpected '{' ". + +You probably have an open `(` somewhere in your code. Double check it with a fine tooth comb to make sure you've closed all of your `()`, `{}`and `[]`. + + + +####6) My code seems to work but my answers don't match the sample output. + +Are you calculating the mean value for each file and then taking the mean of those means? That's not the correct approach. You need to combine all of the relevant data into a single data frame or vector and take the mean of *that*. + + a <- c(1:5) + b <- c(1:10) + c <- c(10:15) + + mean(c(a,b,c)) + [1] 6.904762 + + mean(c(mean(a),mean(b),mean(c))) + [1] 7 + +You want the first approach, not the second. + + + +####7) My function seems to work when `id` is a single value but I get the following error message when it's something like `70:72`: "In pollutant1$ID == 1:332 : longer object length is not a multiple of shorter object length". + +You probably have a line of code that looks something like this: `dat_subset <- dat[which(dat[ , "ID"] == id), ]` + +Subsetting by `ID` works when `id` is a vector of length 1. However, when `id = 1:10` for example, you have a problem. The issue goes back to the SWIRL example (and maybe lecture?) regarding how R handles vectors of differing lengths. + +An example: + + > x <- 1:10 + > y <- 1:5 + > x + y + [1] 2 4 6 8 10 7 9 11 13 15 + +Each value of y gets added to x. But because y is shorter than x, after adding 5+5, R starts over from the beginning of y and adds 6+1, 7+2, etc. + +That's what is happening with the subset when id is a vector longer than 1. The first row gets compared to the first value of id. The second row gets compared to the second value of id, etc. It repeats id until it gets to the end of the vector or data frame you're subsetting. The warning is telling you that the length of dat$nitrate or dat$sulfate is not divisible by the length of id. + +Essentially, there are 2 options to solve this. The first is to not use a subset for `id` at all. You presumably already have a loop in your code, so instead of combining all 332 files together, why not use that loop to combine only the files specified by `id` in the first place? + +The other alternative is to replace the `==` with `%in%`. In this case, the %in% operator will check each value of `id` against every value in the `ID` column, which is what you want. The downside to this approach is that it will probably be very, very slow if you've followed the tutorial example to create `pollutantmean()`. + + + +####8) How do I subset for either `nitrate` or `sulfate` when I calculate the mean? + +If you wanted to subset nitrate, you would do that with `dat[, "nitrate"]`. Likewise you would use `dat[, "sulfate"]` for sulfate. When the function gets called you'll have something like: `pollutantmean(directory = "specdata", pollutant = "nitrate", id = 1:332)`. + +So if you have either `pollutant = "nitrate"` or `pollutant = "sulfate"`, what would you put in place of `"sulfate"` and `"nitrate"` in subsetting examples above so that it would work in either case? + + + +####9) I'm subsetting my data frame using `dat$pollutant` but it doesn't seem to be working. + +Recall from the lectures that $ makes R look for a literal name match. That's not what you want. You want to subset by the value of pollutant (either "sulfate" or "nitrate"), not by "pollutant" since you don't have a column by that name. So, you need to use brackets instead of $. + +That could be either something like [[pollutant]] or [, pollutant] + diff --git a/Practice_Assignment.pdf b/Practice_Assignment.pdf index 9188dff..3601d36 100644 Binary files a/Practice_Assignment.pdf and b/Practice_Assignment.pdf differ diff --git a/README.md b/README.md index 5e66448..d421f91 100644 --- a/README.md +++ b/README.md @@ -3,12 +3,10 @@ practice_assignment There's been a lot of discussion regarding the significant gap between the lectures from weeks 1 and 2 and Programming Assignment 1 in the R Programming class on Coursera. -If you're not familiar with R and you're not familiar with programming concepts (sometimes even if you are), Programming Assignment 1 can feel like you're getting thrown in the deep end of the pool after a 5 minute lesson on how to swim. +While technically you're shown everything you need to know in order to complete the assignment, if you're not familiar with programming concepts (sometimes even if you are), Programming Assignment 1 can feel like you're getting thrown in the deep end of the pool after a 5 minute lesson on how to swim. I've created a practice programming assignment that should help bridge the gap. It walks you through some basic concepts and ultimately helps you create a function bringing those concepts all together. It does not provide a solution for Programming Assignment 1, but if you can work your way through this guided example, you should have the tools to complete part 1 of assignment 1. Part 2 and 3 are a little more complicated, but build on the ideas from part 1. The instructions are available in rmd and pdf formats. -Just unzip the diet_data.zip file into your R working directory so that you've got a folder called 'diet_data' which contains 5 csv files. I've included directions for doing this within R in the instructions. - -Hopefully others will find this useful. I'll keep tabs on this thread to answer any questions. +Also, once you start working on assignment 1, take a look at the FAQ included at the end of the practice assignment. It addresses probably 80% of the questions people have when they get stuck. diff --git a/diet_data.zip b/diet_data.zip deleted file mode 100644 index 7c5e164..0000000 Binary files a/diet_data.zip and /dev/null differ diff --git a/practice_assignment.rmd b/practice_assignment.rmd index a3955fc..ed9026d 100644 --- a/practice_assignment.rmd +++ b/practice_assignment.rmd @@ -1,7 +1,13 @@ --- +title: R Programming Practice Assignment output: pdf_document --- -Practice Assignment + +##Derek Franks + +###Twitter: @derek_franks + + ======================================================== The goal of this assignment is to provide a "bridge" between the first two weeks of lectures and assignment 1 for those either new to R or struggling with how to approach the assignment. @@ -47,7 +53,7 @@ Alternatively, you could look at the dimensions of the data.frame: dim(andy) ``` -This tells us that we 30 rows of data in 4 columns. There are some other commands we might want to run to get a feel for a new data file, `str()`, `summary()`, and `names()`. +This tells us that we have 30 rows of data in 4 columns. There are some other commands we might want to run to get a feel for a new data file, `str()`, `summary()`, and `names()`. ```{r} str(andy) @@ -67,7 +73,7 @@ We can do the same thing to find his final weight on Day 30: andy[30, "Weight"] ``` -Alternatively, you could create a subset of the 'Weight' column where the data where 'Day' is equal to 30. +Alternatively, you could create a subset of the 'Weight' column where the value of the 'Day' column is equal to 30. ```{r} andy[which(andy$Day == 30), "Weight"] andy[which(andy[,"Day"] == 30), "Weight"] @@ -229,15 +235,16 @@ Essentially, these are all things that we've done in this example. Now we just So what does the function look like? -```{r} +```{r tidy=TRUE, tidy.opts=list(width.cutoff=60)} weightmedian <- function(directory, day) { files_list <- list.files(directory, full.names=TRUE) #creates a list of files dat <- data.frame() #creates an empty data frame - for (i in 1:5) { #loops through the files, rbinding them together + for (i in 1:5) { + #loops through the files, rbinding them together dat <- rbind(dat, read.csv(files_list[i])) } - dat_subset <- dat[which(dat[, "Day"] == day),] #subsets the rows that match the 'day' argument - median(dat_subset$Weight, na.rm=TRUE) #identifies the median of the subset + dat_subset <- dat[which(dat[, "Day"] == day),] #subsets the rows that match the 'day' argument + median(dat_subset[, "Weight"], na.rm=TRUE) #identifies the median weight #while stripping out the NAs } ``` @@ -250,4 +257,161 @@ weightmedian("diet_data", 4) weightmedian("diet_data", 17) ``` -Hopefully, this has given you some practice applying the basic concepts from weeks 1 and 2. If you can work your way through this example, you should have all of the tools needed to complete part 1 of assignment 1. Parts 2 and 3 are really just expanding on the same basic concepts, potentially adding in some ideas like cbinds and if-else. \ No newline at end of file +Hopefully, this has given you some practice applying the basic concepts from weeks 1 and 2. If you can work your way through this example, you should have all of the tools needed to complete part 1 of assignment 1. Parts 2 and 3 are really just expanding on the same basic concepts, potentially adding in some ideas like cbinds and if-else. + +*** + +##FAQs +I thought I would pull together some frequently asked questions regarding Programming Assignment 1 that regularly show up on the forums. If you run into a problem you can't solve while working on assignment 1, take a look through this section. + +####1) My code runs fine and my answers match the sample output, but whenever I try to submit, I get a message telling me that my answer is incorrect. + +You're not submitting via the Coursera website are you? You need to re-read the Assignment 1 instructions (all of them). + +Instead of submitting via the website, you need to use the `submit()` script. A link and more detailed instructions are included in the "Grading" section of the assignment 1 instructions. + +If you're using the `submit()` script and still getting this error, double check to make sure you're not printing the results rather than returning them. In other words, the final line of your code should not contain `print()`. Use `return()` instead (you may not have to do this as R automagically returns the final line of a function). + +####2) Do I need to round my answers to match the sample output? + +No. You don't need to do any rounding of your results to pass the submission tests. + + + +####3) I only see 3 parts to the assignment. What are these 10 parts listed on the assignment page? + +You're correct that there are only 3 parts to assignment 1. The 10 parts could probably be more accurately described as tests. The `submit()` script will run your code with a variety of different parameters to test it. If there are issues with your function, it may only pass some of the tests. + + + +####4) My `pollutantmean()` passes the first 3 tests, but fails the 4th with the error message: "Error in pollutantmean("specdata", "nitrate") : argument "id" is missing, with no default" + +You didn't assign a default value to `id`. The first line of your function should look exactly like the one in the instructions: `pollutantmean <- function(directory, pollutant, id = 1:332) {` + + + +####5) I get an error stating "unexpected '>' " or "unexpected '{' ". + +You probably have an open `(` somewhere in your code. Double check it with a fine tooth comb to make sure you've closed all of your `()`, `{}`and `[]`. + + + +####6) My code seems to work but my answers don't match the sample output. + +Are you calculating the mean value for each file and then taking the mean of those means? That's not the correct approach. You need to combine all of the relevant data into a single data frame or vector and take the mean of *that*. + + a <- c(1:5) + b <- c(1:10) + c <- c(10:15) + + mean(c(a,b,c)) + [1] 6.904762 + + mean(c(mean(a),mean(b),mean(c))) + [1] 7 + +You want the first approach, not the second. + + + +####7) My function seems to work when `id` is a single value but I get the following error message when it's something like `70:72`: "In pollutant1$ID == 1:332 : longer object length is not a multiple of shorter object length". + +You probably have a line of code that looks something like this: `dat_subset <- dat[which(dat[ , "ID"] == id), ]` + +Subsetting by `ID` works when `id` is a vector of length 1. However, when `id = 1:10` for example, you have a problem. The issue goes back to the SWIRL example (and maybe lecture?) regarding how R handles vectors of differing lengths. + +An example: + + > x <- 1:10 + > y <- 1:5 + > x + y + [1] 2 4 6 8 10 7 9 11 13 15 + +Each value of y gets added to x. But because y is shorter than x, after adding 5+5, R starts over from the beginning of y and adds 6+1, 7+2, etc. + +That's what is happening with the subset when id is a vector longer than 1. The first row gets compared to the first value of id. The second row gets compared to the second value of id, etc. It repeats id until it gets to the end of the vector or data frame you're subsetting. The warning is telling you that the length of dat$nitrate or dat$sulfate is not divisible by the length of id. + +Essentially, there are 2 options to solve this. The first is to not use a subset for `id` at all. You presumably already have a loop in your code, so instead of combining all 332 files together, why not use that loop to combine only the files specified by `id` in the first place? + +The other alternative is to replace the `==` with `%in%`. In this case, the %in% operator will check each value of `id` against every value in the `ID` column, which is what you want. The downside to this approach is that it will probably be very, very slow if you've followed the tutorial example to create `pollutantmean()`. + + + +####8) How do I subset for either `nitrate` or `sulfate` when I calculate the mean? + +If you wanted to subset nitrate, you would do that with `dat[, "nitrate"]`. Likewise you would use `dat[, "sulfate"]` for sulfate. When the function gets called you'll have something like: `pollutantmean(directory = "specdata", pollutant = "nitrate", id = 1:332)`. + +So if you have either `pollutant = "nitrate"` or `pollutant = "sulfate"`, what would you put in place of `"sulfate"` and `"nitrate"` in subsetting examples above so that it would work in either case? + + + +####9) I'm subsetting my data frame using `dat$pollutant` but it doesn't seem to be working. + +Recall from the lectures that $ makes R look for a literal name match. That's not what you want. You want to subset by the value of pollutant (either "sulfate" or "nitrate"), not by "pollutant" since you don't have a column by that name. So, you need to use brackets instead of $. + +That could be either something like [[pollutant]] or [, pollutant] + + +###10) My code runs for a really long time (especially on submission #4) + +This is due to the way that the loop is being used in this practice assignment. The approach I'm showing for building the data frame is suboptimal. It works, but generally speaking, you don't want to build data frames or vectors by copying and re-copying them inside of a loop. If you've got a lot of data it can become very, very slow. However, this tutorial is meant to provide an introduction to these concepts, and you can use this approach successfully for programming assignments 1 and 3. + +If you're interested in learning the better approach, check out Hadley Wickam's excellent material on functionals within R: http://adv-r.had.co.nz/Functionals.html. But if you're new to both programming and R, I would skip it for now as it will just confuse you. Come back and revisit it (and the rest of this section) once you are able to write working functions using the approach above. + +So for those of you that do want to see a better way to create a dataframe.... + +As I said before, the main issue with the approach above is growing an object inside of a loop by copying and recopying it. It works, but it's slow and inefficient. The better approach is to create an output object of an appropriate size and then fill it up. + +So the first thing we do is create an empty list that's the length of our expected output. In this case, our input object is going to be `files_full` and our empty list is going to be `tmp`. + +```{r} +summary(files_full) +tmp <- vector(mode = "list", length = length(files_full)) +summary(tmp) +``` + +Now we need to read in those csv files and drop them into `tmp`. + +```{r} +for (i in seq_along(files_full)) { + tmp[[i]] <- read.csv(files_full[[i]]) +} +str(tmp) +``` + +What we just did was read in each of the csv files and place them inside of our list. Now we have a list of 5 elements called `tmp`, where each element of the list is a data frame containing one of the csv files. It just so happens that this is functionally identical to using `lapply`. + +```{r} +str(lapply(files_full, read.csv)) + +``` + +This is part of the power of the apply family of functions. You don't have to worry about the "housekeeping" of looping, and instead you can focus on the function you're using. When you or somebody else comes back weeks or months later and reads through your code, it's easier to understand what you were doing and why. If somebody says that the apply functions are more "expressive", this is what they're talking about. + +Now we still need to go from a list to a single data frame, although you *can* manipulate the data within this structure: + +```{r} +str(tmp[[1]]) +head(tmp[[1]][,"Day"]) +``` + + +One way to do this would be to manually `rbind` everything together: + +```{r} +output <- rbind(tmp[[1]], tmp[[2]], tmp[[3]], tmp[[4]], tmp[[5]]) +str(output) +``` + + +But of course there's a better way. We can use a function called `do.call()` to combine `tmp` into a single data frame with much less typing. `do.call` lets you specify a function and then passes a list as if each element of the list were an argument to the function. + +The syntax is `do.call(function_you_want_to_use, list_of_arguments)`. In our case, we want to `rbind()` our list of data frames, `tmp`. + + +```{r} +output <- do.call(rbind, tmp) +str(output) +``` + +This approach avoids all of the messy copying and recopying of the data to build the final data frame. It's much more "R-like" and works quite a bit faster than our other approach.