Suggest/EDA.qmd merge two sections of unusual values

EDA.qmd

@@ -73,7 +73,7 @@ You can see variation easily in real life; if you measure any continuous variabl
 This is true even if you measure quantities that are constant, like the speed of light.
 Each of your measurements will include a small amount of error that varies from measurement to measurement.
 Variables can also vary if you measure across different subjects (e.g., the eye colors of different people) or at different times (e.g., the energy levels of an electron at different moments).
-Every variable has its own pattern of variation, which can reveal interesting information about how that it varies between measurements on the same observation as well as across observations.
+Every variable has its own pattern of variation, which can reveal interesting information about how it varies between measurements on the same observation as well as across observations.
 The best way to understand that pattern is to visualize the distribution of the variable's values, which you've learned about in @sec-data-visualization.
 
 We'll start our exploration by visualizing the distribution of weights (`carat`) of \~54,000 diamonds from the `diamonds` dataset.
@@ -151,7 +151,21 @@ Some of these questions can be answered with the data while some will require do
 Many of them will prompt you to explore a relationship *between* variables, for example, to see if the values of one variable can explain the behavior of another variable.
 We'll get to that shortly.
 
-### Unusual values
+#### Exercises
+
+1.  Explore the distribution of each of the `x`, `y`, and `z` variables in `diamonds`.
+    What do you learn?
+    Think about a diamond and how you might decide which dimension is the length, width, and depth.
+
+2.  Explore the distribution of `price`.
+    Do you discover anything unusual or surprising?
+    (Hint: Carefully think about the `binwidth` and make sure you try a wide range of values.)
+
+3.  How many diamonds are 0.99 carat?
+    How many are 1 carat?
+    What do you think is the cause of the difference?
+
+### Unusual values {#sec-unusual-values-eda}
 
 Outliers are observations that are unusual; data points that don't seem to fit the pattern.
 Sometimes outliers are data entry errors, sometimes they are simply values at the extremes that happened to be observed in this data collection, and other times they suggest important new discoveries.
@@ -161,7 +175,7 @@ The only evidence of outliers is the unusually wide limits on the x-axis.
 
 ```{r}
 #| fig-alt: |
-#|   A histogram of lengths of diamonds. The x-axis ranges from 0 to 60 and 
+#|   A histogram of widths of diamonds. The x-axis ranges from 0 to 60 and 
 #|   the y-axis ranges from 0 to 12000. There is a peak around 5, and the 
 #|   data appear to be completely clustered around the peak.
 
@@ -174,7 +188,7 @@ To make it easy to see the unusual values, we need to zoom to small values of th
 
 ```{r}
 #| fig-alt: |
-#|   A histogram of lengths of diamonds. The x-axis ranges from 0 to 60 and the 
+#|   A histogram of widths of diamonds. The x-axis ranges from 0 to 60 and the 
 #|   y-axis ranges from 0 to 50. There is a peak around 5, and the data 
 #|   appear to be completely clustered around the peak. Other than those data, 
 #|   there is one bin at 0 with a height of about 8, one a little over 30 with 
@@ -185,7 +199,7 @@ ggplot(diamonds, aes(x = y)) +
   coord_cartesian(ylim = c(0, 50))
 ```
 
-`coord_cartesian()` also has an `xlim()` argument for when you need to zoom into the x-axis.
+`coord_cartesian()` also has an `xlim` argument for when you need to zoom into the x-axis.
 ggplot2 also has `xlim()` and `ylim()` functions that work slightly differently: they throw away the data outside the limits.
 
 This allows us to see that there are three unusual values: 0, \~30, and \~60.
@@ -212,36 +226,16 @@ options(old)
 ```
 
 The `y` variable measures one of the three dimensions of these diamonds, in mm.
-We know that diamonds can't have a width of 0mm, so these values must be incorrect.
+We know that diamonds can't have a width of 0mm, so these must be missing values.
 By doing EDA, we have discovered missing data that was coded as 0, which we never would have found by simply searching for `NA`s.
 Going forward we might choose to re-code these values as `NA`s in order to prevent misleading calculations.
-We might also suspect that measurements of 32mm and 59mm are implausible: those diamonds are over an inch long, but don't cost hundreds of thousands of dollars!
+We might also suspect that measurements of 32mm and 59mm are implausible: those diamonds are over an inch wide, but don't cost hundreds of thousands of dollars!
 
 It's good practice to repeat your analysis with and without the outliers.
 If they have minimal effect on the results, and you can't figure out why they're there, it's reasonable to omit them, and move on.
 However, if they have a substantial effect on your results, you shouldn't drop them without justification.
 You'll need to figure out what caused them (e.g., a data entry error) and disclose that you removed them in your write-up.
 
-### Exercises
-
-1.  Explore the distribution of each of the `x`, `y`, and `z` variables in `diamonds`.
-    What do you learn?
-    Think about a diamond and how you might decide which dimension is the length, width, and depth.
-
-2.  Explore the distribution of `price`.
-    Do you discover anything unusual or surprising?
-    (Hint: Carefully think about the `binwidth` and make sure you try a wide range of values.)
-
-3.  How many diamonds are 0.99 carat?
-    How many are 1 carat?
-    What do you think is the cause of the difference?
-
-4.  Compare and contrast `coord_cartesian()` vs. `xlim()` or `ylim()` when zooming in on a histogram.
-    What happens if you leave `binwidth` unset?
-    What happens if you try and zoom so only half a bar shows?
-
-## Unusual values {#sec-unusual-values-eda}
-
 If you've encountered unusual values in your dataset, and simply want to move on to the rest of your analysis, you have two options.
 
 1.  Drop the entire row with the strange values:
@@ -299,7 +293,7 @@ You can do this by making a new variable, using `is.na()` to check if `dep_time`
 #| fig-alt: |
 #|   A frequency polygon of scheduled departure times of flights. Two lines 
 #|   represent flights that are cancelled and not cancelled. The x-axis ranges 
-#|   from 0 to 25 minutes and the y-axis ranges from 0 to 10000. The number of 
+#|   from 0 to 25 hours and the y-axis ranges from 0 to 10000. The number of 
 #|   flights not cancelled are much higher than those cancelled.
 
 nycflights13::flights |> 
@@ -316,17 +310,21 @@ nycflights13::flights |>
 However this plot isn't great because there are many more non-cancelled flights than cancelled flights.
 In the next section we'll explore some techniques for improving this comparison.
 
-### Exercises
+#### Exercises
+
+1.  Compare and contrast `coord_cartesian()` vs. `xlim()` or `ylim()` when zooming in on a histogram.
+    What happens if you leave `binwidth` unset?
+    What happens if you try and zoom so only half a bar shows?
 
-1.  What happens to missing values in a histogram?
+2.  What happens to missing values in a histogram?
     What happens to missing values in a bar chart?
     Why is there a difference in how missing values are handled in histograms and bar charts?
 
-2.  What does `na.rm = TRUE` do in `mean()` and `sum()`?
+3.  What does `na.rm = TRUE` do in `mean()` and `sum()`?
 
-3.  Recreate the frequency plot of `scheduled_dep_time` colored by whether the flight was cancelled or not.
+4.  Recreate the frequency plot of `scheduled_dep_time` colored by whether the flight was cancelled or not.
     Also facet by the `cancelled` variable.
-    Experiment with different values of the `scales` variable in the faceting function to mitigate the effect of more non-cancelled flights than cancelled flights.
+    Experiment with different values of the `scales` argument in the faceting function to mitigate the effect of more non-cancelled flights than cancelled flights.
 
 ## Covariation
 
@@ -340,9 +338,9 @@ For example, let's explore how the price of a diamond varies with its quality (m
 
 ```{r}
 #| fig-alt: |
-#|   A frequency polygon of prices of diamonds where each cut of carat (Fair, 
+#|   A frequency polygon of prices of diamonds where each cut of diamond (Fair, 
 #|   Good, Very Good, Premium, and Ideal) is represented with a different color 
-#|   line. The x-axis ranges from 0 to 30000 and the y-axis ranges from 0 to 
+#|   line. The x-axis ranges from 0 to 20000 and the y-axis ranges from 0 to 
 #|   5000. The lines overlap a great deal, suggesting similar frequency 
 #|   distributions of prices of diamonds. One notable feature is that 
 #|   Ideal diamonds have the highest peak around 1500.
@@ -362,7 +360,7 @@ Instead of displaying count, we'll display the **density**, which is the count s
 ```{r}
 #| fig-alt: |
 #|   A frequency polygon of densities of prices of diamonds where each cut of 
-#|   carat (Fair, Good, Very Good, Premium, and Ideal) is represented with a 
+#|   diamond (Fair, Good, Very Good, Premium, and Ideal) is represented with a 
 #|   different color line. The x-axis ranges from 0 to 20000. The lines overlap 
 #|   a great deal, suggesting similar density distributions of prices of 
 #|   diamonds. One notable feature is that all but Fair diamonds have high peaks 
@@ -470,9 +468,9 @@ One way to do that is to rely on the built-in `geom_count()`:
 #| fig-alt: |
 #|   A scatterplot of color vs. cut of diamonds. There is one point for each
 #|   combination of levels of cut (Fair, Good, Very Good, Premium, and Ideal) 
-#|   and color (D, E, F, G, G, I, and J). The sizes of the points represent 
+#|   and color (D, E, F, G, H, I, and J). The sizes of the points represent 
 #|   the number of observations for that combination. The legend indicates 
-#|   that these sizes range between 1000 and 4000.
+#|   that these sizes mainly range between 1000 and 4000.
 
 ggplot(diamonds, aes(x = cut, y = color)) +
   geom_count()
@@ -496,7 +494,7 @@ Then visualize with `geom_tile()` and the fill aesthetic:
 #|   cut/color combination and tiles are colored according to the number of 
 #|   observations in each tile. There are more Ideal diamonds than other cuts, 
 #|   with the highest number being Ideal diamonds with color G. Fair diamonds 
-#|   and diamonds with color I are the lowest in frequency.
+#|   and diamonds with color J are the lowest in frequency.
 
 diamonds |> 
   count(color, cut) |>  
@@ -511,7 +509,7 @@ For larger plots, you might want to try the heatmaply package, which creates int
 
 1.  How could you rescale the count dataset above to more clearly show the distribution of cut within color, or color within cut?
 
-2.  What different data insights do you get with a segmented bar chart if color is mapped to the `x` aesthetic and `cut` is mapped to the `fill` aesthetic?
+2.  What different data insights do you get with a segmented bar chart if `color` is mapped to the `x` aesthetic and `cut` is mapped to the `fill` aesthetic?
     Calculate the counts that fall into each of the segments.
 
 3.  Use `geom_tile()` together with dplyr to explore how average flight departure delays vary by destination and month of year.
@@ -545,7 +543,7 @@ You've already seen one way to fix the problem: using the `alpha` aesthetic to a
 #| fig-alt: |
 #|   A scatterplot of price vs. carat. The relationship is positive, somewhat 
 #|   strong, and exponential. The points are transparent, showing clusters where 
-#|   the number of points is higher than other areas, The most obvious clusters 
+#|   the number of points is higher than other areas. The most obvious clusters 
 #|   are for diamonds with 1, 1.5, and 2 carats.
 
 ggplot(smaller, aes(x = carat, y = price)) + 
@@ -696,8 +694,8 @@ Once you've removed the strong relationship between carat and price, you can see
 ```{r}
 #| fig-alt: |
 #|   Side-by-side box plots of residuals by cut. The x-axis displays the various 
-#|   cuts (Fair to Ideal), the y-axis ranges from 0 to almost 5. The medians are 
-#|   quite similar, between roughly 0.75 to 1.25. Each of the distributions of 
+#|   cuts (Fair to Ideal), the y-axis ranges from 0 to almost 4. The medians are 
+#|   between roughly 0.75 to 1.25, and increase in better cuts. Each of the distributions of 
 #|   residuals is right skewed, with many outliers on the higher end.
 
 ggplot(diamonds_aug, aes(x = cut, y = .resid)) +