Project

Coffee ratings: variability, outliers and shape

How much do the ratings of 1,339 coffee samples differ? Range, IQR, standard deviation, fences, skewness and cleaning faulty values.

10 steps~50 min200 points in total

The Coffee Quality Institute had trained graders taste and score thousands of coffee samples from all over the world. In this project you work with 1,339 samples and describe how much the ratings differ: you will compute measures of variability, find outliers, decide which of them are errors and which are genuine exceptions, and find out what a single strange sample does to the measures.

Each step is a separate task with a check and hints. The code steps already have the data loaded, so a mistake in one step does not spoil the next ones.

Data

The file kava.csv: semicolon separator, decimal point, an empty cell means a missing value. One row is one rated coffee sample. The column names and processing categories are in Slovak and stay that way in the code.

column	meaning	unit or values
`celkove_body`	overall quality rating	points, scale 0 to 100
`druh`	coffee species	Arabica, Robusta
`krajina`	country of origin	36 countries
`sposob_spracovania`	processing of the cherries after harvest	`premývaná` (washed), `sušená` (natural, dried), `polopremývaná` (semi-washed), `medová` (honey), `iná` (other)
`aroma`, `chut`, `kyslost`, `telo`	partial sensory ratings: aroma, flavour, acidity, body	points, scale 0 to 10
`vyska_m`	mean altitude of the farm	metres above sea level
`pocet_vriec`	number of bags the rating applies to	count
`vady_1`, `vady_2`	number of severe and less severe defects of the green coffee	count

The point ratings are scales without a natural zero: 0 points for flavour does not mean "no flavour".

Source: the Coffee Quality Institute ratings database collected by James LeDoux (MIT licence) and made available by the TidyTuesday project (CC0). The column names and processing categories are translated, the values are original.

Data

kava.csv1339 rows, 12 columns

Download CSV

First 5 rows of the file

celkove_body	druh	krajina	sposob_spracovania	aroma	chut	kyslost	telo	vyska_m	pocet_vriec	vady_2
90.58	Arabica	Ethiopia	premývaná	8.67	8.83	8.75	8.5	2075.0	300	0
89.92	Arabica	Ethiopia	premývaná	8.75	8.67	8.58	8.42	2075.0	300	1
89.75	Arabica	Guatemala	NaN	8.42	8.5	8.42	8.33	1700.0	5	0
89.0	Arabica	Ethiopia	sušená	8.17	8.58	8.42	8.5	2000.0	320	2
88.83	Arabica	Ethiopia	premývaná	8.25	8.5	8.5	8.42	2075.0	300	2

Steps

The steps build on each other. In code steps the data and the results of earlier steps are already prepared.

Step 1
How much does flavour differ
code20 pts
pandasrangeIQRstandard deviation
Describe how much the flavour rating differs (chut, scale 0 to 10). The table df is already loaded.
1. Store the range in rozpatie and the interquartile range of flavour in iqr.
2. Store the sample standard deviation (dividing by $n - 1$ ) in sd and the population standard deviation (dividing by $n$ ) in sd_pop.
Python loads when you first use the editorPackages:pandas
The output appears after you run the code.
Run the code and fix the errors until all checks pass.
Step 2
Why the range is 20 times larger
choice15 pts
rangeIQRoutliers
The flavour rating has a range of 8.83 points, but an interquartile range of only 0.42 points.

What is the best explanation?
Pick one option.
Step 3
Fences and outliers
code20 pts
pandasIQRoutliersbox plot
Find the samples with an outlying flavour rating.
1. Store the fences by the $1.5 \cdot IQR$ rule in dolny_plot (lower fence) and horny_plot (upper fence).
2. Store the number of samples below the lower fence in pocet_pod and above the upper fence in pocet_nad.
3. Draw a horizontal box plot of flavour.
Python loads when you first use the editorPackages:pandasmatplotlib
The output appears after you run the code.
Run the code and fix the errors until all checks pass.
Step 4
Impossible altitudes
code25 pts
pandasdata cleaningoutliersmedianstandard deviation
The column vyska_m (mean altitude of the farm) has 1,109 known values. Some of them look suspicious.
1. Store the number of samples with an altitude above 8,849 m (Mount Everest) in nemozne.
2. Store the altitudes without these impossible values in vyska_ok.
3. Store the median and the sample standard deviation before and after cleaning in median_pred, median_po, sd_pred and sd_po (pred = before, po = after).
Python loads when you first use the editorPackages:pandas
The output appears after you run the code.
Run the code and fix the errors until all checks pass.
Step 5
Error or exception?
choice15 pts
outliersdata cleaning
For the altitude of the farm, the upper fence is 2,350 m. Above it lies, for example, the value 190,164 m, but also 2,560 m.

What will you do with these two values?
Pick one option.
Step 6
Skewness and kurtosis of flavour
code20 pts
pandasskewnesskurtosisshape of distribution
Find out what shape the distribution of flavour has and how much a single sample with a flavour rating of 0 points changes it.
1. Store the skewness and kurtosis of the flavour of all samples in sikmost (skewness) and spicatost (kurtosis).
2. Store the flavour without the sample with the zero in chut_bez_nuly and its skewness and kurtosis in sikmost_bez_nuly and spicatost_bez_nuly.
Python loads when you first use the editorPackages:pandas
The output appears after you run the code.
Run the code and fix the errors until all checks pass.
Step 7
What a kurtosis of 94 means
choice15 pts
kurtosisshape of distribution
For the flavour of all 1,339 samples, pandas chut.kurt() returned the value 94.75.

What does this say about the distribution of flavour?
Pick one option.
Step 8
Where the coefficient of variation makes sense
choice15 pts
coefficient of variationmeasurement scales
A colleague wants to compare the variability of different columns using the coefficient of variation. The columns are pocet_vriec (number of bags), chut (flavour), kyslost (acidity) and celkove_body (overall points).

For which column does the coefficient of variation have a substantive meaning?
Pick one option.
Step 9
Variability by processing method
code25 pts
pandasgroupbyIQRbox plotcomparing groups
Does the flavour of coffee differ by processing method (sposob_spracovania)?
1. Store the count, median and standard deviation of flavour for each processing method in porovnanie (columns count, median, std).
2. Add a column iqr with the interquartile range of flavour in each group.
3. Draw box plots of flavour side by side for washed (premývaná) and natural, dried (sušená) coffee.
Python loads when you first use the editorPackages:pandasmatplotlib
The output appears after you run the code.
Run the code and fix the errors until all checks pass.
Step 10
Washed or natural?
text30 pts
interpretationcomparing groupsvariabilitybox plot
A roastery is deciding whether to buy washed (premývaná) or natural, dried (sušená) coffee. It wants to know which has a better and more consistent flavour.

Based on the previous step, write a recommendation (5 to 8 sentences). Compare the typical flavour and the variability of both groups with specific numbers, say whether the data support claiming a difference, and state what to watch out for.
Your answer
0 words, minimum 6060 words to go