Hi, I'm Adriene Hill, and Welcome back to Crash Course, Statistics.
In the last episode we talked about Null Hypothesis Significance testing and p-values and how
these two things help us make decisions about things we care about.
Like whether babies who drink non-dairy milk are more likely to have allergies, or whether
the number of hours you spend watching home makeover shows tends to increase with age.
We don't always come up with the right answer, even if it seemed reasonable.
We want to limit our errors as much as possible.
Today we'll talk about when and why we might get it wrong.
INTRO
In the last episode we briefly touched on "rejecting" the null hypothesis.
P-values tell us how "rare" or "extreme" our sample data would be if it really did
come from the null distribution.
Null means nothing so null hypotheses tend to say that there's no effect, or nothing's
going on.
For example, for whether babies who drink non-dairy milk are more likely to have allergies,
the null hypothesis (or H0) would be that there is no difference in proportion of babies
with allergies between babies who drink non-dairy milk, and those who do not.
In the case of home makeover shows, the null hypothesis might be that there's no relationship.
So the regression slope--or coefficient--between number of home makeover shows watched and
age would be 0:
By looking at this slope, we can see it's not exactly flat, but we don't know whether
this slope is due to a real relationship, or just random variation.
When we get low p-values, we "reject" the null hypothesis because we've decided
that our data would be pretty rare if the null was true since the probability of getting
data as or more extreme than ours is below our alpha level.
That's option 1.
Option 2 is that our p-value is not lower than our pre-selected cutoff which means that
we "fail to reject" the null hypothesis.
So, we've narrowed it down to two decisions: we can either reject, or fail to reject the null.
The null can either be true, or not true.
This means that there are four possible situations: either you correctly reject the null, mistakenly
reject the null, correctly fail to reject the null, or mistakenly fail to reject the null.
In two of these situations we make the correct decisions, and in the other two, we'd have
made an error.
The first error is called a Type I error, which is rejecting the null, even if it's true.
It can therefore only happen if the null is true.
Say we've decided that our alpha level is 0.05, so we'll reject the null if our p-value
is smaller than 0.05, which means that our sample is in the 5% most extreme values we
can expect to get if the null hypothesis were true.
So, if the null is true, 5% of the time, we'll still reject it mistakenly, just because we
happened to get a rare value.
The red shaded region represents all the values from the null distribution that would cause
us to decide to "reject" the null, even if it was true.
Since our type 1 error rate is equal to alpha, we get to choose exactly how often we are
willing to make Type 1 errors when we choose our alpha.
We control our Type I errors by explicitly deciding how often we'll make them.
We could also make an error by failing to reject the null hypothesis when it actually is false.
In order for the null hypothesis to be false, some other, alternative, hypothesis must be true.
We mentioned in the last episode that we don't actually know any specifics about which hypothesis
is correct when we "reject the null", it could be anything.
But we can estimate which distribution might be correct, we'll show it outlined in gray,
this helps us to compare two distributions instead of just looking at one.
We estimate the alternative distribution based on the mean and standard deviation of our
experimental group.
The sample mean is our best guess at what he effect size is, so we often use that if
we're estimating the alternative after we've collected our data.
But sometimes we want to estimate it before we collect data, in which case we use the
sample estimates from other, related studies.
We're assuming the Alternative (Ha) distribution looks like this.
Our cutoff line is still in the same place; it marks the cutoffs that tell us where the
5% most extreme values are.
Any value we get that is to the right of the line causes us to "reject the null" and
any value to the left of the line causes us to "fail to reject the null".
The cutoff value doesn't change depending on whether H0 or HA is true.
So, if the alternative is true, we still might fail to reject the null if we happen to get
a value that is to the left of the cutoff.
The blue shaded region shows you the values where we'll make this Type II error.
Just like the rate of Type I errors is equal to alpha, the rate of type II errors is equal to Beta.
Since we're only estimating what the alternative distribution looks like, we can't know what
Beta is for sure, but again we can estimate it by using our cutoff (alpha) and our best
estimates of the shape and position of our alternative distribution to find the approximate
area of the shaded region.
There's often a trade off between Type I and Type II errors.
Type I errors are essentially False positives: we think we've detected an effect, but there isn't one.
And Type II errors are False negatives: there was an effect, we just didn't see it.
And while both of these mean we were wrong, there's a lot of times where we may prefer
one type of error over the other.
Take smoke alarms.
While the sound of the smoke alarm going off is annoying, there's not a lot of cost to
having a false positive--or type I error.
All you have to do is press a button to reset it.
There is however a huge risk if your smoke alarm does not go off when there really is a fire.
For this reason, fire alarms tend to favor having type I errors over type II errors.
Which is why sometimes particularly long, hot showers can cause them to go off.
Better safe than sorry.
But yeah, turning off an alarm when you're naked and wet.
Not fun.
Think about someone in your life who is constantly worried, they operate on the assumption that
Type I errors--thinking there'll be an issue when there won't be--are preferable to Type
II errors--not preparing for a problem when there really could be one.
You can see on this graph that if we assume our null distribution is here, and the alternative
is here, then moving the cutoff threshold to the right will cause us--all other things
being equal--to have fewer type I errors.
But we'll have more type II errors since less of the null distribution is in the "reject"
region, and more of the alternative distribution is in the "fail to reject region".
And the opposite happens if we move our cutoff threshold to the left.
We'll have more False positives since more of the null is in the "reject" region,
but fewer False Negatives because less of the alternative distribution is in the "fail
to reject" region.
If the error types hard to keep straight, think of the Boy who cried wolf.
In that story the villagers first made a type I error (thinking there was a wolf when there
really wasn't), but by the end--and to the detriment of the little boy--they made a type
II error: thinking there WASN'T a wolf when there really.
Sometimes we do make the right decision and there are two ways to be right: either the
null hypothesis is true and we fail to reject it, or the null hypothesis is false and we
do reject it.
If the null is true, you'll reject it 1 - alpha of the time.
When alpha is 0.05 that means that when the null is true, we'll correctly fail to reject
0.95 or 95% of the time.
If the null is false and the alternative is true, we'll correctly reject the null 1-Beta
of the time.
If Beta--the proportion of times we will fail to reject the null even though it's false--is
10%, then we'll correctly reject the null 90% of the time.
This proportion (1-Beta) is called our statistical power.
As the name suggests, statistical power is really important and something that we want.
I mean, it's a power.
I want powers!
Statistical power tells us our chance of detecting an effect if there is one.
Imagine we design a study to look at whether fish oil makes cat's hair shinier and it
has 80% statistical power.
That means we know that if there really is an effect of a certain type of fish oil and
if we ran the same experiment multiple times with different samples of cats, the data from
80% of the experiments will lead us to make the correct decision and reject the null hypothesis
that fish oil has no effect.
This is important because the whole reason that we do experiments is to see whether there's
an effect.
We don't just test whether fish oil makes cat's hair shiner just for fun, we want
shinier cats!
Statistical power tells us about our ability to detect these effects if they exist.
It would be a waste of time and money to run an experiment on whether people who play video
games have quicker reaction times than those who don't if we only have an estimated 20%
power, because that means that even if there gameplay effects reaction time, we often wouldn't
be able to tell.
Experiments cost money, so if you're going to go through the process of growing cells
in a petri dish, or of giving cats fish oil you want to be relatively confident you'll
be able to detect an effect if there is one.
Visually we see that statistical power is affected by how much the null and alternative
hypothesis distributions overlap.
The more they overlap, the less statistical power we'll have, because less of the alternative
distribution will be to the right of the cutoff.
There are two main ways to get the two distributions to overlap less.
Either you can move them further apart, or you can make them skinnier.
The distance between the mean of the two distributions represents something called "effect size".
If we're looking at the difference between two groups--like level of neuroticism between
cat people and dog people--effect size tells us how big the difference in neuroticism is
between the two groups.
If effect size is large, the groups are further away from each other, if it's small, they're
pretty close.
If two things are really different from each other, it's easier to tell them apart.
Say we're researching whether the amount of time people spend in the sun leads to more freckles.
If one group that spent 10 minutes in the sun led to an average of 5 new freckles over
the body, it'd be a lot harder to tell than if 10 minutes in the sun led to an average
of 500 new freckles.
Unfortunately, effect size is largely out of our control.
Researchers can't magically change the efficacy of a drug, or the difference in heart rate
between people who do kickboxing and people who do Crossfit.
We can also make our distributions overlap less by making them skinnier.
And remember, the null and alternative distributions are just sampling distributions.
We've seen that as you increase the size of your samples, the distribution of sample
means gets thinner.
And all other things being the same, they overlap less and we have more power to detect an effect.
This shrinking represents the fact that in general, the more data we have, the more information
we have.
Thankfully we can change sample size.
It might be a pain to sample more people, feed more cats more fish oil, or measure more
ocean temperatures, but at least it's within our control, unlike effect size.
And that's just what researchers do.
We already mentioned that if we're going to take the time to run an experiment or do
a study... we want to make sure it has sufficient power to detect any effects out there, and
since almost everything else is out of our control, scientists will increase their sample
size to get sufficient statistical power to detect these effects.
Across many fields it's considered sufficient to have 80% statistical power or more, and
often when researchers are designing studies, they'll decide how many subjects they need
based on estimates of effect size and power.
So now you're playing with power…and in the next few episodes we'll talk a lot more
about exactly when and how you can use p-values, and also some completely different methods
for testing ideas.
Thanks for watching, I'll see you next time.
For more infomation >> Volkswagen up! 1.0 Move Up! Bleumotion 5drs - Duration: 1:07. 
For more infomation >> Mazda 3 Sport 1.6 TOURING AIRCO 5DRS 100% MAZDA ONDERH.APK 6-2019 - Duration: 0:57. 
For more infomation >> Opel Astra 1.4TURBO 140PK ANN.EDITION 5DRS - Duration: 1:08. 
For more infomation >> I Like Me Better (Official Video)
For more infomation >> Fui trollada e chorei - (TROLLAGEM) - Duration: 6:58. 
For more infomation >> Seja Uma Pessoa PIOR - Duration: 4:01. 
For more infomation >> Tina Cipollari, rieccola con il suo Chicco. E rotondità sospette !! - Duration: 1:31. 
For more infomation >> Marco Ferri tronista di Uomini e Donne dopo L'Isola dei Famosi? - Duration: 4:46. 
For more infomation >> Valentina Dallari e l'anoressia: 'Sto ancora lottando' - Duration: 3:05. 
For more infomation >> COMO EU DESCOBRI QUE ESTAVA GRÁVIDA | KAROL CLORADO | LIBRAS E ATIVE LEGENDADO | - Duration: 7:27. 
For more infomation >> Porsche Panamera V6 E-Hybrid PDK Sport Turismo - Duration: 1:05. 
For more infomation >> Como elaborar um orçamento na Consultoria Ambiental? - Duration: 7:48. 
For more infomation >> BesouroBass - Hábitos Noturnos (PROD. Lerym) - Duration: 3:36. 
For more infomation >> Chiara Ferragni, l'addio al nubilato a Ibiza diventa un docu video - Duration: 2:30. 
For more infomation >> WP3348552 - Replacing Your Kenmore's Washer Drain Hose - AP6008028, PS11741157 - Duration: 14:35. 
For more infomation >> Skoda Fabia - Duration: 0:48. 
For more infomation >> Skoda Fabia - Duration: 1:07. 
For more infomation >> Skoda Fabia - Duration: 1:04. 
For more infomation >> Volvo XC40 T5 247PK AWD | AUTOMAAT | R-DESIGN | PILOT ASSIST | LUXE | - Duration: 1:14. 
For more infomation >> Volvo V90 D3 150pk Geartronic R-Design 19" Scandinavian Intro Line - Duration: 0:55. 
For more infomation >> Lawmakers are losing patience over Trump's tariffs - Duration: 6:45. 
For more infomation >> Big Spook「Celeste 🏔 Ep2」 - Duration: 1:32:37. 
For more infomation >> Volkswagen T-Roc 1.0 TSI STYLE - Duration: 0:55. 
No comments:
Post a Comment