What is Hypothesis Testing and Why it is used - simplified (Part 1)

Hypothesis Testing is an interesting topic for the interviewers to decide whether the interviewee is good at statistics. Unfortunately, This is the topic on where Most of us fumble upon.

In this blog, I will try to explain what and why exactly the Hypothesis Testing is needed in layman way. Let’s get into it.

 

Whenever I listen the word Hypothesis Testing, The following quote comes into my mind.                      

“INNOCENT UNTIL PROVEN GUILTY”

 

Let’s understand a practical scenario, what will happen in a court room if a person is guilty of a crime?

1. Firstly, The Judge always presumes that the person is Innocent.

2. Secondly, The Lawyer tries to prove that the person is guilty based on the Evidence. 

 

Here, the lawyer is indirectly trying to reject the Judge’s presumption based on the collected evidence. This is what exactly happens in Hypothesis Testing.

There will be a fact and A claim is made against the fact. We need to try to prove that the fact is no more valid so that the claim will become true. 

 

Simple right.

Now if you got some understanding of the above scenario, you almost understood the Hypothesis Testing.

 

Hypothesis – A claim that we want to investigate or test.

Hypothesis Testing is a technique which helps us to get rid of certain element of randomness in a given claim based on the data. 

The Hypothesis Testing is used to know about the population parameters based on sample data taken from it.

Hypothesis Testing is taken from the Inferential Statistics to determine with how much probability the given Hypothesis is True. It evaluates two mutually exclusive statements about the population to find which statement is best suited based on sample data.

 

Real Time Example:

Scenario: Drug A heals the pain in few hours, A Scientist came and claimed that the new Drug B heals the pain much faster than the Drug A.

Claim: Drug B takes less time to heal the pain than Drug A.

 

In this scenario, we can’t give the Drug B to millions of people directly without any testing. A clinical trial is to be done.

 

Case A:

Let’s say there are 100 volunteers, 50 people took Drug A and 50 people took Drug B.

Result: Drug A took 3hrs to heal and Drug B took 1hr to heal.

Now can we decide the claim that Drug B is better than Drug A is true?   Obviously, No. Because we tested on only 100 people out of Billion people. So the result may be a random luck.

 

Case B:

Let’s say there are 1000 volunteers, 500 people took Drug A and 500 people took Drug B.

Result: Drug A took 3hrs to heal and Drug B took 1hr to heal.

Now can we decide the claim that Drug B is better than Drug A is true?   Obviously, Again No. Because there might not be any diversity in the two groups. All people that took Drug B may be younger with good immune power than the people who took Drug A. so again the result may be a random luck.

  

So, Here It is hard to come to the conclusion about the claim. There comes Hypothesis testing which helps us to figure out the observed result is not due to random chance but there is a reason behind it.

Note: Since collecting the data from whole population is impossible, sample data is taken and try to decide whether the claim is true or not.

In the next blog I will explain what are Null Hypothesis and Alternate Hypothesis and how to identify them. please find the below link for part 2. 

Part 2: How to decide Null and Alternate Hypothesis? - simplified

Part 3: How to Conduct Hypothesis Testing step by step? - Easy and Elegant

If there are any queries, please specify in the comments below...

For More Content please follow...