Have fun guys. We're done!

Significance Test of Regression parameter (ANOVA Test)

Significance Test of Regression parameter (ANOVA Test)

05/05/2021
Significance Test of Regression parameter (one way ANOVA Test)
Edubomb Edubomb

SIGNIFICANCE TEST OF REGRESSION PARAMETER

In the significance test, of the regression coefficient, we test whether the given regression coefficient is significant or not. In another word, these tests are performed to know the relation between the dependent and independent variables. There are different tests for regression coefficients which are discussed below.

ANALYSIS OF THE VARIANCE (ANOVA) TEST

ANOVA test is performed to test the linearity of the regression equation as well as the goodness of fit of parameter or overall regression equation. It means this test is performed to test the relation between the dependent and independent variables. It shows whether it is different between the observed or calculated value of a parameter or not also. Hence it measures the goodness of fit of regression parameter or regression line.

To test, we use the F ration test.  F-test is the same as the Z test and T-test but in the F test, it can test significance for large numbers of variables I.e. more than two variables. In this case, we are dealing with two variables I.e. Y and X hence we use only one-way ANOVA.

We use the following procedure to test the significance of the regression line using the ANOVA test.

Null hypothesis (H­0): β = 0

There is no relationship between the dependent and independent variables Y and X. or the test is not significant

Alternative Hypothesis (H1): β ≠ 0 (two-tailed test)

There is a linear relationship between the dependent and independent variables.  Or test is significant

TEST STAT

F statistics test for simple regression equation Y = α + βX is given by

 

   

Where explained variance

 

where K is the number of variables in the regression. In this case two variables.

And Unexplained variance

 

where K is the number of variables in regression and n is the sample size or numbers of observations.

 

CONSTRUCTION OF ANOVA TABLE 

ANOVA table for the test of significance of the regression line is given below:

Source of variation sum of squares Degree of Freedom Mean sum of squares F-statistics
Regression Unexplained K-1
Error or Residual  Explained n-k
Total SST n-1    

 

DECISION 
If the tabulated value of F-ration at numerator degree of freedom K-1 and denominator degree of freedom n-k is greater than the calculated value then H0 is accepted otherwise H0 is rejected it means an alternative Hypothesis is accepted.

 

EXAMPLE:

1.Consider the following data for the supply and price of a commodity for the last seven years.

year 1980 1981 1982 1983 1984 1985 1986
supply 80 84 86 88 92 96 97
Price 12 11 15 15 18 16 18

Estimate the most likely price in 1981 when the supply is 110. Also, calculate the explained and total variation also interpret them using ANOVA test. 

Answer:

Let us consider X denotes the supply and Y denotes the price then we have

X Y x=X-x̅ y=Y-y̅ xy x2 y2
80 12 -9 -3 27 81 9
84 11 -5 -4 20 25 16
86 15 -3 0 0 9 0
88 15 -1 0 0 1 0
92 18 3 3 9 9 9
96 16 7 1 7 49 1
97 16 7 3 24 64 9
∑X=623 ∑Y=105     ∑ xy=87 ∑x2=238 ∑y2=44

Now,

Mean of X i.e. average of supply

 

Mean of Y i.e. average of price

 

Let us consider the regression line

   

 Where Y and X are dependent variables and independent variables and a and b are the regression parameter.

Now first of all we try to find the value of a and b which are regression parameters.

Equation (1) can be written as the

 

Subtracting equation (2) from (1) we get

 

 

 

Multiplying both sides by x and taking summation both side  we get

 

 

b=0.365

From equation 2 we get

   

 

a= -17.48

Now when supplying i.e. X=110, price Y =?

 

 

Ŷ =22.67 (estimated price)

Sum of a square of the regression

 

 

 

 

 

Sum of the square of the total

SST=SSE+SSR

Sum of the square of the error

SSE=SST-SSR=44-12.20=31.80

Now, 

Source of variation sum of squares Degree of Freedom Mean sum of squares F-statistics
Regression =12.20 K-1=2-1=1

12.20/1=12.20

=6.36/12.20

=0.52

Error or Residual  =31.80 n-k=7-2=5

=31.80/5=6.36

Total SST=44 n-1=6  

 

Null hypothesis (H­0): β = 0

There is no relationship between the dependent and independent variables supply and price X. or the test is not significant

Alternative Hypothesis (H1): β ≠ 0 (two-tailed test)

There is a linear relationship between the dependent and independent variables i.e. between supply X and price Y. Or test is significant

TEST STAT

F statistics test for simple regression equation Y = α + βX is given by

 

   

DECISION

the tabulated value of F-ration at numerator degree of freedom K-1=1 and denominator degree of freedom n-k =7-1=5 is 6.61 which greater than the calculated value 0.52 hence H0 is accepted i.e. test is significant.

F-table

Note: this table shows F value at a 5% level of significance only. FOr another level of significance use another table. 

HOW TO USE TABLE

1. first of all find the value of the denominator degree of freedom. Here we have n-k =7-1=5 where n is the total number of observations (count on the given question) and k is a number of variables. here we have two variables X and Y hence K=2.

2. Now, find the value of the numerator degree of freedom. Here we have K-1=2-1=1

3. we have numerator 1 and denominator 5 so from the table, the value is 6.61. 

EXERCISE:

1. what is F-test? describe ANOVA test for regression Parameter.

2. How F-test is different From the T-test and Z-test?

3. What is the Significance test of regression Parameter?

4. What is the difference between one-way ANOVA and Two way ANOVA?

5. what are SSR and SSE? write down the formula for each.

6. What is the degree of Freedom?

7. if there are only two variable X and Y, does it means K=2?

8. How do you derive Y-estimated from Y = α + βX?

9. What is Error or residual?

10. linearity and goodness of fit are related to the ANOVA test?

11. Does ANOVA mean analysis of variance?

12. what is a variance?

 

Want To Study More Topics?




LEAVE A COMMENT

Facebook Conversations