Showing posts with label regression. Show all posts
Showing posts with label regression. Show all posts
Sunday, December 22, 2013
Content of this tutorial blog
2.9 Multipe or simple linear regression analysis with categorical variables
Multipe or simple linear regression analysis with categorical variables
In the previous two sections the regression analysis is performed with numeric variables, variables with numerical values. It might appear that a dataset contains not-numerical values or variables. These are called categorical variables because these contain categories or characters instead of numbers.
The example dataset
In the last section of this tutorial the file Projects2.csv is used in the examples. By opening the file you can see that it contains the same data like the file Projects.csv. However, in this file three new categorical variables with character values are added: WorkingGroup (A,B or C), Month (januari t/m december) and TypeProject (Alfa, Beta, Gamma, Delta). The data is imported and converted to a variable and matrix with the commands: Projects2<-read.csv('Projects2.csv') and attach(Projects2).
 |
| Figure 26: Importing the new dataset into the R console |
The command
The command that is used to perform a regression analysis with categorical variables is quietly the same as a regression analysis with numerical variables. The difference is that categorical variables have to be converted by a command into numerical variables. This could be done with the command factor(*name of the categorical variable*). This piece of command has to be applied in the regression analysis. To make this more clear, an example is being used.
Simple linear regression in with categorical variables in R
In the example in Figure 27 the dependent variable Profit (Winst) is being predicted by the independent variable WorkingGroup. Because WorkingGroup is a categorical variable and it has to be calculated as a numerical variable in the regression analysis, the categorical variable has to be converted to a numerical variable. In the example this is done by the command: Regression<-lm(Profit~factor(WorkingGroup),Projects2). By typing in the command Regression the prediction model is presented: Profit =1479.1 + (0A) + (1987 B) + (4097 C). In this case only the variable of the specific working group is included in the model. For example if the profit has to be estimated if working group C executes the project, the following calculation has to be performed: 1479.1 + 1987 = 3466.1.
By typing in the command summary(Regression) the regression analysis is presented. You can see that the command factor() is the command that is placed in the regression. This could be seen as a little addition to the categorical variable.
 |
| Figure 27: Multiple linear regression with categorical variables in R |
Multiple linear linear regression with categorical variables in R
To execute a regression analysis with multiple categorical variables you use the following command: *name of regression*<-lm(*dependent variable*~factor(*independent variable 1*) + as.numeric(*independent variable 2*) + factor(*independent variable.....etc*), *variable of the dataset*). In the example of Figure 28 you see that the dependent variable Profit (Winst) is predicted by the independent variables WorkingGroup and TypeProject. For that the following command is used:
Regression<-lm(Profit~factor(WorkingGroup)+as.numeric(ProjectType), Projects2).
By typing in the command Regression, the prediction model appears: Profit = 2706 + (0 A) + (1123 B) + (3274 C) + (-1121 Beta) + (-2563 Delta) + (-140 Gamma). Only the categorical variables that are considered should be included when calculating the formula. For example, if the profit of working group C is going to execute project type Gamma, the predicted value is 2706 + 3274 -140 = 5840.
By entering the code summary(Regression) the regression analysis is presented. By looking at the rows in the analysis you can see which variables are not significant. In the example of Figure 28 shows the the row with the factor Gamma is not significant. This means that predictions that are done with the independent variable Gamma are not quietly reliable.
The Adjusted R-sqaure of 0.5759 shows that the quality of the prediction model is quite high.
 |
| Figure 28: Interpreting the multiple regression analysis with categorical variables in R |
- End of Part 2 -
2.8 Multiple linear regression in R
Multiple linear regression in R
In the previous section it is explained how a simple linear regression could be executed with R, where a dependent variable is predicted by only one independent variable. In this section it is explained how to execute a multiple linear regression analysis in R. In this case the dependent variable is explained with more than one independent variable.
The command to perform a multiple linear regression with R
In R, there is a slight difference between the command of a simple regression analysis and a multiple regression analysis. Executing a multiple linear regression could be executed in R with the following command: (*name of the regression*)<-lm(*dependent variable*~*independent variable 1* + *independent variable 2* + *independent variable.... etc.*, *variable of the dataset*). You could add as much independent variables you want to predict the dependent variable, as long as you use a + symbol when adding these variables.
In the same way as the simple linear regression, you can present the predicted model with the information of the regression analysis respectively with the commands *naam of the regression* and summary(*name of the regression*).
 |
| Figure 25: Executing a multiple linear regression with R |
Intepretation of the regression
In contrast to a simple linear regression, the results of a multiple linear regression show more rows in the information section of the overview of the regression. The principle stays the same, based on the indications of stars after the specific row the levels of significance of the variables are shown.
Figure 25 shows the result of the following regression analysis. Predicting the profit based on information about the costs of labour, material and satisfaction of the customer use the following command:
Regressing<-lm(Profit~PersonnelCosts+MaterialCosts+SatisfactionCustomer,Projects). By putting in the name of the Regression, the following prediction model appears:
Profit = 8845.8805 + (-0.2964)X1 + (-1.9292)X2 + (-1338.5787)X3. Where X1 stands for personnel costs, X2 for material costs and X3 for the level of customer satisfaction.
With the command summary(Regression) the regression analysis appears. Notice that all rows of variables are significant. The variables vary in level of significance, however all the variables are significant enough to predict valuable outcomes. In the example of Figure 25 the Adjusted R-squared is only 0.2529. The three independent variables are quite acceptable variables to make predictions of the profit, one out of four predictions (25%) from the model are right.
In this section you could see that a multiple linear regression works in the same way as a simple linear regression.
To the next step: 2.9 Multiple or simple linear regression analysis with categorical variables in R
To the next step: 2.9 Multiple or simple linear regression analysis with categorical variables in R
2.7 Performing a simple linear regression analysis with R
2.7 Performing a simple linear regression analysis with R
With a regression analysis you can, just like with a correlation, find the cohesion between different variables. With a regression analysis you get a formula with which you can predict future outcomes with some degree of reliability. With a regression analysis the interpretation is important.
Variables in a regression analysis
In this section the simple linear regression analysis is explained. This means that a variable gets predicted with the results of other variables. The predicted variable is called the dependent variable. The variables with which the dependent variable is predicted is called the independent variable.
Command
The command you have to use in R to perform a regression analysis is:
lm(*name of the dependent variable*~*independent variable*, *name of variable of the dataset*). You could use every numeric variable for a regression analysis. Variables vary in prediction power when serving as independent variable.
To be able to interpret the regression analysis, you have to give a name to the regression. You can do this by using the code <-. If you want to execute a regression analysis and want to interpret this one later, you use the command *name of the regression*<-lm(*name of the dependent variable*~*independent variable*,*name of variable of the dataset*).
In the example of Figure 23 the following command is used:
Regression1<-lm(SatisfactionCustomer~DistanceCustomer,Projects).
The estimated model
In the example of Figure 23 the satisfaction of the customer is estimated by looking at the distance between the company and the customer. R shows the following formula based on the regression analysis: SatisfactionCustomer = 2.524811 + (-0.003408x), where x stands for the number of kilometers. This formula is called the estimated model. By only putting in the name of the regression, the estimated model appears.
 |
| Figure 23: Performing a regression analysis with R |
Executing the regression analysis
With the command summary(*name of the regression*) you get an overview of the regression. In this overview you have to look at the part underneath the text Coefficients. After the rows (Intercept) and DistanceCustomer (in this case look at AfstandKlant) the same values as in the estimated model.
In the example of Figure 23 the same values are shown as in the results of the regression analysis.
Significance
After the rows 3 stars (*) are presented. The stars indicate the level of significance. This indicates the importance and strength of the independent variable in predicting the dependent variable. An indication with one star is enough to conclude significance. The more stars after a row, the more significance the variables have. If there are now stars after a row, the conclusion is that the specific independent variable is not significant in predicting the dependent variable. This indicates that the variable has not a lot of prediction power in this regression model, and could be removed from the regression analysis.
Prediction power of the regression analysis / Adjusted R-squared
In the overview of the regression analysis you find the text Adjusted R-squared followed by a value. This is the percentage presented in decimals of the results/outcomes that are correctly be predicted by the current regression model. An Adjusted R-squared of 1 means that 100% of the outcomes can be predicted by the current regression model. An Adjusted R-squared of 0,5 means that 50% of the predictions are correctly predicted by the current regression model.
It is up to the user to judge, based on the Adjusted R-squared value, if the regression model is good enough to give predictions. Generally, a Adjusted R-squared value of 0,5 (or higher) is quite an acceptable regression model to make predictions for.
In the example given in Figure 23 the Adjusted R-squared is 0.1041. This means that in 10,41% of the cases the prediction/regression model could predict the satisfaction of the customer correctly. Because this is quite a low value, the prediction model SatisfactionCustomer = 2.524811 + (-0.003409x) could be judged as a low level prediction model.
New regression analysis
Figure 24 shows that another regression analysis is executed. In this case there is searched for a prediction model that could predict the turnover of a project by looking at the hours spent on a specific project. In this case the following command is used: Regression2<-lm(TurnoverProject~HoursProject,Projects).
By typing in the command Regression2, the following prediction model appears. Turnoverproject = 0 + 40x. In this case x is the number of hours that is spent in the project.
With the command summary(Regression2) the following information about the regression analysis appears (see Figure 24). This shows that only the variable HoursProject (UrenProject) is significant (the intercept could be left away since this has a value of 0).
The information also shows that the value of the Adjusted R-squared is 1. This means that the prediction model is 100% reliable. This is quite logical since the company asks $ 40,- per hour. The turnover is simply the sum of hours and fee per hour.
 |
| Figure 24: Another regression analysis in R |
Subscribe to:
Posts (Atom)