1 Cleaning Invalid Data

Clean data by using assignment statements in the DATA step. Clean data by using IF-THEN / ELSE statements in the DATA step. 2

Invalid Data to Clean The orion.nonsales data set contains invalid data that needs to be cleaned. After checking the data and finding the invalid data, the correct data values are needed. 3

Invalid Data to Clean VariableObsInvalid ValueCorrect Value Employee_ID Gender 12GF 101F Job_Title 10Security Guard I Country 2, 84, 87, 125, 197, and 200au or usAU or US Salary Hire_Date 521/01/195321/01/ /11/ /01/196801/01/1998 4

Programmatically Cleaning Data The DATA step can be used to programmatically clean the invalid data. Use the DATA step to clean the following observations: 5 VariableObsInvalid ValueCorrect Value Country 2, 84, 87, 125, 197, and 200au or usAU or US Salary Hire_Date 521/01/195321/01/ /11/ /01/196801/01/1998

The Assignment Statement The assignment statement evaluates an expression and assigns the resulting value to a variable. General form of the assignment statement: variable names an existing or new variable. expression is a sequence of operands and operators that form a set of instructions that produce a value. 6 variable = expression ;

The Assignment Statement Expression Operands are character constants numeric constants date constants character variables numeric variables. Operators are symbols that represent an arithmetic calculation SAS functions. 7

The Assignment Statement Expression 8 Country = upcase(Country); Salary = 26960; Gender = 'F'; Hire_Date = '21JAN1995'd; functionvariable numeric constant character constant date constant

SAS Functions A SAS function is a a routine that returns a value that is determined from specified arguments. The UPCASE function converts all letters in an argument to uppercase. General form of the UPCASE function: The argument specifies any SAS character expression. 9 UPCASE(argument)

10 %clearall libname orion "&path/prg1"; data work.clean; set orion.nonsales; Country=upcase(Country); run; proc freq data=clean; tables country; run;

The Assignment Statement All the values of Country in the data set orion.nonsales need to be uppercase. 11 data work.clean; set orion.nonsales; Country=upcase(Country); run; PDV Employee_IDJob_TitleCountry DirectorAU...

The Assignment Statement 12 data work.clean; set orion.nonsales; Country=upcase(Country); run; PDV Employee_IDJob_TitleCountry DirectorAU... upcase(AU)

The Assignment Statement 13 data work.clean; set orion.nonsales; Country=upcase(Country); run; PDV Employee_IDJob_TitleCountry Administration Managerau...

The Assignment Statement 14 data work.clean; set orion.nonsales; Country=upcase(Country); run; PDV Employee_IDJob_TitleCountry Administration ManagerAU... upcase(au)

The Assignment Statement The assignment statement executed for every observation regardless of whether the value needed to be uppercased or not. 15 %clearall proc print data=clean; var Employee_ID Job_Title Country; run;

Programmatically Cleaning Data 16 VariableObsInvalid ValueCorrect Value Country 2, 84, 87, 125, 197, and 200au or usAU or US Salary Hire_Date 521/01/195321/01/ /11/ /01/196801/01/1998 The assignment statement was applied to all observations. The assignment statement needs to be applied to specific observations.

Programmatically Cleaning Data The DATA step can be used to programmatically clean the invalid data. Use the DATA step to clean the following observations: 17 VariableObsInvalid ValueCorrect Value Country 2, 84, 87, 125, 197, and 200au or usAU or US Salary Hire_Date 521/01/195321/01/ /11/ /01/196801/01/1998

IF-THEN Statements The IF-THEN statement executes a SAS statement for observations that meet specific conditions. General form of the IF-THEN statement: expression is a sequence of operands and operators that form a set of instructions that define a condition for selecting observations. statement is any executable statement such as the assignment statement. 18 IF expression THEN statement ;

IF-THEN Statements 19 PDV Employee_IDSalaryJob_Title Secretary I... %clearall data work.clean; set orion.nonsales; if Employee_ID= then Salary=26960; if Employee_ID= then Salary=26500; if Employee_ID= then Salary=24015; run;

IF-THEN Statements All the values of Salary must be in the range of – data work.clean; set orion.nonsales; if Employee_ID= then Salary=26960; if Employee_ID= then Salary=26500; if Employee_ID= then Salary=24015; run; PDV Employee_IDSalaryJob_Title Secretary I... FALSE

IF-THEN Statements All the values of Salary must be in the range of – data work.clean; set orion.nonsales; if Employee_ID= then Salary=26960; if Employee_ID= then Salary=26500; if Employee_ID= then Salary=24015; run; PDV Employee_IDSalaryJob_Title Secretary I... FALSE

IF-THEN Statements All the values of Salary must be in the range of – data work.clean; set orion.nonsales; if Employee_ID= then Salary=26960; if Employee_ID= then Salary=26500; if Employee_ID= then Salary=24015; run; PDV Employee_IDSalaryJob_Title Secretary I... FALSE

IF-THEN Statements All the values of Salary must be in the range of – data work.clean; set orion.nonsales; if Employee_ID= then Salary=26960; if Employee_ID= then Salary=26500; if Employee_ID= then Salary=24015; run; PDV Employee_IDSalaryJob_Title Office Assistant II...

IF-THEN Statements All the values of Salary must be in the range of – data work.clean; set orion.nonsales; if Employee_ID= then Salary=26960; if Employee_ID= then Salary=26500; if Employee_ID= then Salary=24015; run; PDV Employee_IDSalaryJob_Title Office Assistant II... TRUE

IF-THEN Statements All the values of Salary must be in the range of – data work.clean; set orion.nonsales; if Employee_ID= then Salary=26960; if Employee_ID= then Salary=26500; if Employee_ID= then Salary=24015; run; PDV Employee_IDSalaryJob_Title Office Assistant II... FALSE

IF-THEN Statements All the values of Salary must be in the range of – data work.clean; set orion.nonsales; if Employee_ID= then Salary=26960; if Employee_ID= then Salary=26500; if Employee_ID= then Salary=24015; run; PDV Employee_IDSalaryJob_Title Office Assistant II... FALSE

IF-THEN Statements When an IF expression is TRUE in this IF-THEN statement series, there is no reason to check the remaining IF-THEN statements when checking Employee_ID. The word ELSE can be placed before the word IF, causing SAS to execute conditional statements until it encounters the first true statement. 27 data work.clean; set orion.nonsales; if Employee_ID= then Salary=26960; if Employee_ID= then Salary=26500; if Employee_ID= then Salary=24015; run; TRUE

IF-THEN / ELSE Statements All the values of Salary must be in the range of – data work.clean; set orion.nonsales; if Employee_ID= then Salary=26960; else if Employee_ID= then Salary=26500; else if Employee_ID= then Salary=24015; run; PDV Employee_IDSalaryJob_Title Office Assistant II...

IF-THEN / ELSE Statements All the values of Salary must be in the range of – data work.clean; set orion.nonsales; if Employee_ID= then Salary=26960; else if Employee_ID= then Salary=26500; else if Employee_ID= then Salary=24015; run; PDV Employee_IDSalaryJob_Title Office Assistant II... SKIP TRUE

Programmatically Cleaning Data The DATA step can be used to programmatically clean the invalid data. 30 VariableObsInvalid ValueCorrect Value Country 2, 84, 87, 125, 197, and 200au or usAU or US Salary Hire_Date 521/01/195321/01/ /11/ /01/196801/01/1998

IF-THEN / ELSE Statements All the values of Hire_Date must have a value of 01/01/1974 or later. 31 %clearall data work.clean; set orion.nonsales; Country=upcase(Country); if Employee_ID= then Salary=26960; else if Employee_ID= then Salary=26500; else if Employee_ID= then Salary=24015; else if Employee_ID= then Hire_Date='21JAN1995'd; else if Employee_ID= then Hire_Date='01NOV1978'd; else if Employee_ID= then Hire_Date='01JAN1998'd; run;