2008 FAEIS Annual Longitudinal Assessment With a Comparison to the 2007 Survey Results The purpose of the FAEIS annual evaluation is to develop longitudinal and trend analyses in order to assess progress and develop program components for the future.

Objectives  Describe selected demographic and other variables that have potential association with FAEIS use.  Determine the level of usefulness of FAEIS data resources and applications.  Determine the level of satisfaction with selected components of FAEIS.  Determine levels of adoption of FAEIS and factors that explain the levels of adoption.  Determine the relationship of selected demographic and usage variables (involvement, use, experience, primary job title) with perceived usefulness and satisfaction with FAEIS.

Methodology  Panel of Experts - 7 FAEIS project staff, USDA representatives and chair of the FAEIS Panel  FAEIS Panel Review and Approval  Instrument Development  Population 1070 Users; 285 Respondents  Follow Up Reminders

2008 Participants  Staff  Faculty  Department Head  College Associate/Assistant Deans  College Dean or VP  University level administration  “Government” added with recoding

Question 1 in 2008 My Involvement with FAEIS Includes: (select as many as apply) (all possible combinations displayed; N=285)

Question 1 My Involvement with FAEIS Includes: (select as many as apply) (all possible combinations displayed; N=285) Involvement Type2007 Percent of Total 2008 Percent of Total Data Entry User of FAEIS Products89.1 Data Entry & Analytical77.7 Analytical & User of FAEIS products 36.0 Analytical 65.6 Data entry & User of FAEIS products 55.6 None32.8 Data entry, Analytical, & User of FAEIS products 42.5 No answer22.5 Analytical, Product developer, & User of FAEIS products Data entry & Product developer 31.1 Data entry, Analytical, Product developer, & User of FAEIS products 20.7 Data entry, Analytical & Product developer 20.4 Analytical & Product developer Product developer10.4 Product developer & User of FAEIS products

Question 1 Involvement with FAEIS of Survey Respondents (“none” and no answer responses eliminated) 2007 Percent of Total (N=308) 2008 Percent of Total (N=285) Data Entry8175 User of FAEIS Products 2127 Analytical2515 Product Developer 95 Analysis – in 2008 there were fewer data entry survey respondents, more users of FAEIS products, but fewer respondents indicated that they are analyzing and developing their own products, compared to 2007 survey respondents.

Question 2 How often have you used FAEIS in the past 12 months? (N=285) 2008

Question 2 How often have you used FAEIS in the past 12 months? 2007 Percent of Total (N=308) 2008 Percent of Total (N=285) Not at all times times More than 4 times No answer0.3-- Analysis – slight increase in the more frequent use categories (3-4 times & >4 times) in 2008

Question 3 My total experience using FAEIS is? (N=285) 2008

Question 3 My total experience using FAEIS is? 2007 Percent of Total (N=308) 2008 Percent of Total (N=285) More than 3 years to 3 years Less than 1 year No answer22.5 Analysis –greater number of “experienced” users (> 3 yrs) and greater number of “less experienced” users (<1 yr) in 2008 compared to 2007.

Question 4 My professional position is best described as? (N=285) 2008

Question 4 My professional position is best described as? 2007 Percent of Total (N=308) 2008 Percent of Total (N=285) Staff Faculty187.7 Department Head619.3 College Associate/Assistant Dean College Dean or VP57.7 University level Administration 52.5 Government--0.7 No answer1-- Analysis – More administrators (non-staff/faculty) participated in the survey in 2008 (44.9%) vs 2007 (32%).

Question 5 My level of adoption (use) of FAEIS is best described as? (N=285) 2008

Question 5. “My level of adoption (use) of FAEIS is best described as:” (no answer – eliminated from analysis) 2007 Percent of Total (N=290) 2008 Percent of Total (N=285) 1=unaware =aware of it but do not use it =Use it but not sure of long term use =Use it and it is integral to my job =Integral to my job and expect long term use =So important as to never be without FAEIS Mean Score

Question 5. “My level of adoption (use) of FAEIS is best described as:” (no answer – eliminated from analysis) 2007 Mean 2008 Mean Data Entry Users Only 2.49 (N=159) 2.30 (N=152) All Other Users3.16 (N=117) 3.24 (N=125) Total2.78 (N=276) 2.73 (N=277) 1=unaware 2=aware of it but do not use it 3=Use it but not sure of long term use 4=Use it and it is integral to my job 5=Integral to my job and expect long term use 6=So important as to never be without FAEIS

Question 6 Please rate the usefulness of the data resources in FAEIS (for student enrollment, degrees awarded and placement, disciplines, gender, ethnicity, faculty salaries, etc.). (1 = not useful, 5 = very useful) 2008

Question 6 Please rate the usefulness of the data resources in FAEIS (for student enrollment, degrees awarded and placement, disciplines, gender, ethnicity, faculty salaries, etc.). Score 2007 Percent of Total (N=192) 2008 Percent of Total (N=176) 1, Not useful , Very useful2421 Mean score (“no answer” and “not used” responses eliminated) Analysis – no significant changes between ‘07 & ‘08

Question 7 Please rate the usefulness of FAEIS at your institution for internal applications (institutional benchmarking, student and faculty recruitment, faculty hiring and fundraising). (1 = not useful, 5 = very useful) 2008

Question 7 Please rate the usefulness of FAEIS at your institution for internal applications (institutional benchmarking, student and faculty recruitment, faculty hiring and fundraising). (1 = not useful, 5 = very useful) Score 2007 Percent of Total (N=160) 2008 Percent of Total (N=158) 1, Not useful , Very useful1512 Mean score (“no answer” and “not used” responses eliminated) Analysis – slight increase in higher ratings in 2008 (totals of 4,5 in ’07 = 34%, ’08 = 38%). Lower percentage of “not used” answer in 2008 (’07 = 43%; ’08 = 36%).

Question 8 Describe your level of satisfaction with FAEIS components. (i.e., newsletter help desk, data entry, report builder, instructions) 1=not satisfied; 5=very satisfied 2008

Question 8 Describe your level of satisfaction with FAEIS components. (i.e., newsletter help desk, data entry, report builder, instructions) 1=not satisfied; 5=very satisfied Score 2007 Percent of Total (N=222) 2008 Percent of Total (N=208) 1, Not satisfied , Very satisfied2018 Mean score (“Unknown,” “no answer” and “not used” responses eliminated) Analysis – slight increase in higher ratings in 2008 (totals of 4,5 in ’07 = 52%, ’08 = 55%). Similar percentage of “not used” answer (’07 = 23%; ’08 = 23.7%).

2008 Annual FAEIS Evaluation Did the type of involvement (Q1) have a significant relationship with ratings of:  Frequency of use (Q2)  Experience with FAEIS (Q3)  Level of adoption (Q5)  Usefulness of data (Q6)  Usefulness of FAEIS (Q7)  Satisfaction with FAEIS components (Q8) Respondents listing “data entry” as their only level of involvement were compared to all other levels of involvement. “Not used” and “no answer” responses for Q6, Q7 and Q8 were removed from the analysis. There were significant differences for “Frequency of Use (Q2),” “Level of adoption” (Q5) and “Usefulness of data” (Q6), and Usefulness of FAEIS to your institution (Q7) and “Satisfaction with FAEIS components” (Q8) between the two groups. There was not a significant difference between the means of the two groups for Q3. Respondents listing “data entry” as their only level of involvement had significantly lower means for Q2, Q5, Q6, Q7 and Q8 than the other group.

2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with:  Adoption of FAEIS (Q5) Yes, there is a positive correlation between “Frequency of use” (Q2) and “Adoption of FAEIS” (Q5). As the frequency of use increases, so does the rating of the adoption of the FAEIS. F=45.0 (P<0.05) Key: How often have you used FAEIS in the past 12 months? 1=Not at all 2=1-2 times 3=3-4 times 4=more than 4 times

2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with:  Usefulness of data (Q6) Yes, there is a positive correlation between “Frequency of use” (Q2) and “Usefulness of data” (Q6). As the frequency of use increases, so does the rating of the usefulness of the data. F=58.2 (P<0.05) Key: How often have you used FAEIS in the past 12 months? 1=Not at all 2=1-2 times 3=3-4 times 4=more than 4 times

2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with:  Usefulness of FAEIS (Q7) Yes, there is a positive correlation between “Frequency of use” (Q2) and “Usefulness of FAEIS” (Q7). As the frequency of use increases, so does the rating of the usefulness of FAEIS. F=25.2; P<0.05 Key: How often have you used FAEIS in the past 12 months? 1=Not at all 2=1-2 times 3=3-4 times 4=more than 4 times

2008 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with:  Satisfaction with FAEIS components (Q8) Yes, there is a positive correlation between “Frequency of use” (Q2) and “Satisfaction with FAEIS Components” (Q8). As the frequency of use increases, so does the rating of the satisfaction with FAEIS components. F=26.1;P<0.05 Key: How often have you used FAEIS in the past 12 months? 1=Not at all 2=1-2 times 3=3-4 times 4=more than 4 times

2008 Annual FAEIS Evaluation Did the “years of experience with FAEIS” (Q3) have a significant relationship with ratings of:  Level of adoption (Q5) For Level of Adoption (Q5), the “Less than One Year of Experience with FAEIS” group was significantly lower than the other two “level of experience” groups. This seems to reason – the less experience with FAEIS, the less the level of adoption of FAEIS.

2008 Annual FAEIS Evaluation Did the “years of experience with FAEIS” (Q3) have a significant relationship with ratings of:  Usefulness of data (Q6)  Usefulness of FAEIS to your institution (Q7)  Satisfaction with FAEIS components (Q8) There was not a significant difference between the three groups for “Usefulness of data” (Q6), “Usefulness of FAEIS to their Institution” (Q7) and “Satisfaction of FAEIS Components” (Q8).

2008 Annual FAEIS Evaluation Did the professional position (Q4) have a significant relationship with ratings of :  Frequency of use (Q2)  Years of experience with FAEIS (Q3)  Level of adoption (Q5)  Usefulness of data (Q6)  Usefulness of FAEIS (Q7)  Satisfaction with FAEIS components (Q8) The only significant difference was with the “government” group for Frequency of use (Q2) and Level of adoption (Q5). This group had higher means than the other groups for both questions.

2008 Annual FAEIS Evaluation Did the level of adoption (Q5) have a significant relationship with ratings of:  Usefulness of data (Q6)  Usefulness of FAEIS (Q7)  Satisfaction with FAEIS components (Q8) Groups responding “unaware,” & “aware but do not use” to Q5 were compared to the other response groups for Q5 - “Use it, but not sure of long term use;” “Use it and it is integral to my job;” “Integral to my job and expect long term use;” & “So important as to never be without FAEIS.” There were significant differences for all three of the questions above between the two groups. The means of the “unaware,” & “aware but do not use” groups were always lower for each of the questions when compared to the other group.

2008 Annual FAEIS Evaluation Responses by non data entry users only2007 Mean 2008 Mean “My level of adoption (use) of FAEIS is best described as:” SCALE: 1=unaware 2=aware of it but do not use it 3=Use it but not sure of long term use 4=Use it and it is integral to my job 5=Integral to my job and expect long term use 6=So important as to never be without FAEIS

2008 Annual FAEIS Evaluation Responses by all users2007 Mean 2008 Mean Rate the usefulness of the data resources in FAEIS. Scale: 1=Not useful, 5=Very useful Rate the usefulness of FAEIS at your institution for internal applications. Scale: 1=Not useful, 5=Very useful Describe your level of satisfaction with FAEIS components. Scale: 1=Not satisfied, 5=Very satisfied