Annual Longitudinal Assessment The purpose of the FAEIS annual evaluation is to develop longitudinal and trend analyses in order to assess progress and develop program components for the future.

Objectives Describe selected demographic and other variables that have potential association with FAEIS use. Determine the level of usefulness of FAEIS data resources and applications. Determine the level of satisfaction with selected components of FAEIS. Determine levels of adoption of FAEIS and factors that explain the levels of adoption. Determine the relationship of selected demographic and usage variables (involvement, use, experience, primary job title) with perceived usefulness and satisfaction with FAEIS.

Participants Staff Faculty College associate dean College dean or VP University level administration “Department head” – added with recoding

Methodology Panel of Expert - 7 FAEIS project staff, USDA representatives and chair of the FAEIS Panel FAEIS Panel Review and Approval Instrument Development Population 960 Users; 308 Respondents 2 Follow Up Reminders

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation Question 5. “My level of adoption (use) of FAEIS is best described as:” (1=unaware; 2=aware of it but do not use it; 3=Use it but not sure of long term use; 4=Use it and it is integral to my job; 5=Integral to my job and expect long term use; 6=So important as to never be without FAEIS;) (7=No Answer – eliminated from analysis)

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation Mean=3.5

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation Mean=3.1

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation

2007 Annual FAEIS Evaluation Mean=3.5

2007 Annual FAEIS Evaluation Did the type of involvement (Q1) have a significant relationship with ratings of: Level of adoption (Q5) Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8) Respondents listing “data entry” as their only level of involvement were compared to all other levels of involvement. There were significant differences for “Level of adoption” (Q5) and “Usefulness of data” (Q6) between the two groups. Respondents listing “data entry” as their only level of involvement had lower means than the other group. There was not a significant difference between the two groups for “Usefulness of FAEIS” (Q7) and “Satisfaction with FAEIS components” (Q8).

2007 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Usefulness of data (Q6) Yes, there is a positive correlation between “Frequency of use” (Q2) and “Usefulness of data” (Q6). As the frequency of use increases, so does the rating of the usefulness of the data.

2007 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Usefulness of FAEIS (Q7) Yes, there is a positive correlation between “Frequency of use” (Q2) and “Usefulness of FAEIS” (Q7). As the frequency of use increases, so does the rating of the usefulness of FAEIS.

2007 Annual FAEIS Evaluation Did “Frequency of Use” (Q2) have a significant relationship with: Satisfaction with FAEIS components (Q8) Yes, there is a positive correlation between “Frequency of use” (Q2) and “Satisfaction with FAEIS Components” (Q8). As the frequency of use increases, so does the rating of the satisfaction with FAEIS components.

2007 Annual FAEIS Evaluation Did the “years of experience with FAEIS” (Q3) have a significant relationship with ratings of: Level of adoption (Q5) Usefulness of data (Q6) For Level of Adoption (Q5), the “Less than One Year of Experience with FAEIS” group was significantly lower than the other two “level of experience” groups. This seems to reason – the less experience with FAEIS, the less the level of adoption of FAEIS. There was not a significant difference between the three groups for “Usefulness of data” (Q6).

2007 Annual FAEIS Evaluation Did the “years of experience with FAEIS” (Q3) have a significant relationship with ratings of: Usefulness of FAEIS to your institution (Q7) Satisfaction with FAEIS components (Q8) For “Usefulness of FAEIS to their Institution” (Q7) the “Less than one Year of Experience with FAEIS” group had a significantly higher mean than the other two groups. In other words, the least experienced FAEIS users rated “Usefulness of FAEIS” higher than more experienced users. For “Satisfaction of FAEIS Components” (Q8) the “Less than one Year of Experience with FAEIS” group again had a significantly higher mean than the other two groups. Again, the least experienced FAEIS users rated “Satisfaction of FAEIS Components” higher than more experienced users.

2007 Annual FAEIS Evaluation Did the professional position (Q4) have a significant relationship with ratings of : Frequency of use (Q2) Years of experience with FAEIS (Q3) Level of adoption (Q5) Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8) No, there were no significant differences.

2007 Annual FAEIS Evaluation Did the level of adoption (Q5) have a significant relationship with ratings of: Usefulness of data (Q6) Usefulness of FAEIS (Q7) Satisfaction with FAEIS components (Q8) Groups responding “unaware,” & “aware but do not use” to Q5 were compared to the other response groups for Q5 - “Use it, but not sure of long term use;” “Use it and it is integral to my job;” “Integral to my job and expect long term use;” & “So important as to never be without FAEIS.” There were significant differences for all three of the questions above between the two groups. The means of the “unaware,” & “aware but do not use” groups were always lower for each of the questions when compared to the other group.

Comments from Respondents Used internally for planning Categories established to represent types of responses

Key Questions Is a longitudinal study appropriate? Are the objectives appropriate? Is the methodology appropriate? How can the results best be presented?