1 Using Factor Analysis to Clarify Operational Constructs for Measuring Mission Perception Ellen M. Boylan, Ph.D. NEAIR 32 nd Annual Conference November 7, 2005

2 Purpose  Develop an instrument to measure mission perception  Test the instrument’s reliability  Uncover constructs  Observe relationships among constructs

3 Research Questions Research Questions 1. Are the Research Mission Questions valid and reliable measures of student perception of institutional mission? 2. Are there factors present in the Research Mission Questions? 3. If there are factors, what is the relationship among them?

4 Study Design Study Design  Select the National Survey of Student Engagement (NSSE) 2004 instrument (The College Student Report) to test first year and senior students  Assemble an NSSE 2004 Catholic College Consortium for mission research  Develop Research Mission Questions  Administer the NSSE 2004 with added Research Mission Questions to Consortium students  Test the Research Mission Questions: reliability, factor analysis, and correlation analysis

5 Developing the instrument  Conduct a qualitative analysis of mission statements  Distill common constructs  Draft question items based on mission statement constructs  Vet with participating institutions, experts, and prospective subjects

6 Sample Mission Statement Concepts common to the Consortium Colleges’ mission statements were drawn out, reduced, and formulated into 20 Research Mission Questions leaders in service professional disciplines spiritual, ethical, and religious values learning diverse and interdependent world. A Catholic university sponsored by the Congregation of the Sisters, Servants of the Immaculate Heart of Mary, Marywood University roots itself in the principle of justice and a belief that education empowers people. Enacting its ideals, Marywood offers students a welcoming and supportive community that encourages men and women of all backgrounds to shape their lives as leaders in service to others. Proud of its liberal arts tradition and host of professional disciplines, Marywood challenges students to broaden their understanding of global issues and to make decisions based on spiritual, ethical, and religious values. Marywood calls upon students to seek their full potential and invites all to engage in a lifelong process of learning. Witnessing the efficacy of teaching and scholarship, Marywood educates students to live responsibly in a diverse and interdependent world.

7

8 Administration The NSSE Survey with attached Research Mission Questions was administered to 3,827 first-year and senior students at 14 Catholic Colleges in the United States in Spring 2004.

9 NSSE Consortium and Data Release Authorization I authorize The National Survey of Student Engagement (NSSE) to release my institution’s NSSE data, for the year 2004 in which data were collected for my institution, to the NSSE 2004 Catholic College Consortium administrator (Ellen Boylan). Data at an individual respondent level, without student identifiers, will be released. NSSE may identify my institution by name in data distributed to the Catholic Colleges Consortium administrator. I understand that the Catholic College Consortium administrator will not make public any results identified by institution other than for their own use. I understand that the Catholic College NSSE data-sharing administrator will maintain the confidentiality of these data, and will use them for analyses and reports in accord with the conditions indicated above and only for analyses performed under the auspices of the Catholic College NSSE Consortium data-sharing project. Institution: _______________________________________________ Your Name: ______________________________________________ Your Title: ________________________________Date: __________ Signature: _______________________________________________ Yes, I would like to receive a customized report for my institution!

10 Characteristics of Institutions College by level and percent male/female

11 Characteristics of Institutions

12 Research Question 1 Are the Research Mission Questions valid and reliable measures of student perception of institutional mission?

13 Research Question 2 Are there factors present in the Research Mission Questions?

14 Rotated Factor Matrix 1 Sense of Mission 2 Respect for Diversity 3 Individual Actions 4 Religious Practice Social and personal development of students is an important part of the mission at this institution Ethical and spiritual development of students is an important part of the mission at this institution This institution offers opportunities for: Volunteering and community service Developing leadership skills At this institution, there are opportunities for students to strengthen their religious commitment The heritage of the founding religious community… is evident here The mission of this institution is widely understood by students The professors at this institution discuss the ethical implications of what is being studied The faculty, staff, and students here are respectful of people of different religions The faculty, staff, and students here are respectful of people of different races and cultures Students feel free to express their individual spirituality here People of different sexual orientations are accepted socially here The environment here encourages students to develop an appreciation of diversity How important is it to you that you accomplish the following: Volunteering in community service Influencing social values Within the past week… participated in a religious service? Within the past week…spent time in private prayer or meditation?

15 Reliability and factor analysis  Reliability analysis of Research Mission Questions (20 items). Three items removed.  Mission Perception Inventory (MPI) is produced (17 items).  Factor analysis of MPI conducted.  Four subscales produced: Sense of Mission Respect for Diversity Individual Actions Religious Practice

16 Mission Perception Inventory (MPI) (17 items) Sense of Mission (8 items) Respect for Diversity (5 items) Individual Actions (2 items) Religious Practice (2 items) Research Mission Questions (20 items) Derivation of Mission Perception Inventory (MPI) and four subscales from Research Mission Questions

17 Sense of Mission Scale (8 items  =.867)  The mission of this institution is widely understood by students  Social and personal development is an important part of the mission  Ethical and spiritual development of students is important  This institution offers opportunities for volunteering and community service  This institution offers opportunities for developing leadership skills  There are opportunities for students to strengthen their religious commitment  This institution’s religious heritage is evident  Professors here discuss the ethical implications of what is being studied

18 Respect for Diversity Scale (5 items  =.854) The faculty, staff, and students here…  Respect different religions  Respect different races and cultures  Students free to express individual spirituality  Different sexual orientations are accepted  Environment encourages appreciation of diversity

19 Research Question 3 If there are factors, what is the relationship among them?

20 Reliability and Correlation of Subscales **Correlation is significant at the 0.01 level (2-tailed). N=3,827.

21 Findings and Conclusions  Reliability analysis of the Research Mission Question instrument indicates it is a reliable and valid measure of student perception of institutional mission  A stronger instrument is defined when lower- correlated items are removed from the Research Mission Questions to make the Mission Perception Inventory (MPI)  Four reliable subscales, Sense of Mission, Respect for Diversity, Individual Actions, and Religious Practice, were produced from the MPI

22 Findings and Conclusions  Reliability of the Mission Perception Inventory indicates it can be employed with confidence as a measure of student mission perception in subsequent studies.  The four subscales produced by analysis indicate the presence of those concepts in students’ educational experiences at Consortium colleges.  The MPI subscales, Sense of Mission and Respect for Diversity, can be used alone or in concert with other instruments to measure mission perception among college students.

23 Conclusion F actor analysis and analysis of reliability indicate the Mission Perception Inventory (MPI) has extremely high reliability and has four correlated, independent and strong factors. The results of this research are replicable.

24