Test-Retest Reliability of The Family Paradigm Assessment Scale (F-PAS) Ariel Adams1 BJ Arnold2 Renee Smith2 Brittany Reed2 1Department of Hearing, Speech, and Language Sciences Gallaudet University 2Department of Speech-Language Pathology University of Central Arkansas

Resources transformed for goals. Control How do important things get done? Time How is it used? Family Members may act as one entity, in small groups, individually, according to their Space How are physical and personal space used? Affect How are caring & support expressed? Energy How much effort to get things done? Family-centered evidence-based practice (EBP) is an approach to intervention in which the clinician considers the research, her clinical experiences, and the family’s values and preferences while making clinical decisions. Speech-language pathology and audiology professionals can look to family science for models and tools to help families discuss their values and encourage family-centered EBP. … Meaning What do you value? family paradigm Material How are possessions viewed? Content How do you determined what is real?

There are four paradigms: Closed Random Open Synchronous Family Paradigm = family’s view of the world. Each family's behavior is guided by its paradigm(s). There are four paradigms: Closed Random Open Synchronous Family Paradigms describe the family’s view of the world – or how they prefer to use their resources to meet their goals. Paradigms guide the way families do everything. Each one represents different decision-making processes that a family might use. The four paradigms are closed, random, open, and synchronous.

Stability through continuity & tradition Family Paradigm = family’s view of the world. Each family's behavior is guided by its paradigm(s). There are four paradigms: Closed Random Open Synchronous Stability through continuity & tradition Families that are particularly closed-paradigmatic prefer stability through continuity and tradition. they stick to a schedule and do the things they have always done

Free exploration through intuition & innovation Family Paradigm = family’s view of the world. Each family's behavior is guided by its paradigm(s). There are four paradigms: Closed Random Open Synchronous Free exploration through intuition & innovation Families of the random paradigm prefer free exploration through intuition and innovation. act in spontaneous and individual ways to reach individual goals.

Adaptation through consensus Family Paradigm = family’s view of the world. Each family's behavior is guided by its paradigm(s). There are four paradigms: Closed Random Open Synchronous Adaptation through consensus Families of the open paradigm prefer adaptation through consensus. make their decisions as a group and change if the group sees fit.

Harmony through identification Family Paradigm = family’s view of the world. Each family's behavior is guided by its paradigm(s). There are four paradigms: Closed Random Open Synchronous Harmony through identification Families of the synchronous paradigm prefer harmony through identification. know and understand what they do and why without a need to communicate with each other because the “just know.”

Some families use multiple paradigms Family Paradigm = family’s view of the world. Each family's behavior is guided by its paradigm(s). There are four paradigms: Closed  Random  Open  Synchronous  Some families use multiple paradigms It is important to remember that families can have members that use different paradigms, or use a combination of paradigms as a family.

The Family Paradigm Assessment Scale (F-PAS) (Imig, 1998) Used to identify paradigms Paper and pencil instrument 10 multi-part questions using a 0-10 rating scale 1 question per resource 1 question to rank resources 1 question per goal 1 question to rank goals Current and Ideal The tool designed to identify paradigms is called the Family Paradigm Assessment Scale. It is a paper and pencil instrument used to help identify individual paradigms. It consists of 10 two part questions using a zero to ten rating scale. An answer of 0 indicates that the participant’s family would never use that strategy, and an answer of 10 indicates that the participant’s family uses that strategy the most. Each question calls for two answers, one for how the family currently operates and one for how they would ideally operate.

The Family Paradigm Assessment Scale (F-PAS) Cluster Scores Calculated from the F-PAS Range from 0-5 0 – family never uses this paradigm 5 – family uses this paradigm most often Define cluster scores Reporting on C&I overall – future analysis for rest

Research Aim To measure test-retest reliability of the F-PAS overall cluster scores. The aim of my research project was to measure the test-retest reliability of the Family Paradigm Assessment Scale.

Participants (N=51) Gender: Age: Race/ethnicity: Education level: 34 females, 17 males Age: Mean Age 30 years Range from 18-70+ Race/ethnicity: 2% American Indian 18% African American 81% Caucasian Education level: 2% High school graduate 39% Some college 31% College graduate 18% Some graduate school 10% Advanced degree Family roles: 26 spouses 11 parents 3 grandparents 20 offsprings 23 siblings Years in family: 53% entire life 29% five years or more 10% less than five years 8% less than one year

Procedures Recruitment: flyers, word of mouth Each participant completed the F-PAS twice, demographics, no major change, Data entered into Excel spreadsheet which calculated F-PAS cluster scores Test-Retest Reliability of the overall F-PAS cluster scores for each paradigm was calculated using weighted kappas (k) with 95% confidence intervals (CI) <0.00 poor 0.00-0.20 slight 0.21-0.40 fair 0.41-0.60 moderate 0.61-0.80 substantial 0.81-1.00 almost perfect (Byrt, 1996) Recruiting was really the hardest part: I sent out and posted flyers, I told my friends, told them to tell their friends, etc. Participants completed the F-PAS once and then again two weeks later. They also filled out a demographics form and reported any major change in the two week period. (death in a family or loss of job that might explain any major difference in results) The scores were then calculated and compared using weighted kappas with 95% confidence intervals A weighted kappa score was determined for each of the four paradigms. A kappa is a way of measuring reliability using a full credit system. The scores are either the same, or they are not. A weighted Kappa is a little bit more complicated in that it measures reliability using a “partial credit” system for having similar scores. So if they have a 5 on the initial test and a 4 on the retest, it would get a score of like, .8 whereas in a Kappa system it would get 0, because it did not match exactly.

Current Closed Paradigm Retest 5 4 3 2 1 Test 11 15 In this chart, the rows represent the first f-pas cluster scores (the test) And the columns represent the second f-pas cluster scores (the retest) The numbers in each box represent the number of participants with that score. (for instance, we have 3 in the 5-4 box – this means that three people had the current-closed paradigm score of 5 on the first f-pas and 4 on the first f-pas.) Numbers on the yellow line are considered to have perfect reliability. those near but outside the yellow boxes, have better reliability than those in the outside 5-0 or 0-5 boxes. (in the top right or bottom left corners) k = .50 95% CI (.30,.70) Moderate

Current Random Paradigm Retest 5 4 3 2 1 Test 18 16 k = .30 95% CI (.09,.52) Fair

Current Open Paradigm 21 10 6 8 Retest 5 4 3 2 1 Test Test 21 10 6 8 k = .26 95% CI (.03,.50) Fair

Current Synchronous Paradigm Retest 5 4 3 2 1 Test 16 k = .54 95% CI (.38,.69) Moderate

Ideal Closed Paradigm 9 7 11 8 Retest 5 4 3 2 1 Test Test 9 7 11 8 k = .51 95% CI (.33,.70) Moderate

Ideal Random Paradigm 16 6 8 Retest 5 4 3 2 1 Test Test 16 6 8 k = .27 95% CI (.05,.49) Fair

Ideal Open Paradigm 33 8 Retest 5 4 3 2 1 Test Test 33 8 k = .50 95% CI (.24,.71) Moderate

Ideal Synchronous Paradigm Retest 5 4 3 2 1 Test 14 9 k = .36 95% CI (.16,.56) Fair

Overall Findings Current-Closed Paradigm Current-Random Paradigm Moderate reliability Current-Random Paradigm Fair reliability Current-Open Paradigm Current-Synchronous Paradigm Ideal-Closed Paradigm Moderate reliability Ideal Random Paradigm Fair reliability Ideal- Open Paradigm Ideal- Synchronous Paradigm Overall test-retest reliability

Clinical Implications Test-retest reliability OK for clinical use Clinicians can use the F-PAS to: Determine paradigms of clients and their families Provide services that best fit client/family Improve family-centered practices Help families better understand how intervention affects their family functioning Fair to moderate test-retest reliability is not a clinical issue because the results can be validated through discussion. Can be used by clinicians to determine paradigms of the families they are working with. This will help them to provide services best fitting each client and therefore improve the overall quality of therapy It can also help families understand how intervention affects their family functioning.

Future Research Analyze reliability of individual cluster scores Need to evaluate ways to improve reliability Wording of questions Data collection method Use family paradigm model of resources and goals in researching family-centered practices Although the Family paradigm assessment scale can be used in clinic, Test-retest reliability should be improved for research purposes. We need to evaluate new ways to improve reliability. These could include the wording of the questions and the data collection method used – possibly making the F-PAS more “user friendly.” Once that is done, the F-PAS can be used in researching family-centered practices

What we learned… There are no short cuts Nothing is as easy as it appears to be Double check EVERYTHING Everybody needs a team Research is fun “Welcome to research” No short cuts – everything must be done the right way – and that usually means the long way. Nothing is easy – recruitment sounds easy Double check everything – proofread before you send things out (with a typo) data entry (much easier to fix now than find it later) THE LITTLE DETAIL MATTER Team – helped recruit – even moreso when I was in a bind, help with abstract (which I had never written before) peer-editing group, moral support system Research is fun – fun to learn, interesting to see how everything fits together. I’ve read many many research articles for classes and never really understood what they meant until I understood the research process. I learned so much information that I didn’t know before. And when all else fails, “welcome to research”

Acknowledgements Those who participated Faculty mentor: Dr. Mary Jo Cooley Hidecker F-PAS creator: Dr. David R. Imig Research team members: BJ Arnold, Morgan Poole, Brittany Reed, Megan Scott, Sara Shaw, Renée Smith, Tammy Soileau and Kara Taylor UCA Sponsored Programs Office (SPO) This research was funded by the Arkansas Department of Higher Education and the University of Central Arkansas Speech-Language Pathology Department

For more information: Byrt, T. (1996). How good is that agreement? Epidemiology, 7(5), 561. Constantine, L. L. (1986). Family paradigms: The practice of theory in family therapy. New York: Guilford. Constantine, L. L. (1993). The structure of family paradigms: An analytical model of family variation. Journal of Marital and Family Therapy, 19(1), 39-70. Hidecker, M. J. C. (2004). An exploratory study of the associative relationships between family paradigms and augmentative and alternative communication satisfaction in families with young Children. Unpublished Thesis (Ph. D.), Michigan State University. Dept. of Audiology and Speech Sciences, 2004. Hidecker, M. J. C., Jones, R. S., Imig, D. R., & Villarruel, F. A. (2009). Using family paradigms to improve evidence-based practice. American Journal of Speech Language Pathology, 18(3), 212-221. Imig, D. R. (1993). Family stress: Paradigms and perceptions. Family Science Review, 6, 125-136. Imig, D. R. (2000). A conversation about interpersonal relationships, family systems and paradigms. Venice, CA: ETEXT.net Electronic Textbook Publishing. Imig, D. R. (2005). Family paradigms, interpersonal relationships, and family systems. Venice, CA: ETEXT.net Electronic Textbook Publishing. Imig, D. R., Pate, S. M., Mitchell, M. M., David, D. A., Pegorraro, C., Barton, E. R., et al. (1996). Paradigmatic family systems theory: Applications and praxis. Paper presented at the 58th Annual Conference Education & Enrichment Section, Kansas City, MO. Imig, D. R., & Phillips, R. G. (1992). Family theory: The family regime assessment scale (FRAS). Family Science Review, 5, 217-234. Kantor, D., & Lehr, W. (1975). Inside the family (1st ed.). San Francisco: Jossey-Bass. Add

The Family Paradigm Assessment Scale (F-PAS) Cluster Scores Two cluster scores for each paradigm Current & Ideal overall Current & Ideal time Current & Ideal space Current & Ideal energy Current & Ideal material Current & Ideal control Current & Ideal affect Current & Ideal meaning Current & Ideal content Calculated from the F-PAS Range from 0-5 0 – family never uses this paradigm 5 – family uses this paradigm most often Define cluster scores Reporting on C&I overall – future analysis for rest