TECHNICAL APPENDIX METHODOLOGY SUMMARY FOR CPM RESEARCH Missions International PO Box – Franklin, TN –

© 2004 Missions International Human Resourcing Design of CPM assessment instrument and analysis support Hired Statistician –Ph.D. from Vanderbilt in Psychology –VP of Methodology for a Market Research Firm –Over 15 years corporate research experience with Fortune 500 companies Previous analytic work for MI on (SG) 2

© 2004 Missions International OBJECTIVES Discover elements (DNA) that drive conversion and planting growth –Measure conversions and planting growth –Measure comprehensive set of potential “drivers” (DNA) –Build predictive models to uncover which elements from comprehensive set are most “important” in driving conversions/planting –Substantive CPM interpretations across predictive models

© 2004 Missions International Assessment Design Qualitative content inputs from key expert leaders in Church Planting Movement - Watson, Sergeant, Sanchez, etc. What are “DNA” elements of church planting movement? Developed initial battery of items to capture that domain Subjected to review by experts Iterated to list of 31 items designed to capture domain broadly All elements measured on five-point frequency- of-occurrence scale: “Very Often” to “Almost Never”

© 2004 Missions International Other Variables Measured Also included several descriptive / demographic measures –age, gender, education, size of community, leader/member, formal training, how long ago church started, how many members attend, etc. Also included “Dependent” Variables –Self-report on frequency of starting new churches On same “Very Often” to “Almost Never” scale –Raw numeric dependent variables: How many converted through your church? How many converted through your church have been baptized? How many churches branched off from your church?

© 2004 Missions International Samples Sample sizes by contributing countries: – (n=542) – (n=507) – (n=858) – (n=61) One sample showed extremely limited variability (interpreted as most elements from expert inputs fully in operation in this movement, thus “topping out” the scale – i.e., all aspects tend to be happening very often in this movement). Analysis conducted on other countries where more variability existed - pooling of other nations (starting n=1110)

© 2004 Missions International More About Dependent Variables Self-report of Church planting activity –“Our church starts new churches…” –“Very Often” –“Often” –“Sometimes” –“Rarely” –“Almost Never” Coded as quasi-interval 1-5 ratings

© 2004 Missions International Numeric Dependent Variables Number of conversions, baptisms, new churches started All influenced by (a) length of existence –24 conversions from 2 year old church, vs. 24 conversions from 2 month old church –1 conversion per month vs. 12 per month All influenced by (b) number of members –50 conversions from church of 10, vs. 50 conversions from church of 25 –5 conversion per member vs. 2 per member Must control for length of existence and number of members to make raw numbers sensible and comparable

© 2004 Missions International “Per Member Per Month”- pmpm Implemented per member per month approach 100 conversions from 10 member church in existence for 1 month = 10 conversions per member per month 200 conversions from 20 members in existence 5 months = 2 conversion per member per month Raw numbers would have made 200 look “better” than 100, when clearly in this scenario 200 was less productive relative conversion growth than 100 Thus created metric – per member per month (pmpm) – to take care of this issue Approach taken with number of conversions, number of baptisms, number of churches planted

© 2004 Missions International Further Treatment of Dependent Variables - indexing Baptism often associated with conversion, thus reported numbers of baptisms highly correlated with numbers of conversions Created a single index of conversions and baptisms This index is a simple average of the two quantities, expressed in pmpm units

© 2004 Missions International Further Treatment of Numeric Dependent Variables- normalization Distributions of variables in original and pmpm forms “positively” / “right” skewed Multivariate procedure - assumptions call for transformations to normality Cube root transformation chosen

© 2004 Missions International Further Treatment of Dependent Variables - outliers Examined distributions on numeric pmpm variables in terms of “standardized” / “z-scores” This procedure helps identify “outliers” – cases that are quite different from most other cases in the sample A typical cut-off would be z-scores greater than 3.0 in absolute value (+/- 3.0) After inspection, allowing retention of more original sample, the cut-off for this study was set at +/- 3.5 Typically less than 1% of cases eliminated

© 2004 Missions International Final Variables For Predictive Models after all Treatments Dependent Variables –Index of conversions and baptisms, pmpm (cube-root transformed) –Number of churches planted, pmpm (cube- root transformed) –Self-reported frequency of church starts Independent Variables –31 items capturing domain of “DNA of CPM”

© 2004 Missions International Model Building - regression Many available approaches to predictive model building Typical statistical tool is multiple regression Predict a dependent variable from a set of independent variables Find a weight for each independent variable such that the weighted combination of independent variable values produces an output that is “as close as possible” to the actual/observed dependent variable values

© 2004 Missions International Model Building - Regression Weights combine independent variables to produce an output – a predicted dv value (w1*iv1 + w2*iv2 + w3*iv3…) = output / predicted dv Output/predicted dv as close as possible to observed dependent variable scores Technically, squared errors between predicted dv and observed dv are minimized, hence the name “least squares” regression When standardized, regression weights can be interpreted as “relative impact” – how much unique impact each iv has on the dv

© 2004 Missions International Model Building Procedure There are different approaches to regression depending on the research objectives Many options for variable selection in building a predictive multiple regression equation –Backward entry –Forward entry –Stepwise entry –Simultaneous/ forced entry A hybrid “procedure” was developed here to arrive at the final models

© 2004 Missions International Hybrid Model Building Procedure Not logical to accept a variable as a significant driver if it shows no meaningful bivariate correlation Thus, screened independent variable set to include only starting variables with meaningful correlations Organized meaningfully correlated independent variables into “tiers”, e.g., –.4s and above –.3s –.2s –.1s

© 2004 Missions International Hybrid Model Building Procedure Used forward entry within each tier, starting with strongest correlation tier first Forward entry added variables one at a time based on biggest improvement in R-square Forward entry thus identified subset of significant predictors in strongest correlational tier That subset became starting basis for consideration of next tier

© 2004 Missions International Hybrid Model Building Procedure Forward entry within next tier added additional significant predictors Significant predictors from first two tiers became starting basis, in moving to the next correlational tier Tested that next tier for any additional variables adding predictive power Procedure continued until all tiers were exhausted (note – not all models had all four tiers of correlational strengths. If only.3s,.2s,.1s existed, general procedure was still same, progressing from highest strength tier to lowest strength tier) After no additional variables added significant predictive power, tried to re-enter any previously eliminated variables When no additional variables add anything, final predictive set has been discovered

© 2004 Missions International Hybrid Model Building Procedure Traditional regression diagnostics checked at each step along the way (e.g., outliers, collinearity, normality of errors of prediction, quality of prediction as evidenced in adjusted R-squared, checks for sign reversal/ suppressor variables) Note 1 – for treatment of “missing” data, large total sample size allowed use of “listwise” procedure – modeling used only those individual respondents with complete data on all variables in any given model Note 2 – a few negative regression coefficients were accepted as valid. This was an acceptable finding only when: –Original bivariate correlation was negative, thus indicating not a sign-reversal / suppressor problem –Interpretation of negative association was plausible and logical given understanding of domain of study

© 2004 Missions International Overall Model Summaries

© 2004 Missions International Self-Report Planting Drivers = negative association

© 2004 Missions International Conversion/ Baptism Drivers = negative association

© 2004 Missions International Numeric Planting Drivers = negative association

© 2004 Missions International Interpretative Note Substantive interpretation is a synthesis across predictive models, also made in light of original qualitative research with experts This synthesis draws out key areas of focus found to be predictive of Church Planting success If generalizability holds beyond the measured samples, based on the data collected for this study, deeper implementation of positively related DNA elements (and avoidance of negatively related ones) should lead to more conversion growth and baptisms, and increased levels of church planting activity