1. In response… How QCA can be combined with other methods in evaluation? How can various ideas that are part of the whole QCA package be useful in themselves? Rick Davies, Stockholm, 27 June 2016

2. The QCA view of causality is rich, and has some correspondence with the complexity of reality Multiple conjunctural causation Asymmetry INUS and SUIN and other variations (A+B) or (C+D) (A or B) + (C or D) Inclusive or Exclusive OR: E.g. combinations of grant and credit assistance Rick Davies, Stockholm, 27 June 2016

3. QCA is especially suited to programmes where there is significant diversity of contexts and interventions Arising from Participatory planning and implementation processes Large scale projects with decentralised management (either by design or necessity) These circumstances are, in my experience, the rule rather than the exception The notion of a standardised “treatment” rarely fits real development projects Rick Davies, Stockholm, 27 June 2016

4. QCA works with “loose” Theories of Change “Loose” ToC = We have an idea about the ingredients but not how they fit together. We can only find out after all the data is in and has been analysed. Suitable to adaptive / participatory programming where local mixes of a menu of interventions will vary. Precisely the situation where searching for a net average affect is not appropriate because it obliterates the very differences that are potentially so important sources of learning Rick Davies, Stockholm, 27 June 2016

5. QCA ideas are useful elsewhere Notions of necessity and sufficiency are useful in the design of programmes, as expressed in a diagram tic version of a ToC. Linkages between events and combinations of these can and should be identified in these terms. It should make a big difference to evaluation planning if a causal link in a ToC diagram is sufficient versus necessary but not sufficient Rick Davies, Stockholm, 27 June 2016

6. Inconsistency is not something that we always want to remove by improving a model. In a configuration of conditions that is associated with the absence of an outcome an inconsistent case is a Positive Deviant, Positive Deviant cases offer hope that outcomes can be achieved despite prevailing conditions which are discouraging. We need to rejoice in finding these and carry out follow up by in-depth investigations of these cases Rick Davies, Stockholm, 27 June 2016

7. QCA practice of eliminating inconsistent cases seems to privilege consistency over coverage But sometimes coverage may be more important In practice the QCA guide does highlight that both are important Iit is important that this choice is made consciously rather than by an automatic procedure Surgeons and stockmarket traders have different needs. Surgeon want high consistency Stock market traders want high coverage, lower levels of consistency are okay Project managers maybe somewhere in the middle Rick Davies, Stockholm, 27 June 2016

8. QCA’s QuineMcCkluskey algrorithm is one of a number of algorithms that can help us find widely applicable solutions. All algorithms have their limitations. QMcL is limited diversity But it can be combined with other algorithms. Exhaustive search still will work when there are lots of conditions relative to cases, so long as there are not many cases. It will often produce multiple equally well fitting solutions. But QM can then sometimes be used to reduce multiple solutions to a smaller number Genetic algorithms can help use search much larger combinatorial spaces, but can also generate multiple good fitting models, that still need selection, and where QmC can be used Decision Tree algorithms can use same data and produce comparable results, which can be used to help us triangulate QCA findings Rick Davies, Stockholm, 27 June 2016

9. Hypothesis led inquiry and algorithmic search can be fruitfully combined. The latter helps use search Searching for and testing alternate theories is something we are supposed to do, but often this is done in very limited ways, by just exploring local variations of a hypothesis Algorithmic search helps use systematically search a finite space of alternate models, where we may find solutions that do better than our existing hypotheses. Rick Davies, Stockholm, 27 June 2016

10. Interpretation of QCA type solutions is the territory that needs further method exploration Resolution of inconsistencies in initial truth table is not sufficient to ensure we understand the underlying causal mechanisms at work We need guidance on case selection and then analysis within those cases. My interest so far has been on case selection strategies And on comparisons that can be made between different categories of cases Two types of cases that fit a solution should be investigated: modal and outlier. Consistent with BB argument re probative value of evidence Confusion matrix examples Consistent and inconsistent cases (TP and FP) Outcomes observed outside the model and within it Models with overlapping coverage are common place. This suggests cases involved in two or models should be selected for detailed within-case inquiry follow up Especially when # of conditions is big relative to number of cases Rick Davies, Stockholm, 27 June 2016