Using Visual Analytics to Evaluate the Effectiveness of Public Subsidies: The Case of Film Projects in South Africa. Alan Collins Alessio Ishizaka Jen Snowball ACEI Montreal Conference 2014

Creative industries and subsidy The case for subsidies – Positive externalities (education, culture) – Merit goods – Infant industry in global market – Competition for FDI (jobs, skills, economic impact) The case against – Inefficiency (negative rates of return & poor quality) – Costly to governments – Crowding out of private investment – Encourages “subsidy wars”

The SA Film Industry 1910: Mainly for white audiences; supported by tax subsidies (1956 – 1992) Post 1992 – Cultural value films with local content (NFVF) VERSUS – Economic impacts of film as a “creative industry” (DTI): employment, skills, GDP

SA Film & TV incentive schemes Foreign Production and Post- Production Incentive SA Film & TV Production and Co-production Incentives Objectives Attract large budget film and TV productions that will enhance job creation, skills and international profile of SA film industry. To support the local film industry and to create employment in South Africa. Benefits 20% of Qualifying South African production expenditure (QSAPE) +5% if post-production 35% of first R6m of QSAPE, and R25% thereafter Eligible Applicants1)QSAPE of R12m and above, at least 50% of principal photography in South Africa, 4 week minimums 2)Post production QSAPE of R1.5m and above, minimum of 2 weeks 3)Compliance with B-BBEE 1)Parent company of which must have a majority of South African shareholders, of whom at least one must play an active role in the production 2)Minimum QSAPE of R2.5m 3)Compliance with B-BBEE

Research Questions and Data Have the DTI film & TV incentives been effective in stimulating economic growth, job creation, skills development and transformation? Data: DTI incentives data (Feb 2009 – June 2012) – boils down to 69 completed project returns for this work plus informed by: – Unpublished industry reports – Interviews with key stakeholders

Does the subsidy work for all films? Subsidy Concentration Index (SCI) % Year Combined SCI (Top 3 Firms) SCI (Top 5 Firms) SCI (Top 10 firms) Source: Authors’ calculations using DTI data SCI determined by % share (of Top N firms) of total subsidy payments in each time period.

Incentive payments, Qualifying SA Expenditure (QSAPE) and GDP impact from 2009 – 2011 (in millions of South African Rands) Production Type Incentive payments: 3 year average (2009 – 2011) QSAPE: 3 year average Ratio of subsidy to QSAPE Ave. increase in real GDP p/a South African Production : Co- Production : Foreign Production : Total : All figures reported in 2010 prices Sectoral multipliers from the South African Industrial Development Corporation used to calculate changes in GDP.

Job creation of subsidised projects Direct FTE jobs Indirect FTE jobs South African Co- Production Foreign Total Transformation? Notes: Employment duration weightings were applied to convert project jobs for each film/television project into 1 year full time equivalent (FTE) employment South African Industrial Development Corporation (2010) employment multiplier for the “Motion picture, radio, television and other entertainment activities” of 4.49

Preference Ranking Organization METHod for the Enrichment of Evaluations (PROMETHEE) Why choose it? It requires only a few parameters and is easy to use and explain via non-technical user- friendly visualization software that is readily available (Gilliams et al. 2005). Graphical Analysis for Interactive Aid (GAIA). Evaluation of each ‘performance’ criterion can be expressed in their own natural units and therefore problems relating to scaling effects are completely eliminated. Normalization of the scores is not required. How and What it does The decision-maker needs to define a preference function that is generally characterized only by an indifference and preference threshold. See section 4 of our paper for an outline summary. Or see the outline and review of studies deploying the technique in Brans and Mareschal (2005) and Mareschal et al. (2008) cited in our paper. The upshot is that a score is produced that is entirely relative to the pool of other projects. The score is relative and sums to 0. This means that if we have, say, only two films and one has a score of 0.5, then the second film will have a score of -0.5.

Criteria Considered by Policymakers Employment * Black employment Black ‘skilled’ employment Spending in the economy * International Investment NOT Concerned with Quality Key finding: Considerable degree of orthogonality among the employment and project spending criteria…. ……Accordingly, there exists potentially considerable policy tension among any partisan advocates of these objectives in the contemporary South African context.

GAIA Plane – Equal Weighting of Criteria

Project Scores

GAIA Plane: Total Black and Skilled Black Criteria Doubled Weighted

Project Scores: Total Black and Skilled Black Employment Criteria Doubled Weighted

GAIA Plane with ‘Ideal’ Film Benchmark

Project Scores – With ‘Ideal’ Film Project Benchmark

Results Baseline Results These are robust and suggest a threefold categorization evident: 1) Projects scoring low in all criteria, which are the majority of projects 2) Projects scoring high in QSAPE and International co-production but scoring low on employment criteria (F43) 3) Projects scoring high in the employment criteria and scoring low in QSAPE and international co- production. Benchmark Analysis The benchmark project features the real top scores on each criteria drawn from actual projects within the data As can be seen from its score value and its relative isolation visually there is considerable distance between it and the real projects.

Policy suggestions More nuanced Subsidy policy Differentiation needed for competitive fringe i.e. those with low score metrics (<0.1)