2. J. Khélif Insee July 2008 A quality report for seasonally and trading day adjusted French IIP

3. Page 2 Quality report on SA J. KhélifJuly 2008 Scheme of the presentation 1.General background 2.Building a quality report for IIP 3.Is the tool useful ? How can it be improved ?

4. Page 3 Quality report on SA J. KhélifJuly 2008 Background When analyzing short term economical indicators, users comment on: –seasonally adjusted (SA) and/or trading day (TD) adjusted series; –these series are expected to reflect recent economical evolutions; –producing good quality SA-TD adjusted series is an important issue.

5. Page 4 Quality report on SA J. KhélifJuly 2008 Background › Different problems have to be faced by the producer: 1.Criteria: there are no universal criteria for “good quality” SA-TD adjusted series; 2.Timeliness : time for analysis is very short, poor quality problems have to quickly be identified; 3.Know-how : Producers are not necessarily SA experts. › Our aim: build a tool that helps us judge and improve the quality of our SA-TD adjusted series –The tool had to be easy and quick to run; –The tool had to propose synthetic analysis tables; –The tool had to be based on widely accepted quality criteria.

6. Page 5 Quality report on SA J. KhélifJuly 2008 2. Developing a tool to evaluate the quality of SA-TD series › Select criteria among many: –There is no unique way of SA series (unobserved components); –Each SA adjustment method comes with a series of widely accepted tests; –We made a selection according to our own local preoccupations. › Synthesize the information : – our aim: check for quality with a top-down approach; –Find a way of judging the quality of aggregates; –Target those series that are “really” problematic. › Our solution : –give grades to elementary series based on the technical criteria chosen; –aggregate them according to their weight in the economy.

7. Page 6 Quality report on SA J. KhélifJuly 2008 2. Developing a tool to evaluate the quality of SA-TD series › We decided on controlling 3 main elements: 1.The quality of pre-adjustments and the Reg-Arima estimation of the raw data; 2.Idempotency : SA and TD adjusted series should not show seasonal and/or trading day patterns. › Our notation system : 1.Quality of pre-adjustments and Reg-Arima modeling Maximum mark : 20 2. IdempotencyMaximum mark : 20

8. Page 7 Quality report on SA J. KhélifJuly 2008 2.1 Measuring the quality of pre-adjustments and Reg-Arima modeling 5 items are checked using Tramo-Seats in Demetra output: –the quality of pre-adjustments : if outliers represent more than 5% of all the observations the series gets a mark of 4 (0 otherwise); –White noise tests on residuals after Arima modeling: if residuals pass the tests (4 of them) we consider that the modeling is ok (16 points if all 4 are ok). Possible solutions when the quality is poor: –When there are too many outliers : change the time span; –When the residuals are not white noise : change the model, change the time span.

9. Page 8 Quality report on SA J. KhélifJuly 2008 2.2 Idempotency 2 items are checked in order to make sure that SA-TD adjusted series show no residual seasonality (TD effects): –Global TD effects detection test (maximum mark=8); –The X11-Arima F-test for identifiable seasonality (maximum mark=8); –The weight of the irregular component (maximum mark=4). Possible solutions when the quality is poor: –The series should not have been TD adjusted; –The pre-adjustments and Reg-Arima modeling is problematic and should be re-run; –The seasonality is changing and nothing can be done at first.

10. Page 9 Quality report on SA J. KhélifJuly 2008 3. The IIP quality report for April 2007 The quality of SA & TD adjustments seems ok for the IIP: -Idempotency is almost always ok -Whereas the Reg-Arima modeling is of poorer quality.

11. Page 10 Quality report on SA J. KhélifJuly 2008 3. The IIP quality report for April 2007 › 33 elementary indices (out of 120) didn’t pass at least one white-noise test; › The quality results weren’t satisfactory for wearing apparel & machinery equipment (decreasing production, new surveys); › An example of what happens when you don’t change your Arima models often enough: the automotive industry in November 2005.

12. Page 11 Quality report on SA J. KhélifJuly 2008 3. The IIP quality report for April 2007 › Idempotency: checking certain properties –The SA-TD adjusted series must not have seasonal/trading day patterns ; –Same criterion for the irregular component; –The irregular component should not explain most of the series evolutions. › In April 2007, for the IIP : –25 times out of 120 we detected residual seasonal and trading day effects ‐ The series are unstable (recent surveys) ‐ This can happen when particular events close to TD effects happen (some firms close during the summer, change in dates of school vacations, long week-ends…). –The average weight of the irregular component is almost always very low (inf. to 80%) –Examples : electricity production was not adjusted for TD effects whereas it should have been ; unstable elementary indexes in “machinery & equipment” ; problems with firms closing in august since 2004 in the automobile industry.

13. Page 12 Quality report on SA J. KhélifJuly 2008 3. Is the tool useful ? How can it be improved ? › In September 2007, the report was implemented again –we improved the global quality of the IIP; –we corrected some options (electricity production). › But the tool can be improved –It should be easier to implement ; –It should make more use of X12 quality tests; –It should be harsher on certain criteria (the outlier problem); –It should take other criteria into account (revisions); –It should be made useful to users.