Public Transport Data from ATCO.CIF files See the specification at www.travelinedata.org.uk/CIF/atco-cif-spec.pdf Tabular format in database is broken down into separate records for each individual trip

Basic Records QS - Journey Header (one) QO - Origin Record (one) QI - Intermediate Record (many) QT - Destination Record (one)

QSNG+++++++++20050606209912311111100++480+++++++BUS+++++++++++++1 QS - Journey Header (1) (S for service) QSNG+++++++++20050606209912311111100++480+++++++BUS+++++++++++++1 What is shown here as a + is a <space> in the real file! G+++ Operator Code ++++++ Unique Identifier 20050606 Start Date 20991231 End Date 1111100 Days of Operation [Mon- Sun] + School Term Time (S/H/blank)

QSNG+++++++++20050606209912311111100++480+++++++BUS+++++++++++++1 QS - Journey Header (2) (S for service) QSNG+++++++++20050606209912311111100++480+++++++BUS+++++++++++++1 What is shown here as a + is a <space> in the real file! + Bank Holidays (A/B/X/blank) 480+ Service Number ++++++ Running Board BUS+++++ Vehicle Type ++++++++ Registration Number 1 Route Direction

QO & QT - Stop Records (O for origin, T for terminating) QO4200F087400+0653+++T1++ QT3400000722++0737+++T1++ What is shown here as a + is a <space> in the real file! 4200F087400+ NaPTAN Stop Code 0653 Time +++ Bay Number T1 Timing Point indicator ++ Fare Stage Indicator

What is shown here as a + is a <space> in the real file! QI - Stop Records (I for intermediate) QI340000668SCH07120712B+++T1++ What is shown here as a + is a <space> in the real file! 340000668SCH NaPTAN Stop Code 0712 Arrival Time 0712 Departure Time B Activity Flag (B/P/S) +++ Bay Number T1 Timing Point indicator ++ Fare Stage Indicator

Simple Journey QSNG 20050606209912311111100 480 BUS 1 QO4200F087400 0653 T1 QI4200F005200 06540654B T0 QI4200F071500 07010701B T0 QI4200F071300 07020702B T1 QI4200F040800 07040704B T0 QI4200F040900 07050705B T1 QI340001473OUT07090709B T0 QI340001473OPP07090709B T0 . QI340001474OPP07290729B T0 QI340003150CNR07330733B T0 QI340001455OPP07350735B T0 QI340001460PO 07360736B T0 QT3400000722 0737 T1

Additional Records (1) QN - Journey Note OCC++ Note Code QNOCC++Financially supported by the Local Authority++++++++++++++++++++++++++++ What is shown here as a + is a <space> in the real file! OCC++ Note Code (everything else) Note Text BUT . . . remember the SPAM filter

Spam Filter (1) Source data may have multiple uses Aim to remove irrelevance & reduce clutter on journey planner & timetables QN record descriptions filtered on text strings Words such as “financial”, “continues” and “rover” List can be seen on DMS (http://80.168.17.69/selta/server/DefinitionFiles/qn_spam.txt)

Spam Filter (2)

QE - Journey Running Date (Exceptions) Additional Records (2) QE - Journey Running Date (Exceptions) QE20081231200812310 20081231 Start of Exceptional Period 20081231 End of Exceptional Period 0 Operation Code 0 = does NOT run 1 = DOES run

Other Records QR - Bus Journey Repetition QP - Bus Operator QD - Route Description QV - Vehicle Type QH - Bank Holiday QL QA QB QC QG QJ QW - Location

Full Journey QSNG 20050606209912311111100 480 BUS 1 QE20051224200512240 QE20051225200512250 QE20051226200512260 QE20051231200512310 QNOCC Financially supported by the Local Authority QO4200F087400 0653 T1 QI4200F005200 06540654B T0 QI4200F071500 07010701B T0 QI4200F071300 07020702B T1 QI4200F040800 07040704B T0 QI4200F040900 07050705B T1 QI340001473OUT07090709B T0 . QI340001474OPP07290729B T0 QI340003150CNR07330733B T0 QI340001455OPP07350735B T0 QI340001460PO 07360736B T0 QT3400000722 0737 T1

Weekly Data Process (1) Latest NPTG Latest NaPTAN Operator Data / Suppression / Composites / Route Descriptions etc (from Service Table) (External) Holiday Tables Local Authority CIFs The ‘glue’ “Build” contains Subnets – EM, EA, SE, London, National Rail & Coach (+ Hampshire & Bedfordshire/Thurrock)

Service Table (1) Adjudicates on duplicated (x-boundary) services Can suppress mistakenly retained services too Specifies a fixed route description Specifies mode (bus, coach, ferry) Specifies DDA information (low floor, wheelchair) Generates unique service ID (persistence) Creates composite timetables (multiple services and/or operators on one timetable)

Service Table (2)

Holiday Tables (1) Overlays normal operating patterns with “standard” holiday patterns without the need to amend database (“Special” timetables DO need to be keyed into database) Default pattern Update regularly, not “one hit” Holiday Tables (and Service Table) available on DMS

Holiday Tables (2)

Holiday Tables (3)

Weekly Data Process (2) Create JP Data Create TT data Precompilation of TTs Build put on test server “Protocols” put on DMS (Message Lists) Testing Release to live servers

“Protocols” - What can go wrong? Data improbabilities Descending Trip Times (error 1) Stop not recognised (error 3) Speed too high (error 40) Speed exceptionally high (error 43) Direction Flags (warning 1018) No trip times (warning 1021) Unclear note (warning 1023) Speed too low (warning 1025)

Speed Exceptionally High Economies of Scale? Error 1 Descending Trip Times Error 3 Stop not recognised Error 40 Speed Too High Error 43 Speed Exceptionally High errors services stops Square 16 5 78 37 30 45 18 23 3 Bucks 14 6 2 Essex 61 4 40 25 26 27 12 Oxon 8 196 31 39

Data Management System (DMS) on server 69 Check weekly Late Wednesday/early Thursday narrow window of opportunity before next CIF is created Finding “needles in a haystack”

(Error 1) Descending Trip Times Impossible Treated as continuing next day Jumps forward 23hrs 55mins rather than back 5 mins JP trips travelling only BEFORE the error Okay JP trips travelling ACROSS the error Too Slow to be offered JP trips travelling only AFTER the error Wrong day! HINT! - Look for ‘out of position’ asterisk on PDF t/t

(Error 3) Stop not Recognised Non-NaPTAN 12 digit reference Stops “thrown away” Legacy Issue? Mistyped? Deleted? Out of Area?

(Errors 40/43) Speed Too High Over 120 kph between stops Covers more than 1.2 km in zero minutes but not a ‘severe’ error Error 43 Severe! >150 kph or >1.5 km in zero minutes Prevention of “Wormholes” Over 240kph/2.4km in zero minutes Data thrown away HINT – Try re-interpolating the service

(Warning 1025) Speed Too Low “Warning” - Used to be an “error” Under 10 kph between stops Never “thrown away” Recovery Time – so possibly cannot do anything about Separate Arrive and Depart times?

Other sources of errors Random checking Less random if you know what has changed Public complaints/comments Call Centre feedback Public Web feedback View as Opportunities, not Problems

Our Advice Crack it now Keep on top of it Encourage feedback Special effort now to get down to a very short list Keep on top of it Little to look at, but every week Encourage feedback DMS for data . . . but there may be problems in the “glue” If in doubt, ask! Stuart and David are here to help (subject to availabilty!)

And finally… Looking to the future, we will be migrating from CIF to TransXChange (TXC) TXC is much more rigorous XML allows validation Harder to debug source, though, unless you have an XML debugger But errors and warnings on DMS will be the same…

Any questions?