WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Introduction to BUFR Simon Elliott EUMETSAT

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 What is BUFR? Binary Universal Form for the Representation of Meteorological Data Used for data that are not on a regular grid, such as observations Conceptually equivalent to CREX, but format is binary rather than alphanumeric

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 What does a BUFR message look like? (In other words, just an apparently random string of 0s and 1s!)

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Sections of a BUFR message 0 Indicator section 1 Identification section 2 Optional local use section 3 Data description section 4 Data section 5 End of message

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 0 – Indicator section The character string BUFR indicating the start of the message The total length of the message The BUFR edition number This section contains:

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 0 - Details Length always 8 Octets 1-4BUFR (in CCITT IA5) Octets 5-7Total length of message (including Section 0) Octet 8Edition number (currently 4, but 3 is still used)

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Now, lets go back and look at that BUFR message again… B U F R end of section 0 + octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | binary string

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 1 – Identification section The table versions referred to by this message An overall description of the message contents, including: –The originating centre and sub-centre –The data category and sub-category –A representative date and time Whether or not the optional section is included This section contains:

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 1 – Details (example based upon BUFR edition 3) Length at least 18 Octets 1-3 Length of section Octet 4 Master table (0 for WMO, 10 for IOC, etc.) Octet 5-6 Originating sub-centre and centre Octet 7 Update sequence number Octet 8 Flag (Optional section?) Octets 9-10Data category and sub-category Octets 11-12Master and local table version numbers Octets 13-17Date and time typical of message contents Octets 18-??Reserved for local use

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Now, lets go back and look at that BUFR message again… B U F R end of section 0 + octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 1 | 2 | binary string octet number 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | binary string end of section 1 + octet number 13 | 14 | 15 | 16 | 17 | 18 | binary string

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 2 – Optional section It typically contains additional information of use to the ADP centre, such as –Database keys to aid searching for specific data without decoding the message –Anything else a processing centre may find useful This section is defined by the ADP (Automated Data Processing) centre generating or using the message

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 3 – Data description section A count of the number of data subsets (typically individual observations) Flags indicating whether or not the data are compressed or uncompressed and observed or forecast A list of the data elements (fields) that are contained in each data subset This section contains:

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 3 - Details Length at least 10 Octets 1-3Length of section Octet 4Set to zero Octets 5-6Number of subsets Octet 7Flag (Obs?, Compressed?) Octets 8-??List of descriptors Each descriptor 2 bits F, 6 bits X, 8 bits Y

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Now, lets go back and look at that BUFR message again… B U F R end of section 0 + octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 1 | 2 | binary string octet number 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | binary string end of section 1 + octet number 13 | 14 | 15 | 16 | 17 | 18 | 1 | 2 | 3 | 4 | binary string end of section 3 + octet number 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | binary string

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 4 – Data section The actual data as specified by Section 3 One of two formats is used –Compressed –Uncompressed Such data are still packed, but not as efficiently as compressed data usually are This section contains:

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 4 - Details Octets 1-3Length of section Octet 4Set to zero Octets 5-??Binary data as specified by Section 3

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Now, lets go back and look at that BUFR message again… B U F R end of section 0 + octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 1 | 2 | binary string octet number 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | binary string end of section 1 + octet number 13 | 14 | 15 | 16 | 17 | 18 | 1 | 2 | 3 | 4 | binary string end of section 3 + octet number 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | binary string end of section 4 + octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | binary string

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 5 – End section The character string 7777 indicating the end of the message Checking for this indicator can be useful to detect some types of data corruption (especially missing bytes in the rest of the message) since the total length of the message is known from Section 0 This section contains:

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Now, lets go back and look at that BUFR message one last time! B U F R end of section 0 + octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 1 | 2 | binary string octet number 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | binary string end of section 1 + octet number 13 | 14 | 15 | 16 | 17 | 18 | 1 | 2 | 3 | 4 | binary string end of section 3 + octet number 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | binary string end of section octet number 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 1 | 2 | binary string end of section 5 octet number 3 | 4 | binary string

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 BUFR Descriptors Section 3 contains a list of BUFR descriptors These describe the data elements that are contained in Section 4 Most descriptors are references to BUFR Tables B, C and D Using the list of descriptors in Section 3, together with the tables, it is possible to unpack the data in Section 4

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Types of BUFR descriptors Element descriptors (Table B) Replication descriptors Operator descriptors (Table C) Sequence descriptors (Table D) Specified by 3 numbers in 16 bits (2 octets) –F: 2 bits 0-3 –X: 6 bits 0-63 –Y: 8 bits 0-255

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Element descriptors Defined by entries in Table B F is 0 Each element descriptor describes an encoded value, such as: –The value of a meteorological parameter (e.g. mean sea level pressure, temperature, wind speed) –Instrument details –Location or date and time information –Quality control information

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Replication Descriptors Describe the repetition of one or more element, operator, sequence or other replication descriptors Used for repetitive data such as the individual levels in vertical soundings or temperature profiles Can be: –Fixed - the number of repetitions is pre-determined and the same for all data subsets –Variable - the number of repetitions can differ from one subset to the next (ie. delayed replication)

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Replication descriptors - continued Replication descriptors are defined by three numbers F X Y F is 1 X is an integer between 1 and 63 –Defines the number of descriptors to be repeated Y is an integer between 0 and 255 –Defines how many times the X descriptors are to be repeated –A count of zero indicates delayed replication, where the repeat count is stored in the data section and can change from one data subset to another.

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Operator descriptors Defined by entries in Table C F is 2 Describe changes to be made to other descriptors Operators exist for applications such as: –Changing the scaling and packing of data –Adding quality control or other associated fields –Describing the descriptors to which quality control information applies –Substituting a better value for an element, while retaining the original value

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Sequence descriptors Defined by entries in Table D F is 3 Shorthand notations for pre-defined lists of other element, replication, sequence and operator descriptors Not really necessary, but useful in reducing the overhead involved in transmitting data in BUFR: –Replace a commonly-used sequence of descriptors with a single descriptor, and thereby reduce the overall length of Section 3

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 BUFR tables Table A –Data Categories, used in Section 1 Table B –Element descriptors, used in Section 3 Table C –Operator descriptors, used in Section 3 Table D –Sequence descriptors, used in Section 3 Code and Flag tables –Numerical values to be encoded where the element values are qualitative, used in Section 4 There are many different tables involved in BUFR:

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table A Defines the general category of the data contained in the BUFR message Encoded in Section 1 Examples of typical entries: Code figureMeaning 0Surface data – land 1Surface data – sea 2Vertical soundings (non-satellite) 3Vertical soundings (satellite) …… 6Radar data …… 10Radiological data 12Surface data (satellite) …… 31Oceanographic data

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B Describes the individual values that are encoded Element descriptors are grouped according to classes (i.e. X value) Class NumberClass NameClass NumberClass Name 01Identification 12Temperature 02Instrumentation 13Hydrological …… 14Radiation and radiance 04Location (time) …… 05Location (horizontal-1) 19Synoptic features 06Location (horizontal-2) 20Observed phenomena 07Location (vertical) 21Radar data …… …… 11Wind and turbulence 33Quality information

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Class 01 – Identification (excerpt) TABLE REFERENCE F X Y TABLE ELEMENT NAME UNITSCALE REFERENCE VALUE DATA WIDTH (BITS) WMO block numberNumeric WMO station numberNumeric WMO region numberCode table Buoy/platform identifier Numeric Aircraft flight number CCITT IA Satellite identifierCode table Ship or mobile land station identifier CCITT IA Station or site nameCCITT IA ICAO location indicator CCITT IA50064

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Class 11 – Wind and turbulence (excerpt) TABLE REFERENCE F X Y TABLE ELEMENT NAME UNITSCALE REFERENCE VALUE DATA WIDTH (BITS) Wind directionDegree true Wind speedm s U-componentm s V-componentm s Relative vorticitys Degree of turbulenceCode table Height of base of turbulence m Height of top of turbulence m Vertical gust velocitym s

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B Columns are: –Table reference –Element name –Unit –Scale –Reference value –Data width (in bits) Scale, reference value, and bit width are chosen so that the desired range of possible data values can be stored in BUFR as positive integers –Preserves the machine-independence of BUFR

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B reference Expressed as 3 small numbers F X Y Used to refer to this descriptor F is always 0 for an element descriptor X is in the range 0 to 63 and refers to a broad class of elements –Classes 48 to 63 are reserved for local use Y is in the range 0 to 255 and refers to the individual descriptor in the class –Within all classes, descriptors 192 to 255 are reserved for local use

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B element name Natural language description of the meaning of the value Usually English but could be translated to other languages For example: –Brightness temperature –Total precipitation past 3 hours –Wind speed at 10m

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B unit The units used for the value –Normally SI units are used –CCITT IA5 (the international version of ASCII) is used for character data such as identifiers –Code Table is used for qualitative data where only one of a set of possible values can be applicable in a given data subset –Flag Table is used for qualitative data where more than one of a set of possible values may be applicable in a given data subset –For qualitative data, the coded values are references to the Code and Flag tables

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B scale Scale –Power of 10 by which to multiply the data value before packing –Determines the precision with which the data are encoded –A scale of 2 means 2 decimal places of precision (eg ) –A scale of –1 means that the data values are rounded to the nearest multiple of 10

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B reference value Used to subtract an offset where negative data have to be encoded Table B contains the value (multiplied by the scale) of the offset to be subtracted For example, scale=2, reference value means that is to be subtracted before scaling (ie after scaling), allowing values as negative as to be represented

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B data width The number of bits to be used to encode the value If all bits are set to ones when encoding (ie. a value of (2 n -1) when n is the data width), then this denotes a missing value. If the scale is s, the reference value is r, and the data width is n, then the representable range of values is: –Minimum (10 -s r) –Maximum (10 -s (2 n -2+r)) and (10 -s (2 n -1+r)) denotes the missing value.

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B examples Table reference F X Y Element nameUnitScaleReference value Data width Wind speed at 10mm s Total precipitation past 3 hourskg m Present weatherCode table Vertical sounding significanceFlag table007

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B examples - continued Wind speed at 10m Scale=1, Reference value=0, Data width=12 Precision is one decimal place (ie. 0.1 m s -1 ) Minimum representable value is: (10 -1 ×0) = 0.0 m s -1 Maximum representable value is: (10 -1 ×( )) = m s -1 Missing value is: (10 -1 ×( )) = m s -1

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B examples - continued Total precipitation past 3 hours Scale=1, Reference value=-1, Data width=14 Precision is one decimal place (ie. 0.1 kg m -2 ) For this descriptor, -0.1 kg m -2 is a special value for trace, according to a specific note in Table B Minimum representable value is: (10 -1 ×-1) = -0.1 kg m -2 (= trace) Maximum representable value is: (10 -1 ×( )) = kg m -2 Missing value is: (10 -1 ×( )) = kg m -2

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table B examples - continued Present weather Scale=0, Reference value=0, Data width=9 Coded values are integers since Scale=0 Minimum representable value is: (10 0 ×0) = 0 Maximum representable value is: (10 0 ×( )) = 510 Missing value is: (10 0 ×( )) = 511 One must refer to Code Table in order to discover the actual meaning of each coded value

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, – Present Weather Code Table (excerpted) Code figureMeaning 0Cloud development not observed or not observable 1Clouds generally dissolving or becoming less developed … 10Mist 11Patches of shallow fog or ice fog 13Lightning visible, but no thunder heard … 171Snow, slight 172Snow, moderate 173Snow, heavy … 511Missing

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Code tables vs. Flag tables (choice of one vs. choice of more than one) WMO region numberVertical sounding significance Code figureMeaningBit numberMeaning 0Antarctica 1Surface 1Region I 2Standard level 2Region II 3Tropopause level 3Region III 4Maximum wind level 4Region IV 5Significant 5Region Vtemperature level 6Region VI 6Significant wind level 7Missing value All 7Missing value For a Code table, the value stored in Section 4 is the code figure corresponding to the applicable meaning. For a Flag table of N bits, the value stored in Section 4 is (2 (N-bit#) + 2 (N-bit#) + …) for the bit(s) corresponding to each applicable meaning. An extra bit is always present in order to be able to distinguish all meanings applicable from missing.

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Some other important regulations pertaining to Table B Elements in classes 01 – 09 are coordinate descriptors which remain in effect until redefined or until the end of the subset Exception: when two identical descriptors from classes 04 – 07 are listed consecutively, they define the boundaries of a range Similar descriptors exist in coordinate vs. non-coordinate classes Example: and are both Pressure with identical scale, reference value and bit width; however, the former is a coordinate for use when pressure is the main defining coordinate measured in the vertical direction (e.g. in radiosondes) vs. the latter which is a non-coordinate for use when pressure is a derived value (e.g. an aircraft calculating pressure as a function of an observed or measured height) Class 08 contains significance qualifiers which can be used to report qualitative information and which can be explicitly cancelled Example: with value 12 can indicate that we are talking about a cloud

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table C Describes the various operators Columns are: –Table reference F X F is 2 X is an integer between 0 and 63 There is no sub-range of X values reserved for local use –Operand A number between 0 and 255 –Operator name A short name describing the operation –Operator definition A detailed description of the operation and its effects

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table C TABLE REFERENCE F X OPERANDOPERATOR NAMEOPERATOR DEFINITION 2 01YChange data widthAdd (Y-128) bits to the data width given for each data element in Table B, other than CCITT IA5, code or flag tables 2 02YChange scaleAdd (Y-128) to scale in Table B for elements which are not CCITT IA5, code or flag tables 2 03YChange reference valueSubsequent element descriptors define new reference values for corresponding Table B entries. Each new reference value is represented by Y bits in Section 4… 2 04YAdd associated fieldPrecede each element with Y bits of information (e.g. quality marker). 2 05YSignify characterY characters from CCITT IA5 are inserted as a field of (Y*8) bits in length. 2 06YSignify data width for following local descriptor Y bits of data are described by the immediately following local descriptor from Table B This is just an excerpt – there are many other (even more complicated!) operators in Table C. There are also many important notes to Table C describing, e.g. how to cancel an operator.

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table C example Table reference F=2 X=01 Operand, in this case represented as Y Operator name Change data width Operator description: Add Y-128 bits to the data width given for each data element in Table B, other than CCITT IA5 (character) data, code or flag tables According to a note under Table C, this operator is cancelled (ie. effect is turned off) by repeating the operator with Y=0, or at the end of each data subset

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table C example - continued The Change data width operator causes the data width to be changed for subsequent elements, in effect giving them a larger (or smaller) range than is otherwise prescribed within Table B. Thus, it can be used to: encode values that exceed the usual representable range for a descriptor, instead of having to introduce a new Table B descriptor (note: in such cases, Y > 128) reduce the size of the data (and thus the overall encoded message as well!) if the required data range can be encoded using a smaller data width than provided within Table B (note: in such cases, Y < 128)

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table C example -continued As an example, one of the standard descriptors for the height coordinate of an observation is with unit=m, scale=0, reference=-1000, data width=17, giving a representable range of –1000 m to m. If one needed to encode a value larger than this, then the 2 01 operator could be used to increase the data width. For example, use of the operator before the descriptor would increase its data width to 19 bits and therefore allow values up to m.

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table D Describes sequences of descriptors Columns are: –Table reference F X Y F is 3 X is in the range of 0 to 63 and refers to a broad category of sequences –Categories 48 to 63 are reserved for local use Y is in the range of 0 to 255 and refers to the individual sequence in the category –Within all categories, entries 192 to 255 are reserved for local use –Table references List of other descriptors, including element descriptors (Table B), replication descriptors, operator descriptors (Table C) and other sequence descriptors (Table D) –Element name Not really necessary, but makes reading the tables easier

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table D categories Categories correspond to the X value of the underlying sequence descriptor. Category numberCategory name 01Location and identification sequences 02Sequences common to surface data 03Sequences common to vertical soundings data 04Sequences common to satellite observations 05Sequences common to hydrological observations 06Sequences common to oceanographic observations 07Surface report sequences (land) 08Surface report sequences (sea)… 18Radiological report sequences 21Radar report sequences

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Category 01 – Location and Identification Sequences (excerpt) TABLE REFERENCETABLE REFERENCESELEMENT NAME WMO block number WMO station number Year Month Day Hour Minute Latitude (coarse accuracy) Longitude (coarse accuracy) Height of station (Land station for vertical soundings) WMO block and station number Radiosonde type Radiosonde computational method Date Time Lat/long (coarse accuracy), height of station

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table D example Table reference F X Y Table References Element name (Buoy/platform – fixed) Buoy/platform identifier Type of station Date Time Latitude and longitude (high accuracy)

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Table D example - continued In this example, the sequence consists of five descriptors Two of these are Table B element descriptors The other three are Table D sequence descriptors, which in turn represent other sequences of descriptors Obviously the repeated replacement of Table D descriptors with lists of descriptors must not continue forever, and a Table D descriptor cannot include itself, directly or indirectly

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Replication descriptors - example The 4 descriptors: – Repeat three descriptors twice – Direction of swell waves – Period of swell waves – Height of swell waves Are equivalent to the 6 descriptors: – Direction of swell waves – Period of swell waves – Height of swell waves – Direction of swell waves – Period of swell waves – Height of swell waves

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Delayed replication descriptor - example The 3 descriptors: – Delayed replication of one descriptor – Replication factor (8 bit) – Winds at pressure levels Are equivalent to: – Winds at pressure levels –… for as many times as specified by the replication factor The number of repeats for each data subset is stored in the data. Each data subset has its own repetition count. In addition to 8-bit delayed replication, there is also 1-bit and 16-bit delayed replication

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 A caveat… The previous example is worth a closer look. Specifically, when replicating a sequence, the replication takes place before the sequence expansion! For example, note that: ( = , , , ) is equivalent to:and not to:

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 4 – Detailed description The form of the data in Section 4 varies depending upon the descriptors and flags in Section 3 –One of the main differences concerns whether the data are uncompressed or compressed –Delayed replication and compression may be combined, but only if the number of repetitions is the same for each subset within a particular BUFR message

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 4 – Uncompressed data Each data item takes up the number of bits specified by the data width in Table B, as adjusted by operators or as otherwise specified. The order for N data subsets, each with M data values is: Set 1 Value 1, Set 1 Value 2, Set 1 Value 3, … Set 1 Value M Set 2 Value 1, Set 2 Value 2, Set 2 Value 3, … Set 2 Value M … Set N Value 1, Set N Value 2, Set N Value 3, … Set N Value M

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Section 4 – Compressed data Each data item is encoded by its minimum over all the data subsets (using the data width specified in Table B or otherwise), the number of bits used to encode the increments (using 6 bits), and a list of increments to be added to the minimum. The order for N data subsets, each with M data values is: Min 1, Nbits 1, Set 1 Inc 1, Set 2 Inc 1, … Set N Inc 1 Min 2, Nbits 2, Set 1 Inc 2, Set 2 Inc 2, … Set N Inc 2 …. Min M, Nbits M, Set 1 Inc M, Set 2 Inc M, … Set N Inc M

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Summary BUFR is flexible –It can represent a wide range of data types BUFR is table-driven –Tables A, B, C and D (and code and flag tables) contain majority of information needed to encode and decode data BUFR is self-describing –The data description section describes the data, using element (Table B), replication, operator (Table C) and sequence (Table D) descriptors Replication descriptors and sequence descriptors can be used to reduce the size of the data description Compression can be used to reduce the size of the data themselves

WORLD METEOROLOGICAL ORGANIZATION RA-VI Regional Training on BUFR and Migration to Table Driven Code Forms Langen, Germany, April, 2007 Acknowledgements Based on: WMO No. 306 Manual on Codes, Volume I.2 ( Guide to WMO Table-Driven Code Forms FM94 BUFR and FM95 CREX ( Special thanks to: Jeff Ator NOAA, United States of America Charles Sanders BOM, Australia (retd.) Joël Martellet WMO