1. FINESTION-2003-2006 a complete dataset of routine air ion measurements 2003 – 2006
in co-operation with: Urmas Hõrrak, Kaupo Komsaare, Pasi P. Aalto, Petri Keronen, and Sander Mirme
Pühajärve 2007

2. T, Tgrad, p, RH, UV, wind, prec, W/m2, visibility
What is included?
Tahkuse
Hyytiälä
Tartu
Instrument
Tahkuse AIS
BSMA1
BSMA2
From To
Meteo
T, Tground, p, RH, wind, W/m2
T, Tgrad, p, RH, UV, wind, prec, W/m2, visibility
T, p, RH, wind, prec, lux, W/m2
Chemistry
NOx,SO2,O3
Dose rate
Mobility
–2.74
0.036–2.74
Size
0.5–65 nm
0.5–6.5 nm
Neutral aerosols are not presented in the distribution version of the dataset

3. Amount of data
diurnal records, mostly 24×6 = 144 measurements per day (some variables 48 or 24 measurements per day) .
Total about 40×106 values (inlc. less than 10% of missing codes).
Unzipped file bytes, about 3.8 bytes per value,
Zipped file 52 megabytes, about 1.4 bytes per value.
Format: DD (DataDiurna) diary.

4. Preprocessing of data BSMA (single-channel scanning) data:
The records with BSMA noise index ≥ 50 were deleted as unreliable (about 15% of measurements was lost in Hyytiälä and 16% in Tartu).
The measurements made with a period of 15 min were converted to the 10-minute period using interpolation by means of the DD data manager.
The values of RH% recorded by BSMA were corrected so that the nighttime maximums do not exceed 100%.
The fraction concentrations were converted to the values of distribution functions dn / d(log d) and dn / d(log Z) assigned to the centers of the fractions in the logarithmic scale of diameter or mobility.
The indices of technical diagnostics were replaced with the integral parameters: cluster ion concentration (Z > 0.56 cm2V−1s−1), intermediate ion concentration (Z = − 0.56 cm2V−1s−1), cluster ion average mobility and air polar conductivity.

5. Tahkuse AIS (multichannel instrument) data:
The physically unbelievable measurements were marked as missing.
The records with too many missing measurements were deleted .
The measurements recorded every 5 minutes were converted to the time step of 10 minutes.
If a missing value was located between known values then the gap was filled using a sophisticated interpolation method.
The data was complemented with the integral parameters: cluster ion (Z > 0.56 cm2V−1s−1), intermediate ion (Z = − 0.56 cm2V−1s−1), and large ion (Z = − cm2V−1s−1) concentrations; cluster ion average mobility and air polar conductivity.
The data was transformed to the dn / d(log d) and dn / d(log Z) values so that a decade is logarithmically uniformly divided into 8 fractions. Thus the first 16 fractions are just the same as in the BSMA measurements. The mobility was converted to the particle size considering the measurements of air pressure and temperature.

6. Included quantities Symbol Quantity
T:C Air temperature near the station
Tg:C Ground temperature near the station
Tgr:K/m Gradient of temperature
Tc:C Chimney temperature near the station
RH% Relative humidity
P:mb Air pressure near the station
Wind:m/s Wind velocity at standard height
Wmax:m/s Wind 5-minute maximum at standard height
Wind:deg Wind direction
Prec:mm/h Precipitation intensity
Illum:lux Solar illuminance
Rad:W/m2 Total insolation sun+sky
UVB:W/m2 UV radiation nm
Vis:km Visibility
DR:uSv/h Gamma radiation dose rate
NOx:ug/m3 Concentration of NOx (1 ppb is converted to 191 ug/cm3)
SO2:ug/m3 Concentration of SO2 (1 ppb is converted to 2.66 ug/cm3)
O3:ug/m3 Concentration of O3 (1 ppb is converted to 2.0 ug/cm3)

7. Symbol Unit Quantity B-noise cm-3 BSMA noise index
RC+ cm-3 Tahkuse index of spectra roughness for positive clusters
RN+ cm-3 Tahkuse index of spectra roughness for positive nanoparticles
RP+ cm-3 Tahkuse index of spectra roughness for positive particles
RC- cm-3 Tahkuse index of spectra roughness for negative clusters
RN- cm-3 Tahkuse index of spectra roughness for negative nanoparticles
RP- cm-3 Tahkuse index of spectra roughness for negative particles
NC+:cm-3 1/cm3 Concentration of positive cluster ions (Z > 0.5 cm2/Vs)
NC-:cm-3 1/cm3 Concentration of negative cluster ions (Z > 0.5 cm2/Vs)
NN+:cm-3 1/cm3 Concentration of positive nanometer ions (Z = cm2/Vs)
NN-:cm-3 1/cm3 Concentration of negative nanometer ions (Z = cm2/Vs)
NP+:cm-3 1/cm3 Concentration of positive particle ions (Z = cm2/Vs)
NP-:cm-3 1/cm3 Concentration of negative particle ions (Z = cm2/Vs)
ZC+cm cm2/Vs Average mobility of positive cluster ions (Z > 0.5 cm2/Vs)
ZC-cm cm2/Vs Average mobility of negative cluster ions (Z > 0.5 cm2/Vs)
L+:fS/m fS/m Positive conductivity of air
L-:fS/m fS/m Negative conductivity of air

8. z+01 cm-3 dN/d(log Z) of positive ions at Z = 2.74 cm2/Vs

9. d+01 cm-3 dN/d(log D) of positive ions at D = 0.49 nm

10. STATISTICS

11. Station Quantity Days Min Ave Max Sigma
Hyyt NC+:cm
Hyyt NC-:cm
Hyyt NN+:cm
Hyyt NN-:cm
Hyyt ZC+cm
Hyyt ZC-cm
Tartu NC+:cm
Tartu NC-:cm
Tartu NN+:cm
Tartu NN-:cm
Tartu ZC+cm
Tartu ZC-cm
Tahk NC+:cm
Tahk NC-:cm
Tahk NN+:cm
Tahk NN-:cm
Tahk NP+:cm
Tahk NP-:cm
Tahk ZC+cm
Tahk ZC-cm

12. Station Quantity Days Min Ave Max Sigma
Hyyt d
Hyyt d
Hyyt d
Hyyt d
Hyyt d
Hyyt d
Hyyt d
Hyyt d
Hyyt d
Hyyt d
Tartu d
Tartu d
Tartu d
Tartu d
Tartu d
Tartu d
Tartu d
Tartu d
Tartu d
Tartu d

13. Station Quantity Days Min Ave Max Sigma
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d
Tahk d

14. Average size distribution of negative ions

15. Average size distribution of negative ions

16. How to use?
Empty folders

17. time shift from UT, s (Helsinki winter time)
tolerable
percent of errors
how many headerlines
delimiter (tab)
base year
time pattern
time shift from UT, s (Helsinki winter time)
variable name in table
code of a
missing value
station
factor table/diary
quantity name in diary
time regime

18. Double- click
Result

20. time shift (s) (Helsinki winter time)
tolerable
percent of errors
how many headerlines
delimiter (tab)
base year
time pattern
time shift (s) (Helsinki winter time)
variable name in table
missing code
station
factor table/diary
quantity name in diary
time regime

21. Why DataDiurna?
A rectangular table can be conveniently processed and analyzed with common software like MS Excel (if the number of rows < 65536).
Different quantities are often saved with different time steps and many cells in a tectangular table will be empty. If a table holds a large complex dataset then it is composed mostly of the missing value codes like –999.
An alternative is a set of different tables joined into a database. Common database systems, e.g. MS Access, are optimized for the business applications. The time structure and the specifity of environmental measurements is not sufficiently exploited in these systems.
Another alternative is the diary format well known in hand-written measurement records. This alternative is applied in DataDiurna, which is optimized for managing of environmental measurements.
An additional advantage: the rules of DataDiurna force the user to include minimum explanations into the dataset, e.g. the data cannot be saved without indicating how much the time is shifted from UT.

22. A source table
An outlet table
A source table
Common diary
A source table
NB: DataDiurna is not a replacement for Excel 1) to manage large data: choose DataDiurna, 2) to analyze the tables: choose Statistica, Excel, etc.

23. Excerpt from a diary: Time regime Date Quantity Station
,Tartu.m,Rad:W/m2,hm,2,2,2,2,2,2,7,20,28,32,39,33,18,8,2,2,2,2,2,2,2,2,2,2
,Tartu.m,Wind:m/s,hm,3.8,3.8,3.9,4.4,4.9,4.9,4.4,2.4,1.8,1.8,2.5,2.1,2,1.9,0.7,1.7,2,1.5,2,1,1.2,1.1,1.3,1.1
,Hyyt,L+:fS/m,m10,9.6,9.1,8.7,8.9,9.1,9.4,10.2,9.8,9.6,10.3,10.9,11.4,11.6,12.2,12.5,11.8,11.9,12.2,12.7,12.5,12.1,11.9,11.3,11,11.6,11.5,11.3,10.9,11.2,11.5,11.5,11.3,11,10.7,11.1,11.5,11.1,11,,,,10.1,10.1,10,9.9,10.1,10.2,10.2,10,10.3,10.8,11.4,11.2,11.1,11.4,11.4,11.3,11.2,11.7,12,11.5,12.2,12.9,12.7,12.9,13.1,13.1,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,
,Hyyt,Rad:W/m2,m30,0,0,0,0,0,0,0,0,0,0,0,0,1,1,2,3,5,4,7,15,24,23,24,28,15,10,3,1,0,0,0,0,0,0,-1,0,0,0,0,0,0,-1,-1,0,0,0,0,0
Time regime
Date
Quantity
Station
Here are missing data

24. DataDiurna is omnivorous:
The time stamp and the numbers can be written as you data provider likes, e.g. following data rows, where the time stamp is highlighted, are considered equal
Time 13:25 Date ,-17,?,1.6e-19,
:25:00,23.1,-17,x, , e3
DOY= , measurements:
) 23.1;-17;-999;2,783,597e-4
The time is automatically converted into UT when importing a source table to the diary and to any time zone when exporting an output table from the diary.
The rows of the source table may be non-uniformly distributed in time: DataDiurna can interpolate and average the data during the import and export.
The code of missing value may be individual for every variable and may be numerical (e.g. –999) or non-numerical (e.g. ?).
etc... etc... etc...
Warning: The data manager DD2007T.exe is still a beta-version.

25. The basic version of the dataset FINESTION-2003-2006
was composed from a large number of source tables with different time steps from 10 minutes to 1 hour

26. The working version of the dataset FINESTION-2003-2006
is located in the computer of the user
Thus: Every user can create extended personal version of the dataset importing additional private data

27. NB: You should have 200 MB free on your hard disk.
How to get?
The DataDiurna data manager is free for unlimited distribution.
The dataset ATMEL2007A is free for unlimited distribution.
Both they can be retrieved from the page:
The dataset FINESTION is free but available only for our research group. It is not published in Internet today. Thus:
Ask the memory stick or the CD with the dataset and copy the folder DD into your own computer.
NB: You should have 200 MB free on your hard disk.

28. Thank you for choosing DataDiurna
Thank you for choosing DataDiurna .....hard to learn but easy to use when managing large complex datasets like FINESTION