Smart Initialization Mike Romberg

Sept , 2002Smart Initialization2 Lecture Smart Init: what, why, how? Configurability Structure of Smart Init Modules Override Procedures Examples of Algorithms Enhancements Laboratory

Sept , 2002Smart Initialization3 What is it? Derives sensible weather elements from model data. Creates the IFP databases from the D2D data. Works on models, not MOS

Sept , 2002Smart Initialization4 Why is it needed? Models do not provide sufficient spatial resolution. Models do not provide set of needed sensible weather elements.

Sept , 2002Smart Initialization5 Derived Surface T Raw model Data at 80km 2 meter Temperature Topography adjustments made Surface Temperature Sampled to 5km resolution

Sept , 2002Smart Initialization6 Characteristics of Smart Init Field tailorable algorithms Can add new weather elements Can add local models, if the netCDF files are in the correct D2D-style format Dependencies automatic Grids are generated as D2D data arrives

Sept , 2002Smart Initialization7 How it works ifpServer IFP Grid Databases Stores generated grids Destination Database 4 Determines what is needed to generate, gets/stores data 3 Smart init module Starts appropriate module 2 D2D netCDF Model Databases ifpserver monitors for data arrival Source Database 1

Sept , 2002Smart Initialization8 Configurability (serverConfig/localConfig) INITMODULES Maps algorithms modules to models INITMODULES = { “MesoEta” : [“MESOETAU”, “MESOETAS”], “Eta” : [“ETA”], “LAPS” : [“LAPS”] } INITMODULES = { “smart init module” : [‘D2D Source Databases’], } Two sources Different names

Sept , 2002Smart Initialization9 Configurability (serverConfig/localConfig) INITSKIPS Can skip certain model runs (INITSKIPS) INITSKIPS = { “RUC” : [1,2,4,5,7,8,10,11,13,14,16,17,19,20,22,23] } INITSKIPS = { “D2D ModelName” : [hour1,hour2,hour3 to skip]

Sept , 2002Smart Initialization10 Configurability (serverConfig/localConfig) Number of D2D Versions Not directly associated with smart init Does NOT override D2D purging. Default is 2 versions. D 2DDBVERSIONS = { “ETA”: 3, “NGM”: 1 } D2DDBVERSIONS = { “D2D model name” : numOfVersions, }

Sept , 2002Smart Initialization11 Configurability (serverConfig/localConfig) Smart init modules Can override and provide own Coded in Numerical Python One file per new model or changed model

Sept , 2002Smart Initialization12 Smart Init structure File locations etc/BASE, etc/SITE Override files through etc/SITE Uses Python inheritance Forecaster (etc/BASE/Init.py) EtaForecaster (/etc/BASE/Eta.py) serverConfig/localConfig INITMODULES import Forecaster MyEtaForecaster (/etc/SITE/MyEta.py) import EtaForecaster tells ifpServer what to run

Sept , 2002Smart Initialization13 Concept of Source/Destination Database – (needed in defining new smart init modules) Source database – D2D netCDF file Destination database – IFP ifpServer Smart Init D2D Database /data/fxa/… AWIPS D2D Source IFP Database (ifpServer) Destination

Sept , 2002Smart Initialization14 Format of Smart Init File Class Statement Levels (function) Calc Functions def calcT(arguments): Helper Functions (if needed) Main Declaration

Sept , 2002Smart Initialization15 Smart Init Inheritance Forecaster (base class) Init module EtaForecaster Eta module MRFForecaster MRF module NGMForecaster NGM module AVNForecaster AVN module

Sept , 2002Smart Initialization16 Format of smart init file from Init import * class EtaForecaster(Forecaster): def __init__(self): Forecaster.__init__(self, "ETA", "Eta") def levels(self): return ["MB1000", "MB950", “MB900", "MB850", "MB800", "MB750", "MB700","MB650","MB600“] Class Statement and Levels Import from next class Constructor for parent Available levels ETA is D2D source, Eta is IFP destination

Sept , 2002Smart Initialization17 Format of smart init file def calcSnowAmt(self, T, FzLevel, QPF, topo): m1 = less(T, 9) m2 = greater_equal(T, 30) snowr = T * snowr = where(m1, 20, snowr) snowr = where(m2, 0, snowr) snowamt = where(less_equal(FzLevel , topo * 3.048), snowr * QPF, 0) return snowamt Calc Functions Arguments Returns the calculated grid

Sept , 2002Smart Initialization18 Format of smart init file def linear(self, xmin, xmax, ymin, ymax, we): m = (ymax – ymin) / (xmax – xmin ) b = ymin – m * xmin return m * we + b Example of a helper function Arguments Returns the answer

Sept , 2002Smart Initialization19 Format of smart init file def main(): EtaForecaster().run() if __name__ == "__main__": main() Main declaration Needs to match class name

Sept , 2002Smart Initialization20 calc Arguments. WARNING: These are different than smart tool arguments Weather element in destination database TparmName Model topo in meters, from source database stopo High-res topo in meters, from destination database topo Cube of data, for the levels(), from source database rh_cparmName_c Single grid for ParmName/level – source database t_FHAG2parmName_level

Sept , 2002Smart Initialization21 calc Arguments (cont.) calc Arguments (cont.) ctime Time from the source database grid currently being calculated, as a time range tuple (startTime, endTime) in seconds since epoch mtime Time in the destination database grid currently being calculated, as a time range tuple (startTime, endTime), in seconds since epoch stime Number of seconds from the model basetime currently being calculated. (Seconds since the model analysis time.)

Sept , 2002Smart Initialization22 calc Arguments NOT the Fcst database as in smart tools Can use any combination of source/destination weather elements in the calcXXX functions. At present time, cannot easily have multiple input source databases

Sept , 2002Smart Initialization23 Overriding Algorithms Why override? I don’t like existing algorithms. I need additional sensible wx elements. ForecasterEtaForecaster def calcSnowAmt() MyEtaForecaster We will redefine the equation for SnowAmt

Sept , 2002Smart Initialization24 Steps to Override Create new smart initialization module. If you modify the ones in etc/BASE they will go away when you upgrade! Test out new smart initialization modules. Put it into routine operations.

Sept , 2002Smart Initialization25 Format of Override File Similar to existing smart init modules. Subtle differences, mainly due to inheritance.

Sept , 2002Smart Initialization26 Format of override file from Eta import * class MyEtaForecaster(EtaForecaster): def __init__(self): EtaForecaster.__init__(self) def calcSnowAmt(self, T, QPF): m2 = less_equal(T, 32) snowamt = where(m2, 10.0*QPF, 0) return snowamt def main(): MyEtaForecaster().run() if __name__ == “—main__”: main() Override Eta Override function Needs to match class name New class name, different inheritence structure

Sept , 2002Smart Initialization27 Testing your new module Can run from command line ifpInit –t modeltime algFile ifpInit –t modeltime –a algFile Model time in yyyymmdd_hhmm format. -a switch forces all grids to be generated ifpInit –t _1200 –a MyEta

Sept , 2002Smart Initialization28 Testing your new module Run from the command line to see progress and problems. Check the log files for errors …/data/logfiles/dateDirectory/EtaInit Check the data in the GFE.

Sept , 2002Smart Initialization29 Okay, it works, now what? Put it into operational use by: Redefining INITMODULES in localConfig serverConfig.INITMODULES[“MyEta”] = [“ETA”] del serverConfig.INITMODULES[“Eta”] Don’t forget to disable the original module!

Sept , 2002Smart Initialization30 Adding New Algorithms Define new algorithm module In localConfig, add new weather elements to destination database(s) before testing. Test Modify localConfig’s INITMODULES.

Sept , 2002Smart Initialization31 Adding Local Models netCDF files must have certain information in order for GFESuite to recognize them If displayable on d2d, probably close to ok. Need geographic information in netCDF. Inherit from Forecaster, look at Eta.py for example. Similar procedure: write module, test, modify INITMODULES. May need to modify D2DDIRS. Don’t forget - you can see these models on GFESuite without smart init!

Sept , 2002Smart Initialization32 Useful functions in Init.py def linear(self, xmin, xmax, ymin, ymax, we): -- linear interpolation def FtoK(self, t): -- convert F to Kelvin def KtoF(self, t): -- convert Kelvin to Farhenheit def esat(self, temp): -- saturation vapor pressure def self._empty –- returns grid of all zeros def self.levels() –- returns list of levels (MB500, MB450,) def self._minus –- return grid of all –1s def self.pres –- return list of levels as numbers (500, 450,)

Sept , 2002Smart Initialization33 Accessing Data Types Scalar T, simply a numerical grid Vector tuple, V[0] is magnitude, V[1] is direction Weather tuple, W[0] is grid, W[1] is key Key is set of Weather ugly strings Grid value indexed into key gives real value Smart Tool lecture/lab covered specifics of types.

Sept , 2002Smart Initialization34 Gotchas Conditional statements Unlike regular Python, both the true and the false are always executed in “where” statements Standard Python if x != 0: y = z / x else: y = 0.0 Numerical Python where(not_equal(x, 0.0), z/x, 0.0)) where(not_equal(x, 0.0), z/(x ), 0.00)

Sept , 2002Smart Initialization35 Future Enhancements Ability to access multiple sources Useful for model blending Syntax identical for numerical smart tools and smart initialization Fairly close now.

Sept , 2002Smart Initialization36 Examples of Algorithms - sky def calcSky(self, gh_c, rh_c, topo): # only use the first levels (up to MB600) gh_c = gh_c[:9,:,:] rh_c = rh_c[:9,:,:] rh_c = where(less(rh_c, 50), 0, rh_c) m1 = logical_and(greater(rh_c, 50), less(rh_c, 70)) m2 = logical_and(greater(rh_c, 70), less(rh_c, 85)) m3 = greater(rh_c, 85) skylayer = where(m1, self.linear(50, 70, 0, 25, rh_c), where(m2, self.linear(70, 100, 25, 100, rh_c), where(m3, 100, rh_c))) Trim down levels Rh<50%, then 0 Create masks, 50-70%, 70-85%, > 85% Calc sky cover at each level

Sept , 2002Smart Initialization37 Examples of Algorithms - sky skylayer = skylayer / 100 sky = skylayer[0] for i in xrange(1, skylayer.shape[0]): sky = sky + skylayer[i] - sky * skylayer[i] sky = clip(sky, 0, 1) sky = sky * 100 return sky Set to 0->1, from 0->100 Start at 1 st level Sum up the column So…what is wrong with this algorithm?

Sept , 2002Smart Initialization38 Laboratory Exercises Override the Eta algorithm for T Simply use 2m FHAG directly Enhance to use dry adiabatic rate for topo corrections Add a new model (UKMET) Add UKMET to localConfig Calculate QPF, Wind, and T for UKMET