1. Intersite Coordination (ISC) Mark Mathewson

2. Sept 24-27, 2002Intersite Coordination2 Overview Motivation for ISC How ISC Works GFESuite ISC Capabilities Including several lab exercises Diagnosing ISC Problems Discussion

3. Sept 24-27, 2002Intersite Coordination3 Motivation for ISC

4. Sept 24-27, 2002Intersite Coordination4 Motivation for ISC Obvious – want your forecast to be consistent with your neighbors But previously, it wasn’t obvious if your forecast was different. Hard to compare words NDFD (National Digital Forecast Database) changes all of this.

5. Sept 24-27, 2002Intersite Coordination5 Motivation for ISC

6. Sept 24-27, 2002Intersite Coordination6 How ISC Works

7. Sept 24-27, 2002Intersite Coordination7 How ISC Works Sending ISC Grids is triggered by the forecaster in a variety of ways. Grids are sent over the WAN to your neighbors. Grids are mosaic’d with other WFO ISC grids and stored in the ISC database.

8. Sept 24-27, 2002Intersite Coordination8 ISC Internals

9. Sept 24-27, 2002Intersite Coordination9 Spatial Mosaic

10. Sept 24-27, 2002Intersite Coordination10 Temporal Mosaic

11. Sept 24-27, 2002Intersite Coordination11 Sample ISC Database Central Cluster NDFD EAX Perspective

12. Sept 24-27, 2002Intersite Coordination12 Mosaic Technique Characteristics There may be more than one ISC grid for one of your Fcst grids. Max/MinT across time zones are interesting. There may be “blank” areas if not all adjacent WFO grids are available.

13. Sept 24-27, 2002Intersite Coordination13 NDFD and Intersite Coordination Related – but not the same. WFO Fcst WAN Intersite Coordination Grids Official NDFD Central Server NDFD Customers WAN NDFD Grids

14. Sept 24-27, 2002Intersite Coordination14 GFESuite ISC Capabilities

15. Sept 24-27, 2002Intersite Coordination15 GFESuite ISC Capabilities Sending Grids Displaying ISC Mosaic Displaying ISC and Fcst grids together Discrepancy Tools IFPS10 ISC and IFPS11 ISC is different. Improved dialogs and use. IFPS12 has more changes, including ISC for combined public and marine areas

16. Sept 24-27, 2002Intersite Coordination16 Send ISC Dialog Select which weather elements Select time period -- allows single grid Select which sites to transmit your grids to

17. Sept 24-27, 2002Intersite Coordination17 Publish Dialog Can select weather element Can select time period Can choose to send ISC or not All sites always

18. Sept 24-27, 2002Intersite Coordination18 Save Weather Element Dialog Sends to all sites Sends just the grids that are to be saved In IFPS12 In IFPS11, all grids from weather element sent

19. Sept 24-27, 2002Intersite Coordination19 sendISC sendISC program can be used to send ISC grids. Can define: Time range over which grids are sent Weather Elements Sites, including sites that are NOT defined in ISC_Sites. Official database or Fcst database

20. Sept 24-27, 2002Intersite Coordination20 Displaying ISC Grids Load ISC database from Weather Element Browser

21. Sept 24-27, 2002Intersite Coordination21 Lab Exercise 1 Edit today’s Maximum Temperature grid. Send it to all your neighbors. View the ISC mosaic by loading the ISC database through Weather Element Browser Watch as other sites appear/change Status bar indicates receipt of data Your data will also appear

22. Sept 24-27, 2002Intersite Coordination22 GFESuite – “Show ISC Mode” Available from the toolbar Merges Fcst database over your CWA with ISC database over remainder of grid

23. Sept 24-27, 2002Intersite Coordination23 GFESuite – “Show ISC Mode” Entire grid from EAX OAX EAX SGF TOP ICT

24. Sept 24-27, 2002Intersite Coordination24 GFESuite – “Show ISC Mode” Samples show: ISC Update Time ISC Site Identifier “P” for data from published database Data values Markers show: Similar to samples, but no data values Useful for anchoring to monitor ISC

25. Sept 24-27, 2002Intersite Coordination25 GFESuite – “Show ISC Mode” Data Value Site ID Update Time Official DB SAMPLEs MARKERs

26. Sept 24-27, 2002Intersite Coordination26 GFESuite – “Show ISC Mode” Samples/Markers Site ID, UpdateTime, “Publish” flag can be turned off Markers Set are calculated with ifpServer edit areas on startup Suggest you remove extra ones, move existing ones the way you like as “SITE”.

27. Sept 24-27, 2002Intersite Coordination27 Grid Data History

28. Sept 24-27, 2002Intersite Coordination28 Lab Exercise - 2 Use tomorrow’s Maximum Temperature Modify it, choose whether to publish or just send ISC Experiment with “Show ISC Mode” Set up “markers” – MB3 with sample tool Explore turning on/off sample modes and marker modes through Maps->Samples and Maps->Markers menus Try Time-Stepping – observe behavior with/without “Show ISC Mode” on.

29. Sept 24-27, 2002Intersite Coordination29 “Show ISC Mode” effect on tools You already see the change in the Sample Tool. When “Show ISC Mode” is on: Pickup value tool pickups Fcst or ISC data Move/Copy pickups Fcst or ISC data Contour Tool pickups Fcst or ISC data Pencil Tool pickups Fcst or ISC data

30. Sept 24-27, 2002Intersite Coordination30 “Show ISC Mode”… Smart Tools are not affected by ISC mode. If you want to access ISC data from smart tools, “T” is always Fcst, no ISC data. Use getGrids() to access ISC data. Temporal Editor – ignores ISC mode Grid Manager – doesn’t show ISC mode Spatial Editor Legends – do change

31. Sept 24-27, 2002Intersite Coordination31 Lab Exercise - 3 Experiment with “Show ISC Mode” on and off with: Sample Tool Move/Copy Tool Pencil Tool Contour Tool Smart Tools – such as Assign Value and Smooth

32. Sept 24-27, 2002Intersite Coordination32 Discrepancy Tools Discrepancy Tools are simply smart tools. You can extend and write your own. Allow you to analyze all grids quickly. Highlights grids in grid manager if thresholds exceeded Can look at difference grid Can look at edit area that exceeds thresholds Uses discrepancy value, or discrepancy grid.

33. Sept 24-27, 2002Intersite Coordination33 Discrepancy Tools Procedure ISC_Discrepancies Tools Show_ISC_Highlights Show_ISC_Grid Show_ISC_Area Edit Area ISC_Tool_Area

34. Sept 24-27, 2002Intersite Coordination34 ISC_Tool_Area Calculated when ifpServer starts. Tools use this area for calculations. CWA boundary

35. Sept 24-27, 2002Intersite Coordination35 Discrepancy Algorithms Four neighboring points are examined Discrepancy Value or Grid used to determine significant discrepancies Uses “Composite” Grid (Fcst + ISC) Neighboring Points

36. Sept 24-27, 2002Intersite Coordination36 Discrepancy Algorithms ISC-Fcst Difference Subtracts two fields Discrepancy Value or Grid used to determine significant discrepancies Fcst ISC EAX OAX EAXOAX

37. Sept 24-27, 2002Intersite Coordination37 Discrepancy Value and Grid Can set discrepancy value for each weather element.

38. Sept 24-27, 2002Intersite Coordination38 Discrepancy Grid Can create discrepancy grids. Allows varying values per gridpoint. Use normal editing tools to create it.

39. Sept 24-27, 2002Intersite Coordination39 ISC_Discrepancies Dialog Time Period Weather Elements

40. Sept 24-27, 2002Intersite Coordination40 Discrepancy Highlighting Grids highlighted in red indicate that thresholds were exceeded.

41. Sept 24-27, 2002Intersite Coordination41 Difference Grid Original Fcst/ISC Grids 83 56 Difference Grid Fcst – ISC grid = TDiff

42. Sept 24-27, 2002Intersite Coordination42 Discrepancy Edit Area Edit Area, Exceeding thresholds

43. Sept 24-27, 2002Intersite Coordination43 Lab Exercise - 4 Edit MaxT, Save it Send grids Run ISC_Discrepancy tool Grids are highlighted Look at discrepancy edit area Look at discrepancy difference grid Reset the highlights Experiment with various options. Try both algorithms.

44. Sept 24-27, 2002Intersite Coordination44 ISC (current) Limitations Supplied tools don’t handle: Wind direction Weather ISC area based on shapefiles CWAs only through IFPS11 Marine and CWAs IFPS12+

45. Sept 24-27, 2002Intersite Coordination45 ISC (current) Limitations Remember, ISC display and editing tools (non-smart), are hard-coded. Need your input to improve them. However, there really are no other tool limitations, since smart tools are used. You can write your own algorithms/techniques.

46. Sept 24-27, 2002Intersite Coordination46 Diagnosing ISC Problems

47. Sept 24-27, 2002Intersite Coordination47 Diagnosing ISC Problems ISC has a fairly complicated processing path. Numerous log files are used to diagnose problems. Log Files are documented.

48. Sept 24-27, 2002Intersite Coordination48 Common ISC Problems I can’t see other site’s ISC grids. I can’t see my own ISC grids. Other sites can’t see my ISC grids.

49. Sept 24-27, 2002Intersite Coordination49 Log Files – the key to diagnosing problems

50. Sept 24-27, 2002Intersite Coordination50 Log Files – the key to diagnosing problems Receiving Grids for iscDataRec 17:13:52.991 EVENT: received: /data/adapt/GFESuite/iscDataRec/FSLC-339357.001 8593 17:13:53.165 EVENT: Sent to: rpp-fslc 98000000 for iscMosaic 00:20:00.348 iscMosaic.py 791 EVENT: iscMosaic Starting 00:20:00.528 iscMosaic.py 665 EVENT: Processing file=/home/awipsusr/release/products/ISC/iscWO7Sgf 00:20:00.533 iscMosaic.py 665 EVENT: Input file size: 275092 iscMosaic.py 665 VERBOSE: Processing T_SFC #Grids= 23 Site= GID00:20:56.413 iscMosaic.py 791 EVENT: iscMosaic Finished Traceback with problem if fails, starting in IFPS11

51. Sept 24-27, 2002Intersite Coordination51 Log Files – the key to diagnosing problems Sending Grids for ifpnetCDF: 18:36:38.191 ifpnetCDF.py 918 EVENT: ifpnetCDF Starting 18:36:38.734 ifpnetCDF.py 659 EVENT: Saved 10 T grids 18:36:39.234 ifpnetCDF.py 659 EVENT: Saved 9 Td grids 18:36:44.632 ifpnetCDF.py 983 EVENT: ifpnetCDF Finished For iscExtract: 16:53:50.778 EVENT: Sent: size: 8594,wctime: 15.47704494,cputime: 0.14,sites: ['FSLC'],time: -s 19700101_0000 -e 20380119_0314,parms: ['T'] For iscd: 17:15:02.814 EVENT: sending: fname: @7955.1.isc.gz,size: 8591,to: ['FSLC']

52. Sept 24-27, 2002Intersite Coordination52 Collaboration Techniques Is ISC really enough? Coordination of grids may come too late in the forecast process. NDFD Clusters have been experimenting with: FX/C or FX/Collaborate

53. Sept 24-27, 2002Intersite Coordination53 Discussion

54. Sept 24-27, 2002Intersite Coordination54 Discussion Shortcomings of current tools/methods Tools which could help? ISC Consensus Tool Methodology which could help?