September 16-18, 2014 NSSC, Lincoln, NE Part I

 Introductions Who What Where  Expectations  What does each bring to the table, Talents  What “one thing” do you want to know?

 Reductionism A complex system is nothing but the sum of its parts, and that an account of it can be reduced to accounts of individual constituents  Holism Natural systems and their properties should be viewed as wholes, not as collections of parts This often includes the view that systems somehow function as wholes and that their functioning cannot be fully understood solely in terms of their component parts

 Tuesday, Property Scripts Function Focus – component, component horizon… Robust – always runs (no exceptions) Universal – works as expected on all components Frame of Reference Methods of “rolling up”, when to use them Select projects to work on for the session

 Wednesday Review property scripts Evaluation styles and when to use them  Continuous variable  Text variable  Inverted  Limitation  Suitability  Developing Evaluations (Fuzzy Sets)  Boundaries, Splines

 Thursday Review Properties and Evaluations Base Rules Operators Weighting Hedges (the shape of the fuzzy set) Subrules Parent Rules Local Extreme Conditions (Al sat, LEP, gyp…) Model Techniques Testing and Validation

 "When we try to pick out anything by itself we find it hitched to everything else in the Universe."

 Does Reductionism well  Needs help on Holism

 Extract data Component Table, typically use as is  Slope, T, local phase, climate, taxonomic…  Summarize data Component Horizon Table, typically many attributes  Horizon Ksat, Horizon pH, Horizon clay… Component Horizon child tables, many attributes  Volumetric rock fragments, Unified, AASHTO…  Synthesize data to make a new attribute AWC to depth Pedotransfer functions  Subroutine DERIVE, O horizon thickness, layer thickness, restriction depth…

 SQL output for component data:  BASE TABLE component.  EXEC SQL  SELECT compname, slope_r rv, slope_l low, slope_h high  FROM component;.

 Component Horizon Table, SQL output for horizon data is an array unless aggregated  BASE TABLE component.   EXEC SQL  SELECT compname, hzdept_r, hzdepb_r, claytotal_l, claytotal_r, claytotal_h  FROM component, chorizon  WHERE JOIN component to chorizon;  SORT BY hzdept_r  AGGREGATE COLUMN hzdept_r NONE, hzdepb_r NONE, claytotal_l NONE, claytotal_r NONE, claytotal_h NONE.

 Interpretation Results for Batson (Component ID , DMU ID )  Property: CLAY CONTENT  Variable Name Value Type Value  compname CHARACTER "Batson"  hzdept_r NUMERIC 0, 15, 74, 89, 140  hzdepb_r NUMERIC 15, 74, 89, 140, 203  claytotal_l NUMERIC 2, 2, 2, 18, 15  claytotal_r NUMERIC 2.8, 5.5, 10.5, 23., 18.7  claytotal_h NUMERIC 8, 8, 15, 30, 30  coiid_key1 CHARACTER " "

 Interpretation Results for Berks (Component ID , DMU ID )  Property: CLAY CONTENT  Variable Name Value Type Value  compname CHARACTER "Berks"  hzdept_r NUMERIC 0, 3, 15, 48, 76  hzdepb_r NUMERIC 3, 15, 48, 76, 86  claytotal_l NUMERIC -NULL-, 7, 5, 5, -NULL-  claytotal_r NUMERIC -NULL-, 14, 18.5, 18.5, -NULL-  claytotal_h NUMERIC -NULL-, 27, 32, 32, -NULL-  coiid_key1 CHARACTER "811695"

 Write a property script that extracts the linear extensibility (high, rv, low) of components.

 BASE TABLE component.   EXEC SQL  SELECT compname, hzdept_r, hzdepb_r, claytotal_l, claytotal_r, claytotal_h  FROM component, chorizon  WHERE JOIN component to chorizon;  SORT BY hzdept_r  AGGREGATE COLUMN hzdept_r NONE, hzdepb_r NONE, claytotal_l FIRST, claytotal_r AVERAGE, claytotal_h LAST.

FIRST AVERAGE LAST

 BASE TABLE component.   EXEC SQL  SELECT compname, hzdept_r, hzdepb_r, claytotal_l, claytotal_r, claytotal_h  FROM component, chorizon  WHERE JOIN component to chorizon;  SORT BY hzdept_r  AGGREGATE COLUMN hzdept_r NONE, hzdepb_r NONE, claytotal_l NONE, claytotal_r NONE, claytotal_h NONE.  DEFINE lowARRAYMIN(claytotal_l).  DEFINE rvARRAYAVG(claytotal_r).  DEFINE highARRAYMAX(claytotal_h).

ARRAYMIN ARRAYAVG ARRAYMAX

 Interpretation Results for Berks (Component ID , DMU ID )  Property: CLAY CONTENT AVERAGE  Variable Name Value Type Value  compname CHARACTER "Berks"  hzdept_r NUMERIC0, 3, 15, 48, 76  hzdepb_r NUMERIC 3, 15, 48, 76, 86  claytotal_l NUMERIC -NULL-, 7, 5, 5, -NULL-  claytotal_r NUMERIC -NULL-, 14, 18.5, 18.5, -NULL-  claytotal_h NUMERIC -NULL-, 27, 32, 32, -NULL-  high NUMERIC  low NUMERIC  rv NUMERIC 17  coiid_key1 CHARACTER "811695"

 Touch your property script from Exercise 1 to return only the highest lep.

 We often do not want to know the properties of the entire profile  We may need to exclude some depths  We may want to exclude some features

 Say we want to know about the clay content of soils from 10 to 100cm  Could say “WHERE hzdept_r>10 and hzdepb_r<100”  What about the O horizon?  What about an R horizon?  What about gypsiferous layers?  What about a component that is one layer?

ROBUST UNIVERSAL  Certain combinations of data can cause unexpected problems: One thick layer plays havoc with depths set in the SQL Component restriction at the surface causes unequal dimensions Faulty aggregation can cause unequal dimensions Thapto-histic O horizons  Primary particles are meaningless in organic and gypsiferous horizons  “Conditional Null” supplies a value to allow a null horizon attribute to run under certain conditions  Soils dominated by amorphous materials do not behave like layer silicates (properties and rules need to be adjusted)

 My philosophy on the frame of reference is to extract all the horizon data using the EXEC SQL and control the frame of reference size and exclusions using DEFINED variables

O YESO NO  Soil will be used “AS IS”  No bulldozer involved  The O horizon is so thick that removal will negate the component, Histosols  Soil will be handled roughly  Bulldozer interpretations  Soil used as a material

RESTRICTION YES PROBLEMRESTRICTION NO PROBLEM  Roots cannot penetrate  Water cannot penetrate  Bulldozer can penetrate  Depends on restriction cementation

 BASE TABLE component.  EXEC SQL SELECT compname, hzdept_r, hzdepb_r, claytotal_l, claytotal_r, claytotal_h  FROM component, chorizon  WHERE JOIN component to chorizon;  SORT BY hzdept_r  AGGREGATE COLUMN hzdept_r none, hzdepb_r none, claytotal_l none, claytotal_r none, claytotal_h none.  # For a weighted average, you need to know the thickness of the layers within the frame of reference, in this case 0 to 200cm  DERIVE layer_thickness from rv using "NSSC Data":"LAYER THICKNESS IN RANGE; ABOVE VSTR RESTRICT BELOW O" (0,200).  # The material below the restriction is excluded  DERIVE depth from rv using "NSSC Data":"DEPTH TO FIRST STR/VSTR CEMENTED BELOW ORGANIC LAYER".  # The organic layer is excluded  DERIVE o_thickness from rv using "NSSC Pangaea":"THICKNESS OF SURFACE ORGANIC HORIZON".  # Find minimum of restriction depth and 200cm  DEFINE min_depth depth < 201 and not isnull(depth) ? depth : 200.  # Find the deepest layer in the frame of reference, min_depth sets the bottom.  DEFINE in_rangeisnull (hzdepb_r) ? hzdepb_r : (hzdepb_r - o_thickness <= min_depth ? 1 :  hzdepb_r - hzdept_r >= min_depth ? 1 : 0).   #When the restriction is at the surface, in_range is never 1, so the weighted average  #calculation fails due to unequal dimensions. If the restriction is at the surface,  #we make depth hzdept_r. We do not want to always make in_range 1, only when the restriction  #depth is 0. Default needs to be dimension 1.  ASSIGN layer_thicknessdepth == 0 ? hzdept_r : layer_thickness.  ASSIGN in_rangelayer_thickness < 0 ? 1 : in_range.  DEFINE default0*layer_thickness.  # The phrase: if hzdepb_r - o_thickness <=0 sets the top of the frame of reference, to lower, add a number higher than 0  DEFINE rv wtavg((if hzdepb_r - o_thickness <=0 THEN default ELSE lookup(1, in_range, claytotal_r)), layer_thickness).

 Interpretation Results for Batson (Component ID , DMU ID )  Property: WTD_AVG CLAY CONTENT 0-200cm OR ABOVE RESTRICTION  Variable Name Value Type Value  compname CHARACTER "Batson"  hzdept_rNUMERIC 0, 15, 74, 89, 140  hzdepb_r NUMERIC 15, 74, 89, 140, 203  claytotal_l NUMERIC 2, 2, 2, 18, 15  claytotal_r NUMERIC2.7, 5.5, 10.5, 23.1, 18.7  claytotal_h NUMERIC 8, 8, 15, 30, 30  layer_thickness NUMERIC 15, 59, 15, 51, 60  depth NUMERIC 300  o_thickness NUMERIC 0  min_depth NUMERIC 250  in_range NUMERIC 1, 1, 1, 1, 1  default NUMERIC 0, 0, 0, 0, 0  low NUMERIC 9.98  rv NUMERIC  high NUMERIC  coiid_key1 CHARACTER " "

 Interpretation Results for Berks (Component ID , DMU ID )  Property: CLAY CONTENT WTD_AVG 0-200cm OR ABOVE RESTRICTION  Variable Name Value Type Value  compname CHARACTER "Berks"  hzdept_r NUMERIC 0, 3, 15, 48, 76  hzdepb_r NUMERIC 3, 15, 48, 76, 86  claytotal_l NUMERIC -NULL-, 7, 5, 5, -NULL-  claytotal_r NUMERIC -NULL-, 14, 18.5, 18.5, -NULL-  claytotal_h NUMERIC -NULL-, 27, 32, 32, -NULL-  layer_thickness NUMERIC 0, 12, 33, 28, 0  depth NUMERIC 73  o_thickness NUMERIC 3  min_depth NUMERIC 73  in_range NUMERIC 1, 1, 1, 1, 0  default NUMERIC 0, 0, 0, 0, 0  low NUMERIC  rv NUMERIC  high NUMERIC  coiid_key1 CHARACTER "811695"

 Dominant condition…………  Thickest layer………………..  Arraymax …………  Arraymin…………………….  Average………………………  Lookup……………………….  Regroup ……………………..  Weighted Average …………  Text variables (Unified)  Not used much lately  Most transmissive or clayeyest layer  Least transmissive layer  Generally would use weighted average  Meets a condition  Aggregate child of chorizon table  Soil will be mixed and used as a material

 Using your prior sql  Now, extract from 30 to 120 cm  Exclude the O horizon, if any  Stop at a restrictive layer  Report the highest rv LEP

 Let us dissect a fairly complex property script to see what the various parts do  UNIFIED DOMINANT CONDITION IN DEPTH CM on NSSC Pangaea  Let us look at a Gilpin component: COIID =

BASE TABLE component. # Get Unified classes in all horizons EXEC SQL SELECT hzdept_r, hzdepb_r, unifiedcl, chunifiediid, chunified.rvindicator, chiid FROM component, chorizon, chunified WHERE join component to chorizon AND join chorizon to chunified; SORT BY hzdept_r, chunified.rvindicator DESC, chunifiediid ASC AGGREGATE COLUMN hzdept_r NONE, hzdepb_r NONE, chunifiediid NONE, chiid NONE, chunified.rvindicator NONE, unifiedcl NONE. DEFINE unified_nameCODENAME(unifiedcl). # Select first unified in each horizon DEFINE prevchiid ARRAYSHIFT(chiid, 1). DEFINE firstunified LOOKUP(0, chiid==prevchiid, unified_name). DEFINE hzdept LOOKUP(0, chiid==prevchiid, hzdept_r). DEFINE hzdepb LOOKUP(0, chiid==prevchiid, hzdepb_r).

# Find thickness of each horizon in cm range # Determine the LAYER THICKNESS IN RANGE; ABOVE A RESTRICTIVE LAYER. define top_limit 25. define bottom_limit 100. DERIVE depth2rest from rv using "NSSC Pangaea":"DEPTH TO FIRST RESTRICTIVE LAYER". DEFINE depth depth2rest < bottom_limit ? depth2rest : bottom_limit. DEFINE top_in_range hzdept < top_limit and not isnull(hzdept) ? top_limit: hzdept > depth ? depth : hzdept. DEFINE bottom_in_range hzdepb < top_limit and not isnull(hzdepb) ? top_limit: hzdepb > depth ? depth : hzdepb. DEFINE layer_thickness bottom_in_range - top_in_range. # Reduce layer_thickness and unifiedcl to one value per horizon. # by selecting the first one for each horizon. ASSIGN layer_thickness REGROUP layer_thickness by hzdept aggregate first. DEFINE unified_hor REGROUP firstunified by hzdept aggregate first.

top_limitNUMERIC 25 bottom_limit NUMERIC 100 depth2rest NUMERIC 77 depth NUMERIC 77 top_in_range NUMERIC 25, 25, 25, 54 bottom_in_range NUMERIC 25, 25, 54, 77 layer_thickness NUMERIC 0, 0, 29, 23 unified_hor CHARACTER "pt", "cl", "gc"

#Find the sum of the horizon thicknesses having each Unified condition. DEFINE sum_thickREGROUP layer_thickness by unified_hor aggregate sum. ASSIGN unified_horREGROUP unified_hor by unified_hor aggregate first. #Find the highest sum of horizon thickness sums. DEFINE max_hthickARRAYMAX(sum_thick). #Find the Unified associated with the highest sum of horizon thicknesses DEFINE max_unifiedLOOKUP(max_hthick, sum_thick,unified_hor). #Tie-breaker takes most restrictive of the dominant conditions, if more than one. ASSIGN max_unifiedARRAYMAX(max_unified). define rv max_unified.

 sum_thick NUMERIC0, 29, 23  max_hthick NUMERIC 29  max_unified CHARACTER "cl"  rv CHARACTER "cl"  coiid_key1 CHARACTER " "

 ECEC HORIZON DEPTH <30cm define rv ARRAYMAX((if hzdepb_r - o_thickness <=0 THEN default ELSE LOOKUP(1, in_range, exchange_r))).  WTD_AVG PASSING NO10 SIEVE cm OR ABOVE RESTRICTION define rv WTAVG((if hzdepb_r - o_thickness <=25 THEN default ELSE LOOKUP(1, in_range, sieveno10_r)), layer_thickness).  UNIFIED (ORGANIC) IN LAYER >=15CM THICK IN DEPTH 0-90CM define unified_lookup LOOKUP(min_thickness, layer_thickness, unified).

 REGROUP used to find dominant condition in a profile  USDA TEXTURE CLASS DOMINANT CONDITION IN DEPTH 0-100CM DEFINE sum_thickREGROUP layer_thickness by texture_name aggregate sum. ASSIGN texture_nameREGROUP texture_name by texture_name aggregate first.  REGROUP used to aggregate a component horizon child table  WTD_AVG VOLUME FRAGMENTS >2mm 0-150cm OR ABOVE RESTRICTION ASSIGN fragvol_rREGROUP fragvol_r by chiid aggregate sum. ASSIGN hzdept_rREGROUP hzdept_r by chiid aggregate first. ASSIGN hzdepb_rREGROUP hzdepb_r by chiid aggregate first. ASSIGN dbthirdbar_rREGROUP dbthirdbar_r by chiid aggregate first. ASSIGN chiidREGROUP chiid by chiid aggregate first.

 # Get low, high, and rv surface cover percentage of rock fragments with an rv >=75mm.  base table component.  exec sql select sfragcov_l, sfragcov_h, sfragcov_r  from component, outer cosurffrags  where join component to cosurffrags  and sfragsize_r >=75;  aggregate column sfragcov_l none, sfragcov_h none,  sfragcov_r none.  #Sum the percentages.  define low arraysum(sfragcov_l).  define high arraysum(sfragcov_h).  define rv arraysum(sfragcov_r).  # Get low, high, and rv surface cover percentage of rock fragments with an rv >=75mm.  base table component.  exec sql select sfragcov_l, sfragcov_h, sfragcov_r  from component left outer join cosurffrags by default and sfragsize_r >= 75;  aggregate column sfragcov_l none, sfragcov_h none,  sfragcov_r none.  #Sum the percentages.  define low arraysum(sfragcov_l).  define high arraysum(sfragcov_h).  define rv arraysum(sfragcov_r).

 MEQ 0-150cm OR RESTRICTION – Volumetric exchange capacity  GROWING DEGREE DAYS ESTIMATOR (BASE 50, SINE)  LOG KSAT MINIMUM LEP 0 TO 50cm ABOVE RESTRICTIVE LAYER  LIQUEFACTION SUSCEPTIBILITY  PEDON IRON OXIDES INDEX  PI AWC * Root Index 200cm (MO)

 The SQL syntax in almost all current property scripts is a legacy of the UNIX INFORMIX system from which NASIS originated  Since NASIS 6.x, Microsoft SQL Server has been used as the NASIS SQL  SQL Server allows more modern syntax which should allow property scripts to run faster