1. A Physical Interpretation of Difference Variances Victor S. Reinhardt 2007 Joint Meeting of the European Time and Frequency Forum (EFTF) and the IEEE International Frequency Control Symposium (IEEE-FCS) Geneva, Switzerland May 29 - June 1, 2007 Copyright 2007 Victor S. Reinhardt--Rights to copy material is granted so long as a source reference is listed on each page, section, or graphic utilized.

2. Page 2 A Physical Interpretation of Difference Variances V.S. Reinhardt A Physical Interpretation of Difference Variances of the Time Error x(t) Most variances of x(t) used in the industry can be written as M th order difference variances  x,M 2 or their sample statistics Most variances of x(t) used in the industry can be written as M th order difference variances  x,M 2 or their sample statistics  E{..} more appropriate for this paper than  E{..} more appropriate for this paper than M defined so all  x,M 2 equal for uncorrelated white noise M defined so all  x,M 2 equal for uncorrelated white noise But not all  x,M 2 equal for neg-  (neg power law) noise But not all  x,M 2 equal for neg-  (neg power law) noise  Then which order M of  x,M 2 most appropriate to measure random residual error in a given problem?  Paper will answer question by providing a rationale for choosing M FSCS2007 figs.doc 9.05” width  x,M 2 = M -1 E{[  (  ) M x(t)] 2 }  (  )x(t)  x(t+  ) - x(t)

3. Page 3 A Physical Interpretation of Difference Variances V.S. Reinhardt To Address Question Must Investigate Behavior of x(t) Over Long Interval T x(t) generally separated into causal environmental & aging factors plus a random residual error x(t) generally separated into causal environmental & aging factors plus a random residual error  Ignoring spurs, etc., but won’t effect over-all conclusions Environmental sensitivity to temp, power supply, etc. Environmental sensitivity to temp, power supply, etc.  Environmental coefficients can be determined from data over time <<T where other factors small  Will assume x(t) has environment already removed & removal has little impact on long term behavior of x(t) Aging – Model as (M-1) th order polynomial Aging – Model as (M-1) th order polynomial  Will assume A & env coeffs not functions of time over T  Must determine A over ~T for accurate determination  Thus long term random errors will impact aging determination Residual & variance Residual x r,M (t) = x(t) – x a,M (t,A) & variance  r,M (t) 2 = E{x r,M (t) 2 }  C  Coefficients in x a,M (t,A) are determined by fit to data over T  Behavior of aging fit will lead us to our rationale for choosing  x,M 2 T x(t) w/o Env t Tot Aging x a,M (t,A) Residual x r,M (t) toto

4. Page 4 A Physical Interpretation of Difference Variances V.S. Reinhardt Behavior of Aging Fit When Neg-  Noise is Present Model x(t) over T as Model x(t) over T as  True aging x a,M (A (0),t) plus random power law noise L x (f)  f  Can use least squares fit (LSQF) to determine A from N samples x n over T Can use least squares fit (LSQF) to determine A from N samples x n over T   r,M = RMS residual from estimated x a,M (A,t)  = Est – true aging   x a,M = x a,M (t,A) – x a,M (t,A (o) ) = Est – true aging For white-x (f 0 ) noise For white-x (f 0 ) noise   r,M 2 &  x a,M  0 as N   (BW large) However for neg-  noise (   -2) However for neg-  noise (   -2)  f -1 noise more like white-x noise   r,M 2 &  x a,M do not  0 as N    True aging can’t be recovered below level of long term noise (in single meas over T)  Must ensemble average over many data sets to reduce error further  Problem for real frequency sources unless aging model valid over t >> T  Each device has unique aging coefficients long term error-1.xls f 0 Noise  r,M approx the same for all  f -2 Noise Est Aging  x a,M True Aging xnxn t  f -4 Noise x r,M

5. Page 5 A Physical Interpretation of Difference Variances V.S. Reinhardt Further Properties of Aging Solutions for Neg-  Noise Numerical LSQF solutions for M=2 Numerical LSQF solutions for M=2  T = 4K points generated of x(t) = x a,2 (A (0),t) + f  noise  Estimated A determined by LSQF using N = 2 to 2K samples spaced over T  RMS  r,2 &  x a,2 averaged over all 4K points & LSQF for all possible start offsets   r,2 relatively insensitive to # of samples in LSQF (esp for neg-  )  For neg-  noise: RMS  x a,2 > RMS  r,2  Expected for correlated noise Theoretical LSQF Solutions Theoretical LSQF Solutions  Can write formal solution in terms of R x (  ) & Green’s functions  Doesn’t lead us to our goal of obtaining a physical interpretation for  x,M 2  Solutions increase in complexity with M  A simpler approach that will lead us to our goal is next topic Errors in LSQF for x a,2 (A,t) = a 0 + a 1 t 0 deg of freedom (2 Samples in LSQF) Samples in LSQF  1101001K RMS  r,2 f 0 Noise 2 1 0 2 1 0 f -2 Noise RMS  x a,2 2 1 0 f -4 Noise RMS  x a,2 RMS  r,2

6. Page 6 A Physical Interpretation of Difference Variances V.S. Reinhardt A Simpler Approach – The Zero Degree of Freedom (N=M) Approximation for  r,M 2 Can use fact that  r,M 2 rel insensitive to # points to simplify calculation of  r,M 2 Can use fact that  r,M 2 rel insensitive to # points to simplify calculation of  r,M 2 Zero degree of freedom solution (N=M) Zero degree of freedom solution (N=M)  M+1 x n (n = 0 to M) spaced by  over T  Calc A from only M points (exclude n=p)  Can solve for  r,M 2 at n=p without actually calculating x a,M (A,t)  Generalization of Newton’s Forward Difference Formula Physical interpretation of  x,M (  ) 2  Approximate measure of  r,M 2 for any # of points in LSQF   Rationale for choosing M in  x,M (  ) 2  Match to M in x a,M (t,A) when poly aging removal specified in problem   When poly aging not specified  Match  x,M (  ) 2 to lowest order x a,M (t,A) that fits actual aging over T (Occam’s razor) x r,M (t o +p  ) =  (  ) M x(t o )/C(M, p)  r,M 2 (t o +p  ) = M  x,M 2 (  )/C(M,p) 2 xMxM x a,M    x0x0    T = M  xpxp x r,M  x n  x(t o + n  ) n = 0 to M

7. Page 7 A Physical Interpretation of Difference Variances V.S. Reinhardt Interpreting  x,M 2 as a Measure of Residual Error After Aging Removal FSCS2007 figs.doc 9” width x r,1 =  (  ) 1 x 0 x0x0 x1x1    xaxa xrxr x r,0 =  (  ) 0 x 0  x 0 -  0 th order aging (time offset) removal 1 st order aging (time & freq offset) removal  r,1 2 = TIErms 2  r,0 2 = Standard No removal  r,2 2 = 0.5  2  y 2 (  ) x r,2 = 0.5  (  ) 2 x 0  x0x0  x1x1  x2x2  xaxa p  0.5M for min  r,M 2 x0x0   xrxr

8. Page 8 A Physical Interpretation of Difference Variances V.S. Reinhardt Can Extend the Definition to Finite Sample Statistics Overlapping statistic (N  , finite data length T d = Ndt > M  ) Overlapping statistic (N  , finite data length T d = Ndt > M  ) Modified statistic Modified statistic Can write similar equation for total statistic Can write similar equation for total statistic These are statistics of  r,M 2 when x a,M (A,t) fit to data over T (not over T d )   Implicit solution for A changes as t (t o  t) is slid over T d - M    Aging model needs only be valid representation over T (not T d ) Finite time average of M -1 [  (  ) M x(t)] 2 x(t) = time average over  Solution for A changes with t xMxM x a,M    x0x0    T = M  xpxp x r,M  tt+T

9. Page 9 A Physical Interpretation of Difference Variances V.S. Reinhardt Spectral Properties of  x,M 2 Can write spectral integral for  x,M 2 as Can write spectral integral for  x,M 2 as  L x (f) = SSB PSD of x(t)  K x,M (f) = x-kernel for  x,M 2 |H s (f)| 2 = System response function (See FCS 2006 paper) |H s (f)| 2 = System response function (See FCS 2006 paper)  Spectral properties specific to system under consideration  Have shown H s (f) often has k th order zero at f=0  Aids in the convergence of low order  x,M 2 for neg-  noise for f<<1  Well-known property of  x,M 2 kernels K x,M (f)  f 2M for f<<1  Well-known property of  x,M 2 kernels  x-kernels of statistics have same f 1  Statistic of  x,0 2 (sample variance) is special case because of  Removing x a,M (t,A) from x(t) causes residual noise to be HP filtered!! Removing x a,M (t,A) from x(t) causes residual noise to be HP filtered!!  Consequence of interpretation of  x,M 2 as approx measure of  r,M 2  Can interpret x a,M (t,A) as information removed from x(t) as well as aging   Thus removing information from x(t) by fitting to data causes residual noise to be HP filtered |H s (f)| 2 Replaces f h

10. Page 10 A Physical Interpretation of Difference Variances V.S. Reinhardt Summary of Consequences of Interpretation of  x,M 2 as measure of  r,M 2 Removing aging or information from data turns a seeming standard variance into a higher order variance Removing aging or information from data turns a seeming standard variance into a higher order variance  Removing x a,M (t,A) from data acts as a highpass filter on the noise  HP filtering order depends on complexity of info extracted (as measured by poly order)  Based on assumption that x-kernel of general  r,M 2 will have same f 2M behavior for f<<1  Need to investigate behavior of exact x-kernels of  r,M 2 further Order of  x,M 2 should match order of x a,M (t,A) that is specified by problem or most appropriately fits the data over T Order of  x,M 2 should match order of x a,M (t,A) that is specified by problem or most appropriately fits the data over T  Aging will contaminate  x,M 2 if x a,M (t,A) doesn’t properly represent the actual aging over T  Explains sensitivity of Allan based variances to freq drift and insensitivity of Hadamard based variances Should not arbitrarily change the order of  x,M 2 to avoid a divergence problem due to neg-  noise Should not arbitrarily change the order of  x,M 2 to avoid a divergence problem due to neg-  noise  Order defined by spec or by best match to actual aging behavior over T  Not free to change without changing problem addressed  Must find other ways of dealing with such divergences

11. Page 11 A Physical Interpretation of Difference Variances V.S. Reinhardt Dealing with Divergences when Neg-  Noise is Present Well-known that  x,0 2 &  x,1 2 can diverge for neg-  noise Well-known that  x,0 2 &  x,1 2 can diverge for neg-  noise  It has been argued that these variances should not be used  But  x,0 2 &  x,1 2 measure  r,M 2 for no aging & time-offset removal  Cannot change these variances (if want  x,M 2 to be a measure of  r,M 2 ) without changing system spec or problem statement  Example: Coherent synthesizer  Freq offset error is a problem   x,2 2 not appropriate because takes out ave freq error over T Will show such divergences indicate real design, spec, or analysis problems in presence of neg-  noise Will show such divergences indicate real design, spec, or analysis problems in presence of neg-  noise  Appropriate response is to fix these problems–not to arbitrarily change the variance Will discuss examples of how to deal with divergences Will discuss examples of how to deal with divergences  Non-essential divergence—Requires only analysis change  Didn’t use correct H s (f) (or only f h ) or wrong variance for problem  Essential divergence—Requires design or spec change  Design change: Change hardware  change H S (f)  Spec change: Change problem to be addressed and/or add side conditions

12. Page 12 A Physical Interpretation of Difference Variances V.S. Reinhardt An Essential Divergence 1 st order PLL with f -3 Noise Standard variance  x,0 2 diverges in linear PLL model Standard variance  x,0 2 diverges in linear PLL model  1 st order PLL: |H s (f)| 2  f 2 (f<<1)  K x,0 (f) = 1  |H s (f)| 2 K x,0 (f)L x (f)  f -1 (f<<1) Divergence is indicator of a physical problem Divergence is indicator of a physical problem  Divergence in linear model indicator of cycle slips in actual PLL  Arbitrarily changing order of variance doesn’t fix cycle slip problem Change design to 2 nd order PLL Change design to 2 nd order PLL  Eliminates cycle slips  |H s (f)| 2  f 4 (f<<1)  System changed not variance Allow slips (don’t change design) but change spec Allow slips (don’t change design) but change spec  Specify mean time to cycle slip  Specify that variance be evaluated using sample variance over finite T excluding data containing cycle slips  Sample variance has x-kernel  f 2 (f<<1) for finite T  Variance change based on spec change Sample Variance  -detector x or  Cycle Slips

13. Page 13 A Physical Interpretation of Difference Variances V.S. Reinhardt A Non-Essential Divergence Radar or Ranging System with f -3 Noise Characterized by “delay” system response |H s (f)| 2  f 2 (f<<1) Characterized by “delay” system response |H s (f)| 2  f 2 (f<<1)   ,o 2 often used to specify  -error residual  But  ,o 2 diverges for f -3 noise Heuristic radar solution: Use BP variance Heuristic radar solution: Use BP variance  T c = Coherent data processing interval for extracting radar information  No rigorous justification for using  ,bp 2 Rigorous solution: Proper error measure is sample variance over T d (not standard variance) Rigorous solution: Proper error measure is sample variance over T d (not standard variance)  Higher order variance if more than just one parameter (a 0 ) is extracted from data  x-kernel  f 2 or greater so no divergence problem for correct variance  Problem was that incorrect variance was used ~ Coherent Radar x D dd  (t)  (t-  d )  Process

14. Page 14 A Physical Interpretation of Difference Variances V.S. Reinhardt Summary and Conclusions  x,M 2 can be interpreted as an approximate measure of  r,M 2 when a poly is removed from data  x,M 2 can be interpreted as an approximate measure of  r,M 2 when a poly is removed from data   x,M 2 should be matched to x a,M (t,A) removed from the data Extracting information from data HP filters the residual noise Extracting information from data HP filters the residual noise  Based on assumption that f<<1 behavior of  r,M 2 same as  x,M 2  Need to investigate exact properties of  r,M 2 kernels further One type of variance is not appropriate for all problems One type of variance is not appropriate for all problems One should not arbitrarily change a variance order simply to avoid a divergence due to neg-  noise One should not arbitrarily change a variance order simply to avoid a divergence due to neg-  noise  There must be a rationale behind any variance change A divergence is a signal to fix a real design, spec, & analysis problem A divergence is a signal to fix a real design, spec, & analysis problem  May be essential or non-essential divergence Paper results can also be applied to variances of  (t) & y(t) Paper results can also be applied to variances of  (t) & y(t) Would like to acknowledge JPL time & frequency section for serving as sounding board and for making several suggestions that improved this paper Would like to acknowledge JPL time & frequency section for serving as sounding board and for making several suggestions that improved this paper