1. Digital Signal Processing Topic 2: Time domain
Discrete-time systems
Convolution
Linear Constant-Coefficient Difference Equations (LCCDEs)
Correlation
Based on: By: Dan Ellis
Saeid Rahati

2. 1. Discrete-time systems
A system converts input to output:
E.g. Moving Average (MA):
x[n]
DT System
y[n]
y[n] = f (x[n])
x[n]
y[n] +
z-1
1/M
x[n-1]
x[n-2]
(M = 3)
Saeid Rahati

3. Moving Average (MA) n x[n] A x[n] y[n] + z-1 1/M x[n-1] x[n-2] n3 4 5 6 7 8 9 A
x[n]
y[n] +
z-1
1/M
x[n-1]
x[n-2] n
x[n-1] -1 1 2 3 4 5 6 7 8 9 n
x[n-2] -1 1 2 3 4 5 6 7 8 9 n
y[n] -1 1 2 3 4 5 6 7 8 9
Saeid Rahati

4. MA Smoother
MA smooths out rapid variations (e.g. “12 month moving average”)
e.g. x[n] = s[n] + d[n]
signal noise
5-pt
moving
average
Saeid Rahati

5. Accumulator Output accumulates all past inputs: x[n] y[n] + z-1 y[n-1]
Saeid Rahati

6. Accumulator n x[n] A n y[n-1] x[n] y[n] + z-1 y[n-1] n y[n] -1 1 2 3 45 6 7 8 9 A n
y[n-1] -1 1 2 3 4 5 6 7 8 9
x[n]
y[n] +
z-1
y[n-1] n
y[n] -1 1 2 3 4 5 6 7 8 9
Saeid Rahati

7. Classes of DT systems Linear systems obey superposition:
if input x1[n]  output y1[n], x2  y2 ...
given a linear combination of inputs:
then output for all a, b, x1, x2 i.e. same linear combination of outputs
x[n]
DT system
y[n]
Saeid Rahati

8. Linearity: Example 1
Accumulator:
Linear
Saeid Rahati

9. Linearity Example 2:
“Energy operator”:
Nonlinear
Saeid Rahati

10. Linearity Example 3: ‘Offset’ accumulator: but Nonlinear
.. unless C = 0
Saeid Rahati

11. Property: Shift (time) invariance
Time-shift of input causes same shift in output
i.e. if x1[n]  y1[n] then
i.e. process doesn’t depend on absolute value of n
Saeid Rahati

12. Shift-invariance counterexample
Upsampler: L
Not shift invariant
Saeid Rahati

13. Another counterexample
scaling by time index
Hence
If then
Not shift invariant - parameters depend on n ≠
Saeid Rahati

14. Linear Shift Invariant (LSI)
Systems which are both linear and shift invariant are easily manipulated mathematically
This is still a wide and useful class of systems
If discrete index corresponds to time, called Linear Time Invariant (LTI)
Saeid Rahati

15. Causality
If output depends only on past and current inputs (not future), system is called causal
Formally, if x1[n]  y1[n] & x2[n]  y2[n]
Causal
Saeid Rahati

16. Causality example
Moving average: y[n] depends on x[n-k], k ≥ 0  causal
‘Centered’ moving average .. looks forward in time  noncausal
Can sometimes make a non-causal system causal by delaying its output
Saeid Rahati

17. Impulse response (IR) Impulse (unit sample sequence)
d[n] -1 1 2 3 4 5 6 7 -2 -3
Impulse (unit sample sequence)
Given a system: if x[n] = d[n] then y[n] = h[n]
LSI system completely specified by h[n]
x[n]
DT system
y[n] ∆
“impulse response”
Saeid Rahati

18. Impulse response example
x[n]
y[n] +
z-1 a2 a3
Simple system: n
x[n] -1 1 2 3 4 5 6 7 -2 -3
x[n] = d[n] impulse n
y[n] -1 1 2 3 4 5 6 7 a1 -2 -3 a2 a3
y[n] = h[n] impulse response
Saeid Rahati

19. 2. Convolution Impulse response: Shift invariance: + Linearity:
Can express any sequence with ds: x[n] = x[0]d[n] + x[1]d[n-1] + x[2]d[n-2]..
d[n]
LSI
h[n]
d[n-n0]
LSI
h[n-n0]
a·d[n-k]
+ b·d[n-l]
a·h[n-k]
+ b·h[n-l]
LSI
Saeid Rahati

20. Convolution sum Hence, since For LSI,
written as
Summation is symmetric in x and h i.e. l = n – k 
Convolution
sum * * *
Saeid Rahati

21. Convolution properties
LSI System output y[n] = input x[n] convolved with impulse response h[n]  h[n] completely describes system
Commutative:
Associative:
Distributive:
x[n] h[n] = h[n] x[n] *
(x[n] * h[n]) * y[n] = x[n] * (h[n] * y[n])
h[n] (x[n] + y[n]) = h[n] x[n] + h[n] y[n] *
Saeid Rahati

22. Interpreting convolution
Passing a signal through a (LSI) system is equivalent to convolving it with the system’s impulse response
x[n]
h[n]
y[n] = x[n] h[n] * n -1 1 2 3 4 5 6 7 -2 -3
h[n] = {3 2 1} n -1 1 2 3 5 6 7 -2 -3
x[n]={ }
Saeid Rahati

23. Convolution interpretation 1k -1 1 2 3 5 6 7 -2 -3
x[k]
Time-reverse h, shift by n, take inner product against fixed x k -1 1 2 3 4 5 6 7 -2 -3
h[0-k] k -1 1 2 3 4 5 6 7 -2 -3
h[1-k] n -1 1 2 3 5 7 -2 -3
y[n] 9 11 -1 1 2 3 4 5 6 7 -2 -3
h[2-k] k
Saeid Rahati

24. Convolution interpretation 2-1 1 2 3 5 6 7 -2 -3
x[n]
h[k] n -1 1 2 3 4 6 7 -2 -3
x[n-1]
Shifted x’s weighted by points in h
Conversely, weighted, delayed versions of h ... k -1 1 2 3 4 5 6 7 -2 -3
x[n-2] n -3 -2 -1 1 2 3 4 5 7 n -1 1 2 3 5 7 -2 -3
y[n] 9 11
Saeid Rahati

25. Matrix interpretation
Diagonals in X matrix are equal
Saeid Rahati

26. Convolution notes
Total nonzero length of convolving N and M point sequences is N+M-1
Adding the indices of the terms within the summation gives n :
i.e. summation indices move in opposite senses
Saeid Rahati

27. Convolution in MATLAB
The M-file conv implements the convolution sum of two finite-length sequences If
then conv(a,b) yields M
Saeid Rahati

28. Connected systems Cascade connection:
Impulse response h[n] of the cascade of two systems with impulse responses h1[n] and h2[n] is
By commutativity, * *
Saeid Rahati

29. Inverse systems d[n] is identity for convolution i.e. Consider
h2[n] is the inverse system of h1[n] x [ n ] d = *
x[n]
y[n]
z[n] z [ n ] = h 2 y 1 x * = x [ n ] if *
Saeid Rahati

30. Inverse systems
Use inverse system to recover input x[n] from output y[n] (e.g. to undo effects of transmission channel)
Only sometimes possible - e.g. cannot ‘invert’ h1[n] = 0
In general, attempt to solve *
Saeid Rahati

31. Inverse system example
Accumulator: Impulse response h1[n] = m[n]
‘Backwards difference’ .. has desired property:
Thus, ‘backwards difference’ is inverse system of accumulator.
Saeid Rahati

32. Parallel connection
Impulse response of two parallel systems added together is:
Saeid Rahati

33. 3. Linear Constant-Coefficient Difference Equation (LCCDE)
General spec. of DT, LSI, finite-dim sys:
Rearrange for y[n] in causal form:
WLOG, always have d0 = 1
defined by {dk}, {pk}
order = max(N,M)
Saeid Rahati

34. Solving LCCDEs “Total solution” Complementary Solution satisfies
Particular Solution for given forcing function x[n]
Saeid Rahati

35. Complementary Solution
General form of unforced oscillation i.e. system’s ‘natural modes’
Assume yc has form
Characteristic polynomial
of system - depends only on {dk}
Saeid Rahati

36. Complementary Solution
factors into roots li , i.e.
Each/any li satisfies eqn.
Thus, complementary solution: Any linear combination will work  ais are free to match initial conditions
Saeid Rahati

37. Complementary Solution
Repeated roots in chr. poly:
Complex lis  sinusoidal
Saeid Rahati

38. Particular Solution Recall: Total solution
Particular solution reflects input
‘Modes’ usually decay away for large n leaving just yp[n]
Assume ‘form’ of x[n], scaled by b: e.g. x[n] constant  yp[n] = b x[n] = l0n  yp[n] = b · l0n (l0  li)
or = b nL+1 l0n (l0  li)
Saeid Rahati

39. LCCDE example Need input: x[n] = 8m[n]
y[n] +
z-1 -1 +
z-1 6
Need input: x[n] = 8m[n]
Need initial conditions: y[-1] = 1, y[-2] = -1
Saeid Rahati

40. LCCDE example Complementary solution: a1, a2 are unknown at this point
roots l1 = -3, l2 = 2
Saeid Rahati

41. LCCDE example Particular solution: Input x[n] is constant = 8m[n]
assume yp[n] = b, substitute in:
(‘large’ n)
Saeid Rahati

42. LCCDE example Total solution
Solve for unknown ais by substituting initial conditions into DE at n = 0, 1, ...
n = 0
from ICs
Saeid Rahati

43. LCCDE example n = 1 solve: a1 = -1.8, a2 = 4.8 Hence, system output:
Don’t find ais by solving with ICs at n = -1,-2
(ICs may not reflect natural modes)
Saeid Rahati

44. LCCDE solving summary
Difference Equation (DE): Ay[n] + By[n-1] = Cx[n] + Dx[n-1] Initial Conditions (ICs): y[-1] = ...
DE RHS = 0 with y[n]=ln  roots {li} gives complementary soln
Particular soln: yp[n] ~ x[n] solve for bl0n “at large n”
ais by substituting DE at n = 0, 1, ... ICs for y[-1], y[-2]; yt=yc+yp for y[0], y[1]
Saeid Rahati

45. LCCDEs: zero input/zero state
Alternative approach to solving LCCDEs is to solve two subproblems:
yzi[n], response with zero input (just ICs)
yzs[n], response with zero state (just x[n])
Because of linearity, y[n] = yzi[n]+yzs[n]
Both subproblems are ‘real’
But, have to solve for ais twice (then sum them)
Saeid Rahati

46. Impulse response of LCCDEs
i.e. solve with x[n] = d[n]  y[n] = h[n] (zero ICs)
With x[n] = d[n], ‘form’ of yp[n] = 0  just solve yc[n] for n = 0,1... to find ais
d[n]
LCCDE
h[n]
Saeid Rahati

47. LCCDE IR example e.g. (from before); x[n] = d[n]; y[n] = 0 for n<0
thus 1
n ≥ 0
Infinite length
Saeid Rahati

48. System property: Stability
Certain systems can be unstable e.g. Output grows without limit in some conditions
x[n]
y[n] +
z-1 2 n
y[n] -1 1 2 3 4
...
Saeid Rahati

49. Stability
Several definitions for stability; we use Bounded-input, bounded-output (BIBO) stable
For every bounded input
output is also subject to a finite bound,
Saeid Rahati

50. Stability example
MA filter:
 BIBO Stable
Saeid Rahati

51. Stability & LCCDEs LCCDE output is of form:
as and bs depend on input & ICs, but to be bounded for any input we need |l| < 1
Saeid Rahati

52. Cross correlation of x against y
Correlation ~ identifies similarity between sequences:
Note:
Cross correlation of x against y
“lag”
call m = n - l
Saeid Rahati

53. Correlation and convolution
Hence: Correlation may be calculated by convolving with time-reversed sequence * *
Saeid Rahati

54. Autocorrelation
Autocorrelation (AC) is correlation of signal with itself:
Note:
Energy of sequence x[n]
Saeid Rahati

55. Correlation maxima Note: Similarly: From geometry,
when x//y, cosq = 1, else cosq < 1
angle between x and y
Saeid Rahati

56. AC of a periodic sequence
Sequence of period N:
Calculate AC over a finite window:
if M >> N
Saeid Rahati

57. AC of a periodic sequence
i.e AC of periodic sequence is periodic
Average energy per sample or Power of x
Saeid Rahati

58. What correlations look like[ ] l xx
AC of any x[n]
AC of periodic
Cross correlation l r [ ] l x ˜ x ˜ l r [ l ] xy l
Saeid Rahati

59. What correlation looks like
Saeid Rahati