Zvi Kohavi and Niraj K. Jha 1 Linear Sequential Machines

2Introduction A linear sequential machine is a network which has a finite number of input and output terminals and is composed of interconnections of three types of basic components (unit delays, modulo-p adders and modulo-p scalar multipliers) The input signals are elements of a finite field GF(p) = {0,1,…,p-1} The operations performed by the basic components on their inputs are carried out according to the rules of GF(p) Block diagram: number of delay elements -- dimension

3 Linear Machine For a machine to be linear: its response to a linear combination of inputs must preserve the scale factor and the principle of superposition Thus, each basic component must be linear Cannot use an AND gate: z = x 1 x 2 modulo 2 Or an OR gate: z = x 1 + x 2 + x 1 x 2 modulo 2 Linear components: 1.Unit delays: y(t) = Y(t-1) 2.Modulo-p adders: z = x 1 + x 2 + … + x l (modulo p) 3.Modulo-p scalar multipliers: z = cx (modulo p), where c is an element of GF(p)

4 Example Example: A four-terminal, four-dimensional linear machine over GF(3) Linear machine over GF(2): binary machine Inert (or quiescent) machine: a linear machine whose delay elements are initially in the zero state Used as encoding and decoding devices

5 Feedforward Shift Registers Simplest type of inert linear machines: two-terminal shift register which contains only feedforward paths and whose output is a modulo-p sum of selected input digits z = a 0 x + a 1 Dx + … + a k D k x –D 0 = 1: identity operator –z = D 2 x: for all t >= 2, z(t) = x(t-2) –Initial conditions: y 1 (0) = y 2 (0) = … = y k (0) = 0 Rewrite equation as: z = (a 0 + a 1 D + … + a k D k )x, or –T(D): transfer function

6 Example Example: Consider the inert linear machine over GF(2) below where z(t) = x(t) + x(t-1) + x(t-3) Corresponding polynomial: z = x + Dx + D 3 x Transfer function:

7 Serial/Parallel Connection of Machines Consider two linear machines with input x 1 (x 2 ), output z 1 (z 2 ) and transfer function T 1 (T 2 ) Transfer function T 3 of the serial connection: –Since x 2 and z 1 are identical: Transfer function T 4 of the parallel connection: T 4 = T 1 + T 2 Example: Let T 1 = D 2 + 2D + 1 and T 2 = D + 1 over GF(3) Then T 3 = (D 2 + 2D + 1)(D + 1) = D T 4 = (D 2 + 2D + 1) + (D + 1) = D 2 + 2

8 Impulse Response and Null Sequences Impulse response h of an inert linear machine: its response to 100…0 Impulse response of the feedforward shift register: a 0 a 1 …a k 0…0 After at most k+1 time units: the output of the k-dimensional feedforward shift register is a sequence of 0’s Response of an inert linear machine to an arbitrary input sequence: can be obtained from its impulse response through discrete “convolution” in GF(p)

9Example Example: Impulse response of T 1 = 1 + D + D 3 is h = …0 Response of T 1 to input sequence 1011: obtained by addition (modulo 2) h + D 2 h + D 3 h of sequences as follows

10 Null Sequence Null sequence X 0 : nonzero input sequence that generates an output sequence consisting of 0’s Thus, TX 0 is a sequence of 0’s Any linear combination of null sequences is also a null sequence Example: Null sequence for T 1 = 1 + D + D 3 determined as follows: 0 = X 0 + DX 0 + D 3 X 0 X 0 = DX 0 + D 3 X 0 Select an arbitrary (nonzero) sequence of length 3 (in general, equal to dimension k) and specify subsequent digits –Selection of 001 X 0 = (001) –After seven digits: the null sequence repeats itself

11 Example Example: Null sequence for polynomial T = 1 + 2D 2 + D 3 over GF(3) 0 = X 0 + 2D 2 X 0 + D 3 X 0 Adding 2X 0 to both sides and recalling that 2X 0 + X 0 = 0 in modulo 3: 2X 0 = 2D 2 X 0 + D 3 X 0 Multiplying both sides by 2 yield: X 0 = D 2 X 0 + 2D 3 X 0 Starting with 111, null sequence: X 0 = (111) Maximal sequences: null sequences that contain p k -1 digits All possible k-tuples except 00…0

12 Inverse Machines A polynomial T(D), where z = Tx, has an inverse, denoted by 1/T(D), if there exists a network which realizes x = (1/T)z Example: Machine and its inverse: Note that (-a) modulo p = (p-a) modulo p When the inverse machine is fed the impulse response of the original machine, i.e., a 0 a 1 …a k 00…0: its response is the original message x = 100…0

13 Inverse Machines (Contd.) Inverse is realizable only if a 0 0 An inert linear machine described by delay polynomial T has a linear inverse described by T -1, which decodes without delay: if and only if T contains a nonzero constant term which is prime to modulo p For a binary machine: a 0 must be 1 If T does not contain a nonzero constant term: no instantaneous inverse can be found –However, an inverse, which decodes the original input after a finite delay, can be found –Let a i be the scalar associated with the lowest power of D for which a i 0, i.e., T = D i + a i+1 D i+1 + … + a k D k –The inverse is given by: –Or:

14 Example Example: A machine with T 1 = 1 + D + D 3 and its inverse with T 1 -1 = 1/(1+D+D 3 )

15 Linear Machines with Nonzero Initial Conditions The inverse of an inert linear machine might not be inert Hence, its response to a sequence of zero input symbols is not necessarily a sequence of zero output symbols: but may be null sequence X 0, whose starting digits are determined by the initial state of the inverse Example: Let the input symbols of T 1 and T 1 -1 be 0’s If they are inert: their respective output symbols will be all 0’s If they are not inert: output symbols will depend on their initial states Autonomous behavior: behavior of a noninert linear machine whose input is a sequence of 0’s Cycle sets: the two loops in the state diagram of T 1 -1

16 Inert Linear Machines and Rational Transfer Functions Realization of rational polynomials: consider the inert linear machine whose output z is given by z = x + Dx + D 2 x + D 4 x + Dz + D 3 z This can be rewritten as: z(1 + D + D 3 ) = x(1 + D + D 2 + D 4 ) The transfer function: Trivial, but inefficient, realization: Serially connect two inert linear machines given by polynomials 1 + D + D 2 + D 4 and 1/(1 + D + D 3 ) Requires seven delay elements Minimal realization: determined by the highest degree in either the numerator or denominator of the transfer function Only four delay elements necessary

17 Example (Contd.) Example (contd.): Rewrite equation as x + z = D(x + z) + D 2 x + D 3 z + D 4 x Or x + z = D{(x + z) + D{x + D(z + Dx)}} Chain realization of T 2 and T 2 -1

18 Chain Realization Chain realization of an arbitrary transfer function over GF(2): Realization of

19 Example Example: Realize the following transfer function over GF(3)

20 Impulse Response and Transfer Function Synthesis of an inert linear machine from its impulse response: If the impulse response is realizable: it consists of two components – transient component h t and periodic component h p For a k-dimensional inert linear machine: the period of the response to a sequence of 0’s is at most p k -1 = n-1, where n is the number of states Thus, a necessary condition for an impulse response h to be realizable: it ultimately becomes periodic Since the length of the transient response is at most k + 1: the transfer function of a realizable two-terminal k-dimensional inert linear machine can be specified uniquely by observing the first k + p k symbols in the impulse response

21 Example Example: Consider impulse response , , , … of an inert linear machine over GF(2) The impulse response can be separated as h = h t + h p From h t : T t = D Periodic component h p described by: 1 + D + D 2 + D 4 Since the period is 7: T p = (1 + D + D 2 + D 4 )(1 + D 7 + D 14 + …) = Hence:

22 Multiterminal Machines A multiterminal inert linear machine with l input terminals and m output terminals can be characterized by a set of lm transfer functions: T ij is evaluated when x i = 0 for all i j One possible realization (not necessarily minimal):

23 The General Model The matrix formulation: Consider a k-dimensional linear machine over GF(p), with l inputs and m outputs Next-state equation for delay Y i : Matrix form: Or Y(t) = y(t+1) = Ay(t) + Bx(t) A, B, C, D: characterizing matrices A: characteristic matrix

24 The General Model (Contd.) Output function z i : Matrix form: Or z(t) = Cy(t) + Dx(t)

25 Definition A machine is said to be linear over a finite field GF(p) if its states can be identified with the elements of a vector space, and its next state and output functions can be specified by a pair of matrix equations GF(p): Y(t) = Ay(t) + Bx(t) z(t) = Cy(t) + Dx(t) Dimension of the machine: dimension of the state vector The machine is Moore or Mealy: depending on whether D is or is not identically zero Example: A four-terminal machine and its characterizing matrices

26 The Response of Linear Machines Relationship between the input sequence to machine {A,B,C,D} and its corresponding output sequence: or where autonomous responseforced response

27 Autonomous Response Internal circuit: that part of the circuit that can be specified by A alone It contains only the delay elements and their interconnections, while the input and output lines have been deleted Autonomous response: generally determined from the analysis of the internal circuit Y(t) = y(t+1) = Ay(t) Since the internal circuit is autonomous: the -successor S j of state S i [where S i = y i (t)] is given by: y j (t) = A y i (t) where denotes the number of state transitions The sequence of predecessors of a given state is established by constructing the inverse internal circuit: –Such an inverse exists: only if each state has a unique predecessor –For an internal circuit given by A: the inverse is given by A -1 –Thus, the inverse circuit exists: if and only if A is nonsingular

28 Reduction of Linear Machines Let L be a k-dimensional linear machine over GF(p) To describe an experiment of length k: compactly represent earlier equations as Z (k) = K k y(0) + V k X (k) K k : diagnostic (or distinguishing) matrix For initial states S a and S b : corresponding state vectors are y a (0) and y b (0)

29 Reduction of Linear Machines (Contd.) If S a is equivalent to S b : then K k y a (0) = K k y b (0) Since the second term V k X (k) is independent of the initial state To simplify the computation: X (k) may be selected as the all-zero sequence X (k) = 0 Thus, the output equation reduces to: Z (k) = K k y(0) Theorem: A k-dimensional linear machine {A,B,C,D} is definitely diagnosable of order k if and only if diagnostic matric K k has k linearly independent rows A linear machine is in reduced form: if and only if the rank of K k is k Every reduced k-dimensional linear machine is definitely diagnosable of order k, and is finite memory of order less than or equal to k: observability and predictability properties

30 Example Example: Consider linear machine L 1 over GF(2) given by Diagnostic matrix K 3 is obtained and Z (3) becomes: Since the rank of K 3 is 3: the dimension of L 1 cannot be reduced For a given initial state: the values of y 1 (0), y 2 (0), y 3 (0) are specified Matrix Z (t) yields the response of L 1 to distinguishing sequence 000 –E.g., if the initial state is (111): then in response to 000, sequences z 1 = 010 and z 2 = 100 are produced

31 The Minimization Procedure Let L be a k-dimensional linear machine {A,B,C,D} over GF(p), and let r be the rank of the diagnostic matrix, r < k Define an r x k matrix T consisting of the first r linearly independent rows of K k, and a k x r matrix R denoting the right inverse of T, s.t. TR = I r Define an r-dimensional machine L* with characterizing matrices {A*,B*,C*,D*}, s.t. A* = TAR, B* = TB, C* = CR, and D* = D Theorem: State y of L is equivalent to state y* = Ty of L*. Machine L* is a reduced machine equivalent to L

32 Example Example: Consider linear machine L 2 over GF(2), defined by The rank of K 3 is 2: thus L 2 is reducible The first two rows of K 3 are linearly independent: therefore The right inverse R of T is constructed by selecting a set of r linearly independent columns from T –Since the rank of T is r and column rank equals row rank: such a set always exists –Form an r x r matrix Q from these columns and find its inverse Q -1

33 Example (Contd.) Example (contd.): The right inverse R, which is a k x r matrix, is formed by placing in it the rows of Q -1 in positions corresponding to the columns selected from T, and where all other rows are set to zero In our case: From definitions of characterizing matrices of L 2 *:

34 Example (Contd.) Circuit diagram of reduced machine L 2 *:

35 Example Example: Consider linear machine L 3 given by {A,B,C,D} over GF(2) and shown in the figure

36 Example (Contd.) Example (contd.): Q occupies the first three columns of T and Q -1 the first three rows of R, since the linearly independent columns in T have been selected from positions 1, 2, and 3

37 Example (Contd.) Example (contd.):

38 Example (Contd.) Example (contd.): The reduced circuit corresponding to {A*,B*,C*,D*} is:

39 Example (Contd.) The first three linearly independent rows of K 3 * are the rows of I 3 in natural order: Matrix (A*) t is related to A t by: (A*) t = TA t R and diagnostic matrix K* is related to K by K* = KR Thus, for initial state y a * = [y 1 *,y 2 *,y 3 *] and the all-0 input sequence: the output corresponding to the unit vector rows of K 3 * are identical to values y 1 *, y 2 *, y 3 *

40 Identification of Linear Machines Let sequential machine M have p k states, denoted by S a, S b,.., S pk Let the l-dimensional vector x and m-dimensional vector z denote the input and output vectors, respectively Construct a distinguishing table, which contains the output symbols generated by M in response to a sequence of 0’s The table contains p k columns corresponding to the states of M –It is formed block by block: the i th block corresponds to z(t) at t = i –The table thus contains at most k blocks of m rows each: corresponding to z(0), z(1), …, z(k-1) –The process of adding blocks to the table is terminated: when, for some t, the set of rows contained in block z(t) is linearly dependent on the rows in preceding blocks Example: Machine M 4 and its distinguishing table U

41 First Test First test: based on the fact that, for every linear machine, the all-0’s sequence is a distinguishing sequence If M is reduced: the columns of U must be distinct, since otherwise there would be two or more states in M which are indistinguishable under the all-0’s sequence, and M is not linear –The U in the example satisfies the test Let U* be the table consisting of the first r linearly independent rows of U, and let S i denote the i th column of U* Assuming that a linear realization of M is possible: let states A, B, … of M correspond to y a, y b, … of its linear realization L This is accomplished: by selecting the p k columns of U* as the state assignment for the p k states of L For machine L 4, the linear realization of M 4, we have

42 Identification (Contd.) To obtain {A,B,C,D} of L: we select r linearly independent columns from U*, corresponding to r state vectors of L, and form an r x r matrix v s.t. v = [y a,y b,…,y r ] Hence, the next-state function of L under 0 input symbols is: [Y a 0,Y b 0,…,Y r 0 ] = Av where Y i 0 denotes the 0-successor of y i. Since v is nonsingular: A = [Y a 0,Y b 0,…,Y r 0 ]v -1 If all r unit vectors appear in U*: v can be chosen as I r, which yields v = v -1 Thus, A = [Y a 0,Y b 0,…,Y r 0 ] Whenever the number of states p k = p r, i.e., k = r: v can be specified as I r

43 Identification (Contd.) Similarly, for x(t) = 0: [z a 0,z b 0,…,z r 0 ] = Cv where z i 0 denotes the output symbol produced by L when in state y i and excited by x = 0. Thus: C = [z a 0,z b 0,…,z r 0 ]v -1 and when v = I r : C = [z a 0,z b 0,…,z r 0 ] In order to obtain B and D: let us denote a unit input vector as u i, where the i th component of u i is 1 and all other components are 0’s We know that: Bx = Y – Ay In order to obtain B: we select some state y i (preferably the zero state if it exists in U*) and specify B in terms of the constraints imposed on it by y i and the unit input vectors

44 Identification (Contd.) Let the input consist of unit vectors: u = [u 1,u 2,…,u l ] The next-state vector, Y i uj, denotes the u j -successor of y i. Thus: Y i u = [Y i u1,Y i u2,…,Y i ul ] and Bu = Y i u - Ay i or B = [Y i u – Ay i ]u -1 Since u generally consists of unit vectors, and y is the zero state: B = [Y i u1,Y i u2,…,Y i ul ] Similarly: D = {[z i u1,z i u2,…,z i ul ] - Ay i }u -1 where z i uj is the output vector associated with the transition from y i under u j. Thus, the reduced equation is: D = [z i u1,z i u2,…,z i ul ]

45 Example Example: Returning to machine M 4, we specify v to be We obtain: The only unit input vector is u = [1]: hence Y i 1 is the 1-successor of y i Since the zero state is contained in U*: let y i = y d. Thus, The state and output equations are:

46 Example (Contd.) The final test is to verify that the above equations indeed represent M 4 under all input and state combinations: this is accomplished by verifying each state transition and its corresponding output symbol E.g., substituting y a for A and 0 for x(t): the machine should go to state y b and produce output symbol 11, corresponding to entry B,11 in column 0, row A Indeed: The characterizing matrices are thus verified. Linear realization:

47 Example Example: Machine M 5 and its distinguishing table are given below Checked rows are linearly independent Since U* contains all possible eight 3-tuples: identification procedure is continued

48 Example (Contd.) Example (contd.): Select We obtain: Setting u = [1] and y i = y a = 0: Thus: Matrices are verified to correspond to M 5 :

49 Example Example: Consider the four-state up-down Gray-code counter and its distinguishing table The state assignment is given by: Though M 6 has only four states: its minimal realization is of a third dimension v cannot be chosen as the identity matrix: and zero state y i = 0 is not contained in the state assignment –Hence, simplified equations cannot be used: matrix inversion cannot be avoided

50 Example (Contd.) Let Hence, we obtain Let y i 1 = y a. Then Minimum-dimension linear circuit:

51 Application of Linear Machines to Error Correction Consider the communication-system model shown below Message, denoted X: consists of a sequence over GF(p) of length n Encoder, with transfer function T: transforms the message into another sequence over GF(p) of length n –This sequence is referred to as the transmitted sequence, designated Z: where Z = TX Sequence Z is transmitted through a noisy channel: whose output sequence Z is called the received sequence In the channel: a noise sequence over GF(p), denoted N, is added to the transmitted sequence, s.t. the received sequence is equal to Z = Z + N = TX + N

52 Decoder Decoder, with transfer function T -1 : processes the received sequence and produces X s.t. X = T -1 Z = T -1 (TX + N) = X + T -1 N Example: Consider the communication system shown below Information digits: the first four digits Checking digits: the remaining three digits Checking digits in X: always 0 If X is received with three 0’s in the last three positions: no noise present –Else, error correction is necessary

53 Example (Contd.) When an error occurs: obtain T -1 N and subtract from X Since the last three digits of X were originally 0’s: the last three digits of X must consist only of digits of T -1 N, without any contribution from X If only a single error occurred at time t: T -1 N is simply the response of decoder T -1 to a unit impulse at t Hence, the checking digits of X consist of a subsequence of three digits of the impulse response of T -1 The decoder is chosen s.t. its impulse response has a maximal period of seven digits: this ensures that by observing the subsequence contained in the last three digits of X, we can determine T -1 N uniquely Suppose is to be transmitted: –Impulse response of the decoder: –Checking digits of X: identical to fourth, fifth and sixth digits of the impulse response »Thus, we conclude that noise impulse occurred in the second information digit

54 Single-error Correction To correct single errors in messages over GF(2) containing m information digits and k checking digits We need a decoder whose impulse response is of length m + k: with each string of k successive digits different from every other subsequence of length k Such an impulse response can be obtained from: a decoder whose transfer function is of degree k and whose impulse response is maximal, i.e., of length m + k = 2 k -1 If the last k digits of X are not zeros: T -1 N must be subtracted from X –Accomplished by shifting X over the decoder’s impulse response: until the last k digits of X match a corresponding subsequence of the impulse response –This is always possible: since the impulse response contains every nonzero subsequence of length k –The modulo-2 sum of X and the digits of the impulse response appearing directly below it: yield X