Digital data processing apparatus using daisy chain control

U.S. Patent Apr. 6, 1993 Sheet 4 of 5 5,201,029

1 2

respective set of operands, (ii) a digital multiplier first DIGITAL DATA PROCESSING APPARATUS and second inputs of which are coupled to said common

USING DAISY CHAIN CONTROL data input and to a data output of the operand store

respectively, for multiplying successive items of the set

This is a continuation of application Ser. No. 5 of sequential input digital data items by respective oper07/427,411, filed Oct. 24, 1989, now abandoned. ands read from the operand store, (iii) an accumulator

This invention relates to digital data processing appa- to the data input of which the output of the multiplier is ratus. coupled, for accumulating the results of the multiplica

It is sometimes required that the same set of input tions, (iv) a latch arrangement for storing and outputdata items is processed several times using different sets 10 ting the respective item of output data, and (v) data of operands to produce respective items of output data. transfer means coupling the output of the accumulator This is the case, for example, in each layer of a multi- to a data mput 0f tne corresponding latch, said data layer perceptron the organisation of an analog example transfer means having a non-linear transfer characterisof which (being a type of so-called "neural network" ) tic ^ ^ outputs 0f said iatch arrangements being is shown diagrammatically in FIG. 1 of the drawings. In 15 ... t0 a common data output, the apparatus includthe example shown in FIG. 1 the perceptron comprises mg clock pulse generator means coupled to the operand three layers each in the form of a set of four data pro- stores md the latch arrangements for reading out the cessing cells 1-*, 5-* and 9-12 respectively Each cell contents of the storage locations of each said operand has four inputs and one output, the outputs 13-16 of the store m succession ^ m step ^ the corresponding cells 1-4 of the first layer each being connected to a 20 readK)ut of the of the st locations of the

respective mput of each cell 5-8 of the second layer othef ^ an(J stores ^ readin Qut the amteBt of and the outputs 17-20 of the cells 5-8 of the second ^ ^ ^gemer* in succession onto the common layer each being connected to a respective mput of each ^ fAlthoueh each resoective set of ooerands

"ii v;,of !rr- ^ °rts 2i-%f* „ ^^ss^ss^^s^js^ssSi

cells 9-12 of the third layer together constitute the 25 ^^^^u, K» A. TM„ ;„ „ii „;,„„„,..,„ , v°

„ „ r j. ^ It- r ■ not necessarily be the case in all circumstances. How

output of the network. Four items of mput data are . ...' . , , . of these

denoted by circles 25,26,27 and 28 respectively, each of ev,' Ti- ♦3? ^iT\

iL . . 3 ,. ,' .. . r. , 1 «« * sets are different from each other),

these being applied to a respective mput of each cell 1-4 Ti, , . ,' . . , ,

of the first laVer. In generaleach of the data processing . 11 has n0^ recognised that the requisite calcula

cells 1-12 is arranged to produce an output signal which 30 t10"5 m «J«J ^ of a multi-layer perceptron can be

is a non-linear function of the result of subtracting a Pffonned digitally m an efficacious manner if the cells

threshold value (which may be zero) from the sum of its of fch ... *re arranged to have a common data input

input signals after these input signals have been TMd a common data output and the set of mput data

weighted by multiplying them by predetermined re- &ntt for each layer is arranged to occur sequentially on

spective weighting factors. The non-linear function 35 the re»evant common data mput. The mput data items

may have, for example, a sigmoid form. As another <** then processed in parallel by the cells of the

example it may be obtained by means of a hard limiting relevant layers and, ii the results are latched and then

process. The perceptron may be employed, for exam- read out sequentially onto the relevant data output,

pie, as a classifier for an input vector represented by the these results can act directly as the sequential input data

four items of input data 25-28, the resulting vector at 40 for tfle next layer (if present). The respective weighting

the outputs 21-24 being indicative of to which of a factors required for the processing in each cell can be

plurality of classes the input vector belongs (these st°red therein and read out as required,

classes being stored in the perceptron in the form of the If> for example, the apparatus is required to form part

set of weighting factors). The choice of four cells per of a multilayer perceptron then, as implied above, said

layer and three layers is, of course, only an example; in 45 common data output may constitute the common data

some cases fewer layers will suffice and, moreover, it is mPut of a further such set of digital data processing

not even essential that each layer comprises the same cells. As an alternative, however, a single set of cells

number of cells. may be used recursively, the common data output

Perceptrons are discussed, for example in an article thereof then being coupled to the common data input

by R. P. Lippmann entitled "An Introduction to Com- 50 thereof. If maximum processing speed for successive

puting with Neural Nets" in IEEE ASSP Magazine, sets of input data items is required then the former ar

April 1987, pages 4-22, particularly pages 13-18. Practi- rangement will be preferred, whereas if rmnimisation of

cal implementations of neural networks up to the pres- the hardware employed takes precedence then the latter

ent time appear, in general, to employ analog circuitry arrangement will probably be preferred. In both cases

(see e.g. EP-A-242110). However, it is often preferred 55 the clock pulse generator means is preferably arranged

to employ digital techniques and it is an object of the to read out the contents of each latch arrangement in

present invention to provide a digital circuit architec- succession in step with the reading out of the contents

ture which enables the multiple crossover connections of the storage locations of the operand stores. If this is

inherent in the architecture shown in FIG. 1 to be the case synchronization between the inputting of suc

avoided. 60 cessive data items to a cell from other cells and the

The invention provides digital data processing appa- reading out of the relevant operands in that cell can be

ratus comprising a set of digital data processing cells for facilitated.

each processing, using a respective set of operands, the Embodiments of the invention will now be described,

same set of sequential input digital data items to produce by way of example, with reference to the accompanying

a respective item of digital output data, said cells having 65 diagrammatic drawings, in which:

a common input for the set of sequential input data FIG. 1 shows the organisation of a multilayer percep

items and each comprising (i) an operand store having a tron which operates in an analog manner, as referred to

storage location corresponding to each operand of the previously,

3 4

FIG. 2 is a block diagram of a first embodiment of the FIG. 3 is a block diagram showing a possible coninvention, struction for each of the cells 29 of FIG. 2. As shown in

FIG. 3 shows a possible construction for several of FIG. 3 the cell 29 has a data input 50 (connected to the

the blocks of FIG. 2 in more detail, bus 30,32 or 33 of FIG. 2) and a data output 51 (con

FIG. 4 shows some clock pulse waveforms occurring 5 nected to the bus 32,33 or 34 of FIG. 2). It comprises an

in the embodiment of FIGS. 2 and 3, operand store 52 the data output 53 of which is con

FIG. 5 is a block diagram of a second embodiment of nected to a data input 54 of digital data processing

the invention, and means 55 and the address signal input 56 of which is

FIG. 6 shows some clock pulse waveforms occurring connected to the parallel output 57 of a counter 58. The

in the embodiment of FIG. 5. 10 clock Signal input 59 of counter 58 is connected to the

FIG. 2 is a block diagram of digital data processing output d of the clock signal generator arrangement 35 apparatus comprising three sets of four digital data of FIG- 2 811(1 tne reset signal input 60 of counter 58 is processing cells 29A-29D, 29E-29H and 29I-29L re- connected to the output c of arrangement 35. The cell spectively. The cells of the set 29A-29D each process, data mPut 50 » connected to a further data input 61 of using a respective set of operands, the same set of input 15 processing means 55 and the data output 62 of processdigital data items presented sequentially to a common m8 means 55 « coupled to the cell data output 51 via a input constituted by a data bus 30 by a data source 31, to latch arrangement comprising a latch circuit 63 and a produce a respective item of digital output data. The buffer circuit 64 which has a three-state output. The resulting four items of digital output data are presented latch <*>ntTC* mPut f5 °f circ,u,t 63 18 conto a common data output constituted by a daU bus 32 in 20 n«ted. to the output c of the clock pulse generator succession; how this is done will be elaborated upon arrangement 35 of FIG. 2, as is a reset signal input 66 of below. Data bus 32 also constitutes a common data the Processing means 55. A clock signal input 67 of input of the cells 29E-29H which each process, using a Processing means 55 is connected to the output bof respective set of operands, the aforesaid four items of „ clock pulse generator arrangement 35 of FIG. 2. The

dighal output data to produce a respective item of digi- 25 outPut ea*\ef cTI°l ?,g"lT 68 f Jf"

. f . . j . Tl „ r rj _;»i » is connected to the output 69 of a smgle-bit second latch

tal output data. The resulting four items of digital out- .. _„ ... . r. . , e t . .

, , i „ ,, e , . „ * circuit 70, which output is also connected to one mput

put data are presented to a common data output consti- „,„ Axrri „ . „ • . _ ., . v .

: * J L ,. T . , of an AND gate 71 via an inverter 72. The other mput

tutedI by a data bus 33 in succession which bus also of AND *n fa reeled t0 the si d mJm

constitutes a common data mput of the cells 29I-29I. ^ ^ of the ^ ^ the of AND ^ {$ ^

The cells 29I-29L each process, using a respective set of nected tQ tne ^ A ^ m ?3 of latch6circuit 70. ^

operands, the last-mentioned four items of digital output latch &- ^ m ?4 of ktch ckcuit ?0 is con.

data to produce a respective item of digital output data. nected t0 the out t d of the dock lse generator

The resultmg four items of digital output data are pres- arrangement 35 of FIG. 2. The digital data processing

ented to a common data output constituted by a data 35 means 55 comprises a digital multiplier 75 to the two

bus 34 in succession. Each of the cells 29A-29L has mputs of which ^ copied the data inputs 54 a„d 61

three clock-pulse inputs b,c and d respectively, which respectively, a digital adder/accumulator 76 to one

are connected to corresponding outputs of a clock pulse mput 77 of which the output of multiplier 75 is coupled

generator arrangement 35. Output d is also connected to ^ the output 78 of which is coupled to the other input

a clock pulse mput of the data source 31 Moreover, 40 79 output 78 is aso coupled to the address signal

each of the cells 29A-29L has a control signal mput 36 mput go of a suitably programmed look-up table mem

and a control signal output 37, the control signal output ory 81 the data output of which constitutes the output

37 of each cell of each set being connected to the con- 62 of processing means 55.

trol signal input 36 of the next cell (if present) of the The arrangement so far described with reference to

same set. The control signal inputs 36A, 36E and 361 of 45 FIGS. 2 and 3 operates as follows, it being assumed as

the first cells 29A, 29E and 291 respectively of each set an example that the processing means 55 of each cell 29

are connected to the Q-outputs 38, 39 and 40 respec- processes each digital data item presented to the input

tively of set/reset flip-flops 41,42 and 43 respectively, 50 under the control of two clock pulses presented to its

the set inputs 44,45 and 46 respectively of which are mput 67 and that each cell 29 processes four items of

connected to the output c of clock pulse generator 50 input digital data to produce one item of output digital

arrangement 35 and the reset inputs 47,48 and 49 respec- data. This being the case the clock pulse generator

tively of which are connected to the output d of clock arrangement 35 of FIG. 2 is arranged to produce the

pulse generator arrangement 35. The control signal dock pulse signals shown in FIG. 4 at its outputs b, d

outputs 37D, 37H and 37L of the last cells 29D, 29H and c respectively. (To this end arrangement 35 may

and 29L of each set are not used. Clock signals pro- 55 comprise a clocked counter provided with three suit

duced on the output d of the arrangement 35 control the ably chosen decoder circuits connected to its parallel

outputting of data items by the source 31 and by the output) As will be seen from FIG. 4, two pulses are

cells 29 under the co-control of signals presented to produced at output b between each pair of successive

their control signal inputs 36, and also resetting of the pulses produced at output d, and one pulse is produced

flip-flops 41-43. Clock signals produced on the output b 60 at output c just prior to each group of four pulses proo: the arrangement 35 control the processing of the data duced at output d. Each c-pulse activates the latch

items inputted to the cells 29 to produce the respective circuit 63 included in each cell 29 to store the data item

items of output data. Clock signals produced on the currently present at the output 62 of the corresponding output c of the arrangement 35 control the latching of data processing means 55, resets processing means 55 to the said respective items of output data in the cells 29 65 an initial state in which the contents of adder/accumulaprior to their outputting. The manner in which this is tor 76 are zero, and also resets counter 58 to zero. The achieved will now be described with reference to c-pulse also sets the flip-flops 41-43 of FIG. 2. If it is

FIGS. 3 and 4. assumed that the contents of all the latch circuits 70 are 60

65