US6141721A - Method of asynchronous memory access - Google Patents

Method of asynchronous memory access Download PDF

Info

Publication number
US6141721A
US6141721A US09/272,521 US27252199A US6141721A US 6141721 A US6141721 A US 6141721A US 27252199 A US27252199 A US 27252199A US 6141721 A US6141721 A US 6141721A
Authority
US
United States
Prior art keywords
data
time
row
decoder
read
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
US09/272,521
Inventor
Donald William Patterson
William P. Robbins
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
TALON RESEARCH LLC
Original Assignee
Discovision Associates
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=26305370&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=US6141721(A) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Priority claimed from GB9415413A external-priority patent/GB9415413D0/en
Priority claimed from US08/400,201 external-priority patent/US5603012A/en
Application filed by Discovision Associates filed Critical Discovision Associates
Priority to US09/272,521 priority Critical patent/US6141721A/en
Application granted granted Critical
Publication of US6141721A publication Critical patent/US6141721A/en
Assigned to COASES INVESTMENTS BROS. L.L.C. reassignment COASES INVESTMENTS BROS. L.L.C. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: DISCOVISION ASSOCIATES
Assigned to INTELLECTUAL VENTURES DRAM 1 LLC reassignment INTELLECTUAL VENTURES DRAM 1 LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: COASES INVESTMENTS BROS. L.L.C.
Assigned to TALON RESEARCH, LLC reassignment TALON RESEARCH, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: INTELLECTUAL VENTURES DRAM 1 LLC
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/0207—Addressing or allocation; Relocation with multidimensional access, e.g. row/column, matrix
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/16—Handling requests for interconnection or transfer for access to memory bus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G06F13/14—Handling requests for interconnection or transfer
    • G06F13/20—Handling requests for interconnection or transfer for access to input/output bus
    • G06F13/28—Handling requests for interconnection or transfer for access to input/output bus using burst mode transfer, e.g. direct memory access DMA, cycle steal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4305—Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/4302—Content synchronisation processes, e.g. decoder synchronisation
    • H04N21/4307—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen
    • H04N21/43072—Synchronising the rendering of multiple content streams or additional data on devices, e.g. synchronisation of audio on a mobile phone with the video output on the TV screen of multiple content streams on the same device
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
    • H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F12/00—Accessing, addressing or allocating within memory systems or architectures
    • G06F12/02—Addressing or allocation; Relocation
    • G06F12/04—Addressing variable-length words or parts of words
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding

Definitions

  • the present invention relates generally to a new and improved system for decoding a plurality of audio and video signals and, more particularly, to a new and improved system for decoding a plurality of MPEG audio and video signals.
  • a serial pipeline processing system of the present invention comprises a single two-wire bus used for carrying unique and specialized interactive interfacing tokens, in the form of control tokens and data tokens, to a plurality of adaptive decompression circuits and the like positioned as a reconfigurable pipeline processor.
  • U.S. Pat. No. 5,111,292 discloses an apparatus for encoding/decoding a HDTV signal for e.g. terrestrial transmission includes a priority selection processor for parsing compressed video codewords between high and low priority channels for transmission.
  • a compression circuit responsive to high definition video source signals provides hierarchically layered codewords CW representing compressed video data and associated codewords T defining the types of data represented by codewords CW.
  • the priority selection processor responsive to the codewords CW and T, counts the number of bits in predetermined blocks of data and determines the number of bits in each block to be allocated to the respective channels. Thereafter the processor parses the codewords CW into high and low priority codeword sequences wherein the high and low priority codeword sequences correspond to compressed video data of relatively greater and lesser importance to image reproduction respectively.
  • the apparatus comprises a plurality of compute modules, in a preferred embodiment, for a total of four compute modules coupled in parallel.
  • Each of the compute modules has a processor, dual port memory, scratch-pad memory, and an arbitration mechanism.
  • a first bus couples the compute modules and a host processor.
  • the device comprises a shared memory which is coupled to the host processor and to the compute modules with a second bus.
  • U.S. Pat. No. 4,785,349 discloses a full motion color digital video signal that is compressed, formatted for transmission, recorded on compact disc media and decoded at conventional video frame rates.
  • regions of a frame are individually analyzed to select optimum fill coding methods specific to each region.
  • Region decoding time estimates are made to optimize compression thresholds.
  • Region descriptive codes conveying the size and locations of the regions are grouped together in a first segment of a data stream.
  • Region fill codes conveying pixel amplitude indications for the regions are grouped together according to fill code type and placed in other segments of the data stream.
  • the data stream segments are individually variable length coded according to their respective statistical distributions and formatted to form data frames.
  • a decoder includes a variable length decoder responsive to statistical information in the code stream for separately variable length decoding individual segments of the data stream.
  • Region location data is derived from region descriptive data and applied with region fill codes to a plurality of region specific decoders selected by detection of the fill code type (e.g., relative, absolute, dyad and DPCM) and decoded region pixels are stored in a bit map for subsequent display.
  • U.S. Pat. No. 4,922,341 discloses a method for scene-model-assisted reduction of image data for digital television signals, whereby a picture signal supplied at time is to be coded, whereby a predecessor frame from a scene already coded at time t-1 is present in an image store as a reference, and whereby the frame-to-frame information is composed of an amplification factor, a shift factor, and an adaptively acquired quad-tree division structure.
  • a uniform, prescribed gray scale value or picture half-tone expressed as a defined luminance value is written into the image store of a coder at the transmitter and in the image store of a decoder at the receiver store, in the same way for all picture elements (pixels).
  • Both the image store in the coder as well as the image store in the decoder are each operated with feed back to themselves in a manner such that the content of the image store in the coder and decoder can be read out in blocks of variable size, can be amplified with a factor greater than or less than 1 of the luminance and can be written back into the image store with shifted addresses, whereby the blocks of variable size are organized according to a known quad tree data structure.
  • U.S. Pat. No. 5,122,875 discloses an apparatus for encoding/decoding an HDTV signal.
  • the apparatus includes a compression circuit responsive to high definition video source signals for providing hierarchically layered codewords CW representing compressed video data and associated codewords T, defining the types of data represented by the codewords CW.
  • a priority selection circuit responsive to the codewords CW and T, parses the codewords CW into high and low priority codeword sequences wherein the high and low priority codeword sequences correspond to compressed video data of relatively greater and lesser importance to image reproduction respectively.
  • a transport processor responsive to the high and low priority codeword sequences, forms high and low priority transport blocks of high and low priority codewords, respectively.
  • Each transport block includes a header, codewords CW and error detection check bits.
  • the respective transport blocks are applied to a forward error check circuit for applying additional error check data. Thereafter, the high and low priority data are applied to a modem wherein quadrature amplitude modulates respective carriers for transmission.
  • U.S. Pat. No. 5,146,325 discloses a video decompression system for decompressing compressed image data wherein odd and even fields of the video signal are independently compressed in sequences of intraframe and interframe compression modes and then interleaved for transmission. The odd and even fields are independently decompressed. During intervals when valid decompressed odd/even field data is not available, even/odd field data is substituted for the unavailable odd/even field data. Independently decompressing the even and odd fields of data and substituting the opposite field of data for unavailable data may be used to advantage to reduce image display latency during system start-up and channel changes.
  • U.S. Pat. No. 5,168,356 discloses a video signal encoding system that includes apparatus for segmenting encoded video data into transport blocks for signal transmission.
  • the transport block format enhances signal recovery at the receiver by virtue of providing header data from which a receiver can determine re-entry points into the data stream on the occurrence of a loss or corruption of transmitted data.
  • the re-entry points are maximized by providing secondary transport headers embedded within encoded video data in respective transport blocks.
  • U.S. Pat. No. 5,168,375 discloses a method for processing a field of image data samples to provide for one or more of the functions of decimation, interpolation, and sharpening. This is accomplished by an array transform processor such as that employed in a JPEG compression system. Blocks of data samples are transformed by the discrete even cosine transform (DECT) in both the decimation and interpolation processes, after which the number of frequency terms is altered. In the case of decimation, the number of frequency terms is reduced, this being followed by inverse transformation to produce a reduced-size matrix of sample points representing the original block of data.
  • DECT discrete even cosine transform
  • U.S. Pat. No. 5,175,617 discloses a system and method for transmitting logmap video images through telephone line band-limited analog channels.
  • the pixel organization in the logmap image is designed to match the sensor geometry of the human eye with a greater concentration of pixels at the center.
  • the transmitter divides the frequency band into channels, and assigns one or two pixels to each channel, for example a 3 KHz voice quality telephone line is divided into 768 channels spaced about 3.9 Hz apart.
  • Each channel consists of two carrier waves in quadrature, so each channel can carry two pixels.
  • Some channels are reserved for special calibration signals enabling the receiver to detect both the phase and magnitude of the received signal.
  • An FFT algorithm implements a fast discrete approximation to the continuous case in which the receiver synchronizes to the first frame and then acquires subsequent frames every frame period. The frame period is relatively low compared with the sampling period so the receiver is unlikely to lose frame synchrony once the first frame is detected.
  • An experimental video telephone transmitted 4 frames per second, applied quadrature coding to 1440 pixel logmap images and obtained an effective data transfer rate in excess of 40,000 bits per second.
  • U.S. Pat. No. 5,185,819 discloses a video compression system having odd and even fields of video signal that are independently compressed in sequences of intraframe and interframe compression modes.
  • the odd and even fields of independently compressed data are interleaved for transmission such that the intraframe even field compressed data occurs midway between successive fields of intraframe odd field compressed data.
  • the interleaved sequence provides receivers with twice the number of entry points into the signal for decoding without increasing the amount of data transmitted.
  • U.S. Pat. No. 5,212,742 discloses an apparatus and method for processing video data for compression/decompression in real-time.
  • the apparatus comprises a plurality of compute modules, in a preferred embodiment, for a total of four compute modules coupled in parallel.
  • Each of the compute modules has a processor, dual port memory, scratch-pad memory, and an arbitration mechanism.
  • a first bus couples the compute modules and host processor.
  • the device comprises a shared memory which is coupled to the host processor and to the compute modules with a second bus. The method handles assigning portions of the image for each of the processors to operate upon.
  • U.S. Pat. No. 5,231,484 discloses a system and method for implementing an encoder suitable for use with the proposed ISO/IEC MPEG standards. Included are three cooperating components or subsystems that operate to variously adaptively pre-process the incoming digital motion video sequences, allocate bits to the pictures in a sequence, and adaptively quantize transform coefficients in different regions of a picture in a video sequence so as to provide optimal visual quality given the number of bits allocated to that picture.
  • U.S. Pat. No. 5,267,334 discloses a method of removing frame redundancy in a computer system for a sequence of moving images.
  • the method comprises detecting a first scene change in the sequence of moving images and generating a first keyframe containing complete scene information for a first image.
  • the first keyframe is known, in a preferred embodiment, as a "forward-facing" keyframe or intraframe, and it is normally present in CCITT compressed video data.
  • the process then comprises generating at least one intermediate compressed frame, the at least one intermediate compressed frame containing difference information from the first image for at least one image following the first image in time in the sequence of moving images. This at least one frame being known as an interframe.
  • detecting a second scene change in the sequence of moving images and generating a second keyframe containing complete scene information for an image displayed at the time just prior to the second scene change known as a "backward-facing" keyframe.
  • the first keyframe and the at least one intermediate compressed frame are linked for forward play, and the second keyframe and the intermediate compressed frames are linked in reverse for reverse play.
  • the intraframe may also be used for generation of complete scene information when the images are played in the forward direction.
  • the backward-facing keyframe is used for the generation of complete scene information.
  • U.S. Pat. No. 5,276,513 discloses a first circuit apparatus, comprising a given number of prior-art image-pyramid stages, together with a second circuit apparatus, comprising the same given number of novel motion-vector stages, perform cost-effective hierarchical motion analysis (HMA) in real-time, with minimum system processing delay and/or employing minimum system processing delay and/or employing minimum hardware structure.
  • HMA hierarchical motion analysis
  • the first and second circuit apparatus in response to relatively high-resolution image data from an ongoing input series of successive given pixel-density image-data frames that occur at a relatively high frame rate (e.g., 30 frames per second), derives, after a certain processing-system delay, an ongoing output series of successive given pixel-density vector-data frames that occur at the same given frame rate.
  • a relatively high frame rate e.g. 30 frames per second
  • Each vector-data frame is indicative of image motion occurring between each pair of successive image frames.
  • U.S. Pat. No. 5,283,646 discloses a method and apparatus for enabling a real-time video encoding system to accurately deliver the desired number of bits per frame, while coding the image only once, updates the anqutization step size used to quantize coefficients which describe, for example, an image to be transmitted over a communications channel.
  • the data is divided into sectors, each sector including a plurality of blocks.
  • the blocks are encoded, for example, using DCT coding, to generate a sequence of coefficients for each block.
  • the coefficients can be quantized, and depending upon the quantization step, the number of bits required to describe the data will vary significantly.
  • the accumulated actual number of bits expended is compared with the accumulated desired number of bits expended, for a selected number of sectors associated with the particular group of data.
  • the system then readjusts the quantization step size to target a final desired number of data bits for a plurality of sectors, for example describing an image.
  • Various methods are described for updating the quantization step size and determining desired bit allocations.
  • U.S. Pat. No. 5,287,420 discloses a method and apparatus for image compression suitable for personal computer applications, which compresses and stores data in two steps.
  • An image is captured in realtime and compressed using an efficient method and stored to a hard-disk.
  • the data is further compressed in non-realtime using a computationally more intense algorithm that results in a higher compression ratio.
  • the two-step approach allows the storage reduction benefits of a highly sophisticated compression algorithm to be achieved without requiring the computational resources to perform this algorithm in realtime.
  • a compression algorithm suitable for performing the first compression step on a host processor in a personal computer is also described.
  • the first compression step accepts 4:2:2 YCrCb data from the video digitizer.
  • the two chrominance components are averaged and a pseudo-random number is added to all components.
  • the resulting values are quantized and packed into a single 32-bit word representing a 2 ⁇ 2 array of pixels.
  • the seed value for the pseudo-random number is remembered so that the pseudo-random noise can be removed before performing the second compression step
  • the system contains a plurality of identical flow processors connected in a ring fashion.
  • the tokens contain a data field, a control field and a tag.
  • the tag field of the token is further broken down into a processor address field and an identifier field.
  • the processor address field is used to direct the tokens to the correct data-flow processor, and the identifier field is used to label the data such that the data-flow processor knows what to do with the data. In this way, the identifier field acts as an instruction for the data-flow processor.
  • the system directs each token to a specific data-flow processor using a module number (MN). If the MN matches the MN of the particular stage, then the appropriate operations are performed upon the data. If unrecognized, the token is directed to an output data bus.
  • MN module number
  • the asynchronous pipeline comprises a plurality of pipeline stages. Each of the pipeline stages consists of a group of input data latches followed by a combinatorial logic circuit that carries out logic operations specific to the pipeline stages.
  • the data latches are simultaneously supplied with a triggering signal generated by a data-transfer control circuit associated with that stage.
  • the data-transfer control circuits are interconnected to form a chain through which send and acknowledge signal lines control a hand-shake mode of data transfer between the successive pipeline stages.
  • a decoder is generally provided in each stage to select operations to be done on the operands in the present stage. It is also possible to locate the decoder in the preceding stage in order to pre-decode complex decoding processing and to alleviate critical path problems in the logic circuit.
  • the elastic nature of the pipeline eliminates any centralized control since all the interworkings between the submodules are determined by a completely localized decision and, in addition, each submodule can autonomously perform data buffering and self-timed data-transfer control at the same time.
  • empty stages are interleaved between the occupied stages in order to ensure reliable data transfer between the stages.
  • the present invention embodies a method of asynchronously accessing a random access memory having a plurality of rows and columns, where each row has a wordline connected to read and write row decoders and each column is connected to bitlines.
  • a row address is assigned to a first and second row, i.e. a pair of rows.
  • a read address is provided to the read row decoder and a write address is provided to the write row decoder.
  • the read and write addresses are decoded by the read and write row decoders, respectively, and one of the first or second rows is selected for reading.
  • Asynchronous with selecting one of the first or second rows for reading one of the first or second rows is selected for writing.
  • Data is then read from the row selected for reading and asynchronously data is written into the row selected for writing.
  • Signals are provided which coordinate the reading and writing of data, where in the event reading or writing is being performed, another of the reading or writing is deferred until completion of the first reading or writing.
  • the read row decoder and the write decoder are unable to select the same row simultaneously and a read control circuit and a write control circuit are unable to select the same bit lines simultaneously.
  • FIG. 1 illustrates data flow through a preferred embodiment in the present invention
  • FIG. 2 shows an example of a 13 bit word used to address 8 bit data in a 64 ⁇ 32 RAM
  • FIG. 3 is a functional block diagram of a Register file in the present invention.
  • FIG. 4 illustrates data flow in a register file as shown in FIG. 3;
  • FIG. 5 is a block diagram illustrating register file address decoding, in accordance with the present invention.
  • FIG. 6 is a block diagram of a Microcodable State Machine, in accordance with the present invention.
  • FIG. 8 is a block diagram of one example of an Arithmetic Core in accordance with the present invention.
  • FIG. 9 illustrates the basis steps in a method, in accordance with the present invention, for performing an IDCT on input data
  • FIG. 10 is a block diagram illustrating the combined, simplified, two-stage architecture of an IDCT system, in accordance with the present invention.
  • Figure 11 is a simplified block diagram of an integrated circuit that comprises the main system components of the IDCT shown in FIG. 10;
  • FIG. 12a and FIG. 12b taken together are a block diagram of a pre-processing circuit corresponding to one of the main system component; for ease of explanation, these figures are referred collectively as FIG. 12;
  • FIG. 13a, FIG. 13b and FIG. 13c depict timing diagrams which illustrate the relationships between timing and control signals in the IDCT system of a preferred embodiment
  • FIG. 14a and FIG. 14b taken together are a block diagram of a common processing circuit in the IDCT system for ease of explanation, these figures are referred to collectively as FIG. 14;
  • FIG. 15a, FIG. 15b, FIG. 15c and FIG. 15d taken together are a block diagram of a post-processing circuit which corresponds to another main component of the system and are referred collectively as FIG. 15;
  • FIGS. 16a and 16b are a block diagram, in accordance with the present invention illustrating an IDCT having a twin data stream, a transpose RAM and an improved buffer and are collectively FIG. 16.
  • FIGS. 17a, 17b, 17c, 17d, 17e, 17f are a block diagram showing in further detail the 1-D IDCT system shown in FIGS. 16a and 16b and are collectively FIG. 17;
  • FIGS. 18a and 18b are a block diagram showing greater detail of the transform system as shown in FIGS. 17a-17f and are collectively FIG. 18;
  • FIGS. 19a and 19b are a block diagram showing in greater detail the input buffer shown in FIGS. 18a and 18b and are collectively FIG. 19;
  • FIGS. 20a and 20b are a simplified block diagram of a pre-processing circuit "PREC", in accordance with the present invention and are collectively FIG. 20;
  • FIGS. 21a and 21b are a block diagram illustrating a common processing circuit "CBLK" found in the IDCT and are collectively FIG. 21;
  • FIGS. 22a and 22b are a block diagram of a post-processing circuit "POSTC" and are collectively FIG. 22;
  • FIGS. 23a, 23b, 23c, 23d are another illustration of the post-processing circuit shown in FIGS. 22a and 22b and are collectively FIG. 23;
  • FIG. 24 is a block diagram depicting a round and saturate block, in accordance with the present invention.
  • FIGS. 25a and 25b are a block diagram of an output buffer in the present invention and are collectively FIG. 25;
  • FIGS. 26a and 26b are a block diagram of a control shift register, in accordance with the present invention.
  • FIGS. 27a, 27b, 27c are a block diagram of a control shift register decode in the present invention.
  • FIGS. 28a, 28b, 28c are depict a control shift register and an input control buffer
  • FIGS. 29a-1, 29a-2, 29b, 29c, 29d, 29e, 29f (collectively FIG. 29) illustrate a control circuit for a T2 data stream
  • FIGS. 30a, 30b, 30c, 30d show data in a counter for a T1 data stream
  • FIGS. 31a, 31b, 31c, 31d, 31e depict data in a counter for a T2 data stream in the present invention
  • FIG. 32 is a timing diagram showing the initialization of the IDCT and associated circuitry
  • FIG. 33 is a timing diagram showing the interleaving of T1 and T2 data
  • FIG. 34 is a timing diagram illustrating slippage and recovery of T2 data
  • FIG. 35 is a timing diagram depicting a flushing operation of the IDCT and associated circuitry in the present invention.
  • FIG. 36 illustrates start-up of the system, in accordance with the present invention.
  • FIG. 37 depicts slippage and recovery in the early stages of interleaving T1 and T2 data
  • FIG. 38 illustrates another preferred embodiment of the IDCT system shown in FIGS. 16a through 37;
  • FIG. 39 shows MPEG information streams being demultiplexed, in accordance with the present invention, into elementary streams containing data and timestamp information
  • FIG. 40 depicts a first embodiment of an elementary stream timestamp error determination and time synchronization system, in accordance with the present invention
  • FIG. 41 illustrates a second embodiment of an elementary stream timestamp error determination and time synchronization system, in accordance with the present invention
  • FIG. 42 shows a third embodiment of an elementary stream timestamp error determination and time synchronization system, in accordance with the present invention.
  • FIG. 43 depicts a first embodiment of a video timestamp error determination and time synchronization system, in accordance with the present invention.
  • FIG. 44 illustrates a second embodiment of a video timestamp error determination and time synchronization system, in accordance with the present invention.
  • FIG. 45 shows the second embodiment of a video timestamp error determination and time synchronization system as shown in FIG. 44 and operating at 30 Hz;
  • FIG. 46 shows timestamp information flow through the system of the present invention
  • FIG. 47 is a block diagram illustrating synchronization time information being processed by a microprogrammable state machine
  • FIG. 48 is a block diagram illustrating a first preferred embodiment of the present invention.
  • FIG. 49 is another block diagram illustrating the first preferred embodiment of the present invention.
  • FIG. 50 depicts a second preferred embodiment of the present invention.
  • FIG. 51 illustrates a detailed method of addressing used by the second preferred embodiment, in accordance with the present invention.
  • FIG. 52 is a block diagram showing an apparatus for decoding Huffman VLCs, in accordance with the present invention.
  • FIGS. 53a, 53b, 53c, 53d are collectively FIG. 53 and are a schematic diagram showing the overall structure of the parallel huffman decoder of the present invention.
  • FIGS. 54a and 54b are collectively FIG. 54 and are a schematic diagram illustrating a ROM adapted for decoding parallel huffman codes
  • FIG. 55 illustrates a first embodiment of a ROM adapted for decoding parallel huffman codes
  • FIG. 56 illustrates a second embodiment of a ROM adapted for decoding parallel huffman codes
  • FIG. 57 depicts a third embodiment of a ROM adapted for decoding parallel huffman codes
  • FIG. 58 is a block diagram illustrating the primary system component of one embodiment of the present invention.
  • FIG. 59 is a block diagram depicting the start code detector of the present invention.
  • FIG. 60 is a block diagram showing the parser of the present invention.
  • FIG. 61 is a block diagram depicting the primary components of the spatial processing circuitry of the present invention.
  • FIG. 62 is a block diagram illustrating the display circuitry, in accordance with the present invention.
  • FIG. 63 illustrates one embodiment of timestamp management, in accordance with the present invention.
  • FIG. 64 shows another embodiment of timestamp management in the present invention.
  • FIG. 65 is a block diagram depicting the hardware components of the system of the present invention.
  • FIG. 66 is a block diagram providing an overview of the system components of the microcontroller of the present invention.
  • FIG. 67 is a simplified diagram illustrating the Arithmetic core of the present invention.
  • FIG. 68 illustrates the ALU of the present invention
  • FIG. 69 depicts a register file, in accordance with the present invention.
  • FIG. 70 illustrates the writing to independent bus registers in the present invention
  • FIG. 83 illustrates an additional mode of prediction filtering
  • FIG. 84 shows still another prediction mode
  • FIG. 85 illustrates yet another prediction mode, in accordance with the present invention.
  • FIG. 86 shows another prediction mode of the present invention.
  • FIG. 87 is a block diagram illustrating the organization of the various system components of the display system of the present invention.
  • FIG. 88 depicts a 4:3 filtering operation
  • FIG. 89 depicts a 3:2 filtering operation
  • FIG. 90 illustrates a 2:1 filtering operation of the present invention
  • FIG. 91 shows a three tap filter used in the present invention
  • FIG. 92 illustrates the repetition of erroneous pels
  • FIG. 93 depicts the filed -- id signal of the present invention.
  • FIG. 94 shows the horizontal timing points (cycles), in accordance with the present invention.
  • FIG. 95 illustrates the PAL vertical timing at 625 lines per field, in accordance with the present invention.
  • FIG. 96 illustrates the NTSCV vertical timing at 525 lines per field, in accordance with the present invention.
  • FIG. 97 shows a horizontal counting machine, in accordance with the present invention.
  • FIG. 98 illustrates border generation in the present invention
  • FIG. 99 depicts picture cropping, in accordance with the present invention.
  • FIG. 100 is a block diagram illustrating the present invention as a chip
  • FIG. 101 illustrates the sysclock requirements of the present invention
  • FIG. 102 depicts the two-wire protocol on a coded data interface, in accordance with the present invention.
  • FIG. 103 shows a DATA token of the present invention
  • FIG. 104 shows a FLUSH token of the present invention
  • FIG. 105 illustrates the timing of the coded data interface
  • FIG. 106 depicts using non-even mark-space ratio CDCLOCK, in accordance with the present invention.
  • FIG. 107 shows output timing in 16 bit mode in the present invention
  • FIG. 108 illustrates output timing in 8 bit mode in the present invention
  • FIG. 109 shows the timing of the video output interface in the present invention.
  • FIG. 110 depicts video output mode signals, in accordance with the present invention.
  • FIG. 111 shows horizontal timing in the present invention
  • FIGS. 112a and 112b (collectively FIG. 112) show the vertical timing for a 525 line system
  • FIGS. 113a and 113b depict the vertical timing for a 625 line system
  • FIG. 114 illustrates the sync and blanking signals for a 525 line system, in accordance with the present invention
  • FIG. 115 shows the sync and blanking signals for a 625 line system, in accordance with the present invention.
  • FIG. 116 illustrates a zero SDRAM connection configuration in the present invention
  • FIG. 117 shows one SDRAM connection configuration in the present invention
  • FIG. 118 depicts a two SDRAM connection configuration, in accordance with the present invention.
  • FIG. 119 illustrates a three SDRAM connection configuration
  • FIG. 120 is a flow chart depicting the flag -- picture -- end operation, in accordance with the present invention.
  • FIG. 121 is a flow chart showing the start -- code -- search operation, in accordance with the present invention.
  • FIG. 122 shows timestamp modification, in accordance with the present invention
  • FIG. 123 illustrates the read timing for the microprocessor interface
  • FIG. 124 shows the write timing for the microprocessor interface.
  • BLOCK An 8-row by column matrix of pels, or 64 DCT coefficients (source, quantized or dequantized).
  • CHROMINANCE A matrix, block or single pel representing one of the two color difference signals related to the primary colors in the manner defined in the bit stream.
  • the symbols used for the color difference signals are Cr and Cb.
  • CODED REPRESENTATION A data element as represented in its encoded form.
  • CODED VIDEO BIT STREAM A coded representation of a series of one or more pictures as defined in this specification.
  • CODED ORDER The order in which the pictures are transmitted and decoded. This order is not necessarily the same as the display order.
  • COMPONENT A matrix, block or single pel from one of the three matrices (luminance and two chrominance) that make up a picture.
  • COMPRESSION Reduction in the number of bits used to represent an item of data.
  • DECODER An embodiment of a decoding process.
  • DECODING PROCESS: The process defined in this specification that reads an input coded bitstream and produces decoded pictures or audio samples.
  • DISPLAY ORDER The order in which the decoded pictures are displayed. Typically, this is the same order in which they were presented at the input of the encoder.
  • ENCODING A process, not specified in this specification, that reads a stream of input pictures or audio samples and produces a valid coded bitstream as defined in this specification.
  • INTRA CODING Coding of a macroblock or picture that uses information only from that macroblock or picture.
  • LUMINANCE A matrix, block or single pel representing a monochrome representation of the signal and related to the primary colors in the manner defined in the bit stream.
  • the symbol used for luminance is Y.
  • MACROBLOCK The four 8 by 8 blocks of luminance data and the two (for 4:2:0 chroma format) four (for 4:2:2 chroma format) or eight (for 4:4:4 chroma format) corresponding 8 by 8 blocks of chrominance data coming from a 16 by 16 section of the luminance component of the picture.
  • Macroblock is sometimes used to refer to the pel data and sometimes to the coded representation of the pet values and other data elements defined in the macroblock header of the syntax defined in this part of this specification. To one of ordinary skill in the art, the usage is clear from the context.
  • MOTION COMPENSATION The use of motion vectors to improve the efficiency of the prediction of pet values.
  • the prediction uses motion vectors to provide offsets into the past and/or future reference pictures containing previously decoded pel values that are used to form the prediction error signal.
  • MOTION VECTOR A two-dimensional vector used for motion compensation that provides an offset from the coordinate position in the current picture to the coordinates in a reference picture.
  • NON-INTRA CODING Coding of a macroblock or picture that uses information both from itself and from macroblocks and pictures occurring at other times.
  • a source or reconstructed picture consists of three rectangular matrices of 8-bit numbers representing the luminance and two chrominance signals.
  • a picture is identical to a frame, while for interlaced video, a picture can refer to a frame, or the top field or the bottom field of the frame depending on the context.
  • PREDICTION The use of a predictor to provide an estimate of the pel value or data element currently being decoded.
  • RECONFIGURABLE PROCESS STAGE A stage, which in response to a recognized token, reconfigures itself to perform various operations.
  • TOKEN A universal adaptation unit in the form of an interactive interfacing messenger package for control and/or data functions.
  • START CODES 32-bit codes embedded in a coded bitstream that are unique. They are used for several purposes including identifying some of the structures in the coding syntax.
  • VLC A reversible procedure for coding that assigns shorter code-words to frequent events and longer code-words to less frequent events.
  • VIDEO SEQUENCE A series of one or more pictures.
  • FIG. 1 of the drawings the data flow through the preferred embodiment 200 of the invention is shown.
  • the embodiment of the present invention is preferably implemented using a two-wire pipeline system having various control and DATA tokens.
  • the major elements of the system are a Start Code Detector 201, a Video Parser 202 incorporating a Huffman Decoder 203 and a Microprogrammable State Machine (MSM) 204, an Inverse Discrete Cosine Transform (IDCT) 205, a synchronous DRAM controller 206 with an associated address generation unit 207, appropriate prediction circuitry 208 and display circuitry 209 which includes upsampling 210 and 211 and video timing generation 212.
  • MCM Microprogrammable State Machine
  • IDCT Inverse Discrete Cosine Transform
  • the present invention provides for deferring variable width bit fields with fixed width words. More particularly, the present invention provides a method of addressing variable width data with a fixed width word.
  • variable bit field is used to specify bits to be substituted into the word or to specify an unused portion of the word in addressing variable width data with a fixed width word.
  • the system of the present invention includes a microcodable state machine having an arithmetic core.
  • the microcodable state machine is intended to be used for solving design problems where there is a need for versatile and/or complicated calculations. Examples of such designs include address generation, stream parsing and decoding, and filter tap coefficient calculations.
  • the addressing must cope with two different features: (1) variable length addresses to access varying width portions of words and (2) address substitution.
  • a RAM having a 64 ⁇ 32 bit configuration can be addressed in partial words having 64 ⁇ 32 bit, 128 ⁇ 16 bit, 256 ⁇ 8 bit, 512 ⁇ 4 bit, 1024 ⁇ 2 bit, or 2048 ⁇ 1 bit formats.
  • variable portions of a word to be known as fields
  • actions such as substitution, variable width data addressing, or the constriction of other parts of the word.
  • the conventional method for defining variable portions of words is to have an additional word (or words) which specify the width of the field (or fields) within the word.
  • a method for encoding this information within the word itself is described.
  • the present method has the advantages of savings bits in the overall definition of the word, simplifying decoding of the encoded word and providing a more intuitive view of what has been encoded.
  • this encoding method is applicable if the variable width fields are most or least significant bit justified within the word.
  • Table 1 shows two examples of variable width fields (marked “F") that are least significant bit justified defined within an eight bit word.
  • a "w” marks other potential fields of these words.
  • Table 2 shows the conventional method of encoding the fields shown in Table 1 using sufficient additional bits to specify the maximum width of the field in binary. (Bits marked “x” "don't care”, i.e., their value is of no consequence. This method is clearly inefficient in its use of bits and, furthermore, provides a less intuitive form than that described in the present invention.
  • the new method defines the field within the word.
  • This method defines the field by using a continuation marker and a termination marker.
  • the field is specified, from one end of the field, as a series of continuation markers followed by a termination marker. In the case of a zero length field, however, only a termination marker is provided at the end of the word.
  • Both the continuation marker and the termination marker are single bits, and they must be complementary.
  • the field must be justified to either end of the word. Accordingly, the method of the present invention for encoding fields requires a width of only one bit extra over the original word width.
  • the encoding is in a more intuitive form allowing the field defined to be more easily identified.
  • this encoding method of the present invention can also be used such that the termination marker and the continuation marker are inverted to provide that the encoding of Table 3 resembles that of Table 4.
  • the use of "1" or "0" is used interchangeably throughout this application.
  • Table 5 illustrates most significant justified fields, i.e., these are encoded in a similar way to least significant bit justified fields except that the field reaches from the most significant bit (hereinafter MSB) towards the least significant bit (hereinafter "LSB") up to and including the first termination marker.
  • MSB most significant bit
  • LSB least significant bit
  • fields may be encoded from the least significant and most significant ends of the word simultaneously.
  • the two fields shown in Table 7 may be encoded as in Table 8, with the addition of just one bit for each field as described previously.
  • Table 9 shows the encoding for substitution into a 12 bit address.
  • One embodiment of the present invention is for addressing a memory which can be accessed at its full width or in 2 n widths up to its full width (these smaller words are called partial words). Hence, it will be shown how the variable field encoding of the present invention can be used to address this memory and to index those addresses into the memory.
  • the implementation of this embodiment is a 64 ⁇ 32 bit memory which can be accessed as 64 ⁇ 32 bits, 128 ⁇ 16 bits, 256 ⁇ 8 bits, 512 ⁇ 4 bits, 1024 ⁇ 2 bits, or 2048 ⁇ 1 bit. It is seen that 5 bits are required to address one of the 64 ⁇ 32 bit locations, while 12 bits are required to address one of the 2048 ⁇ 1 bit locations.
  • the addresses can be of variable length and, in fact, the width of the address specifies the address format of the memory. Accordingly, the address can be defined within a fixed word width by using a most significant justified variable width field which constricts the address and defines its width. This is illustrated in Table 10.
  • substitution portion (or field) of the address can be defined by a least significant bit justified variable length field (The continuation marker "1"; termination marker "0") that is superimposed on top of those shown in Table 10.
  • Table 11 shows how to define the number of the least significant bits to be substituted.
  • the least significant bit added is the substitution indicator (marked "w").
  • FIG. 2 The general case of a Fixed width word for substitution is shown in FIG. 2.
  • the conventional memory circuitry dictates that the memory must always be accessed at it full width.
  • a full (32 bit) width word is read. This full word is rotated until the partial word accessed is justified in the LSB. The upper parts of the word are extended to the full width and then output. Extending may encompass padding with zeros or ones, sign extending, using the sign bit of a sign-magnitude number as the new MSB or any similar conventional method. Extending is dependent on the mode of operation.
  • FIG. 3 shows these steps for the access of a 4 bit partial word in the fourth four bit word of the 32 bit word.
  • the full width word must be rotated to place the partial word at the LSB, as shown in row “2" 214.
  • the four bit word is extended to create a full 32 bit word: This word can now be accessed.
  • a full width word that has been selected to be written back is truncated to the width of the original partial word which is multiplexed into the word shown in row "2" 214. At the LSB position, this is shown in row “4" 216. The resulting word is rotated back in its original significance in the read word, this is shown in row "5" 217. This full word can now be written back into the register file.
  • the inputted partial word is multiplexed into rotated full word from (2);
  • the memory address must be decoded to control the above structure. It should be recognized that the MSB of any width of address is at the same significance with reference to the memory.
  • the top six bits of a decoded address are a 32 bit word address, whereas the remainder is a bit address. Therefore, the stage of decoding (in parallel with the substitution) is to decode the address width defining variable field by detecting the position of the most significant termination marker. This allows the address to be MSB justified (shifting in zeros at the LSB).
  • the top six bits can be used directly as a 32 bit word row address of the memory.
  • the bottom five bits can be used to directly control both barrel shifters (as seen in FIG.
  • a 16 bit address can have a shift of 0bx0000, i.e., 0 or 16 bit shift and a 1 bit address can have a shift of 0bxx, i.e., 0 to 31 bit shifts.
  • the extender and input multiplexer are controlled by the access width decode to mask out the output words and multiplex the input words to an appropriate significance, respectively.
  • the block diagram of the decode is shown in FIG. 5. It can be seen that the decode of the two variable width fields for width and substitution can be done in parallel and independently.
  • FIG. 2 illustrates an example of a fixed width word 13 bits long for addressing variable width data and substitution as shown in the bottom two rows. For these examples, an eight bit word would have been addressed at location 0b110lssss, where "ssss" is substituted from another address source.
  • the substitution into a memory address and the variable width accessing of a memory have been brought together in the implementation of a microcodable state machine the structure of which is shown in FIG. 6.
  • the structure is one of a state machine 218 providing control of an arithmetic core 219 by way of a wide word of control signals called a microcode instruction.
  • the arithmetic core 219 passes status flags and some data to the state machine 218.
  • the jump address is in the form shown in FIG. 7.
  • the substituted value comes from the Arithmetic core 219 as shown in FIGS. 6 and 8. This allows the construction of "jump tables" within the microcode programs. Thus, if a jump is made with 3 bits substituted, for example, there are eight possible contiguous locations that may be jumped to, each dependent on the value from the arithmetic core, i.e., it has so become a programmable jump.
  • the arithmetic core 219 includes a memory called a register file 221, an Arithmetic and Logic unit (ALU) 222, an input port 223 and an output port 224. These components are connected via buses and multiplexers. As previously stated, these components, and the multiplexers defining their connections, are entirely controlled by the microcode instruction issued by the state machine 218.
  • the ALU 222 and the ports 223 and 224 are conventional, however, the register file 221 is a memory which allows variable width indexed accesses. The addresses to the register file 221 is coded directly into the microcode instruction.
  • a procedure for addressing memory having the following steps: (1) providing a fixed width word having a predetermined fixed number of bits to be used for addressing variable width data; (2) defining the fixed width word with a width defining field and an address field providing the width defining field with at least one bit to serve as a termination marker; (3) defining the address field with a plurality of bits defining the address of the data; and (4) varying the size of bits in the address field in inverse relation to the size of the variable width data varying the number of bits in the width defining field in direct relation to the size of the variable width data and maintaining a fixed width word for addressing variable width data while varying the width of the width defining field and the address field.
  • a procedure for addressing memory having the following steps is described: (1) providing a fixed width word having a predetermined fixed number of bits to be used for addressing data; (2) defining the fixed width word with an address field and a substitution field; (3) defining the address field with a plurality of bits defining the address of the data; (4) defining a variable width substitution field with at least one substitution bit; (5) the substitution field has at least one bit to serve as a termination marker between the address field and the substitution field; and (6) using the substitution field to indicate substituted bits from a separate addressing source and maintaining a fixed width word for addressing variable width data while inversely varying the width of the address field and the width of the substitution field.
  • a process for addressing variable width data in a memory is described as having the following steps: (1) providing a memory having words of predetermined width and composed of partial words; (2) rotating the partial word to be accessed to a least significant bit justification; (3) extending the remaining part of the word so that the accessed word will be recognized as the partial word; and (4) restoring the remaining part of the word and rotating the word until the partial word is restored to its original position.
  • This present embodiment in accordance with the present invention, relates to a method for the transformation of signals from a frequency to a time representation, as well as a digital circuit arrangement for implementing the transformation.
  • a telegraph wire is, for example, typically a much faster medium for transmitting information than the mail is, even though it might be faster to type and read a mailed document than to tap out a telegraph key.
  • the method of encoding transmitted information also limits the speed at which information can be conveyed.
  • a long-winded telegraph message will, for example, take longer to convey than a succinct message with the same information content.
  • the greatest transmission and reception speed can therefore be obtained by compressing the data to be transmitted as much as possible, and then, using a high-speed transmission medium, to process the data at both ends as fast as possible, which often means the reduction or elimination of ⁇ bottlenecks ⁇ in the system.
  • a digital television transmission system generates a digital representation of an image by conveying analog signals into binary ⁇ numbers ⁇ corresponding to luminance and color values for the pixels.
  • Modem digital encoding schemes and hardware structures typically enable much higher information transmission rates than do conventional analog transmission systems. As such, digital televisions are able to achieve much higher resolution and much more life-like images than their conventional analog counterparts.
  • HDTV High-Definition TV
  • the eye is insensitive to high-frequency components (or edges) in a picture.
  • Information concerning the highest frequencies can usually be omitted altogether without the human viewer noticing any significant reduction in image quality.
  • the 8-by-8 pixel block containing spatial information (for example, the actual values for luminance) must be transformed in some manner to obtain frequency information.
  • the JPEG, MPEG and H.261 standards all use the known Discrete Cosine Transform to operate on the 8-by-8 spatial matrix to obtain an 8-by-8 frequency matrix.
  • the input data represents a square area of the picture.
  • the transform that is applied must be two-dimensional, but such two-dimensional transforms are difficult to compute efficiently.
  • DCT Discrete Cosine Transform
  • IDCT inverse DCT
  • a one-dimensional DCT of order N is mathematically equivalent to multiplying two N-by-N matrices.
  • 512 multiplications and 448 additions are required, so that 1,024 multiplications and 896 additions are needed to perform the full 2 dimensional DCT on the 8-by-8 pixel block.
  • These arithmetic operations, and especially multiplication are complex and slow and, therefore, limit the achievable transmission rate. They also require considerable space on the silicon chip used to implement the DCT.
  • the DCT procedure can be rearranged to reduce the amount of computation required.
  • two main methods used for reducing the computation required for the DCT both of which use "binary decimation”.
  • the term "binary decimation" means than an N-by-N transform can be computed by using two N2-by-N2 transformations, plus some computational overhead whilst arranging this. Whereas the eight-by-eight transform requires 512 multiplications and 448 additions, a four-by-four transform requires only 64 multiplications and 48 additions. Binary decimation, thus, saves 284 multiplications and 352 additions and the overhead incurred in performing the decimation is typically insignificant compared to the reduction in computation.
  • the Chen method uses a recursive matrix identity that reduces the matrices into diagonals only. This method provides easy binary decimation of the DCT using known identities for diagonal matrices.
  • the calculator Since the calculator is not able to store the four digits required to fully represent the answer 16.86, the calculator must reduce the answer to three digits either by truncating the answer by dropping the right-most ⁇ 6 ⁇ , yielding an answer of 16.8, or it must have the necessary hardware to round the answer up to the closest three-digit approximation 16.9.
  • a much more efficient DCT/IDCT method and hardware structure would ensure that the numbers used in the method could be represented with a fixed decimal point, but in such a way that the full dynamic range of each number could be used. In such a system, truncation and rounding errors would either be eliminated or, at least, greatly reduced.
  • a fixed-coefficient multiplier that is, in this case, a multiplier capable of multiplying only by five, would require only a single shifter and a single adder in order to perform the multiplication (disregarding circuitry needed to handle carry bits).
  • a general purpose multiplier in contrast, would require shifters and adders for each of the eight positions, even though six of them would never need to be used.
  • fixed coefficients can simplify the multipliers since they allow the designer to eliminate rows of adders that correspond to zeros in the coefficient, thus saving silicon area.
  • an IDCT method in accordance with the present invention, a one-dimensional IDCT for each N-row and N-column of N-by-N pixel blocks is decimated and a 1-D IDCT is performed separately on the N-2 even-numbered pixel input words and the N-2 odd-numbered pixel input words.
  • N 8 according to the JPEG standard.
  • the two-dimensional IDCT result is then obtained by performing two one-dimensional IDCT operations in sequence (with an intermediate reordering-transposition-of data).
  • a first pair of input values is passed without need for multiplication to output adders and subtractors.
  • Each of a second pair of input values is multiplied by each of two constant-efficient values corresponding to two scaled cosine values. No other multiplications and only one subtraction and one addition are required in the common processing step.
  • the second pair is then added or differenced pairwise with the first pair of input values to form even or odd resultant values.
  • a pre-common processing stage the lowest order odd input word is pre-multiplied by the square root of two and the odd input words are summed pairwise before processing in the common processing block
  • a post-common processing stage intermediate values corresponding to the processed odd input words are multiplied by predetermined constant coefficients to form odd resultant values.
  • the N/2 high-order outputs are formed by simple subtraction of the odd resultant values from the even resultant values
  • the N/2 low-order outputs are formed by simple addition of the odd resultant values and the even resultant values.
  • the values are preferably and deliberately scaled downward by a factor of two by a simple binary right shift. This deliberate, balanced, upward scaling eliminates several multiplication steps that are required according to conventional methods.
  • selected bits of constant coefficient or intermediate resulting data words are rounded or adjusted by predetermined setting of selected bits to either ⁇ 1 ⁇ or ⁇ 0 ⁇ .
  • Two-dimensional transformation of pixel data is carried out by a second, identical 1-D operation on the output values from the first 1-D IDCT processing steps.
  • An IDCT system includes a pre-common processing circuit, and a common processing circuit, in which the pre-common, common, and post-common processing calculations are performed on input data words.
  • a supervisory controller generates control signals to control the loading of various system latches; preferably, to serially time-multiplex the application of the N/2 even and N/2 odd-numbered input words to input latches of the pre-common block to direct addition of the even and odd resultant values to form and latch low order output signals and to direct subtraction of the odd resultant values from the even resultant values to form and latch the high-order output signals and to sequentially control internal multiplexers.
  • Input data words are preferably (but not necessarily) latched, not in strictly ascending or descending order, but rather in an order enabling an efficient ⁇ butterfly ⁇ structure for the data path.
  • At least the common processing circuit may be configured as a pre-logic circuit, with no clock or control signals required for its proper operation, as may be other processing blocks, depending on the particular application.
  • the architecture is able to support much faster operations
  • the circuitry used to control the flow of data through the transform hardware can be small in area.
  • the forward DCT is used to transform spatial values (whether representing characteristics such as luminance directly, or representing differences, such as in the MPEG standard) into their frequency representation.
  • the inverse DCT operates the other ⁇ direction ⁇ , that is, the IDCT transforms the frequency values back into spatial values.
  • Equation 2 Equation 2 (E2), note that the cosine functions each depend on only one of the summation indices.
  • a 1-dimensional N-point IDCT (where n is an even number) is defined by the following expression. ##EQU4## and where y(n) are the N inputs to the inverse transformation function and x(k) are its N outputs.
  • the formula for the DCT has the same structure under the summation sign, but with the normalization constant outside the summation sign and with the x and y vectors switching places in the equation.
  • the 2-D IDCT can be calculated using a sequence of 1-D IDCT operations separated by a transpose.
  • each of these 1-D operations is, in turn, broken down into sub-procedures that are then exploited to reduce even further the required size and complexity of the semiconductor implementation.
  • an important design goal for IDCT hardware is the reduction of the required number of multipliers that must be included in the circuitry.
  • Most methods for calculating the DCT of IDCT therefore, attempt to reduce the number of multiplications needed.
  • all the input values are deliberately scaled upward by a factor of the square root of two.
  • the right-hand side of the IDCT expression (E) is deliberately multiplied by the square root of two.
  • two 1-D IDCT operations are performed in series (with an intermediate transpose) to yield the final 2-D IDCT result.
  • Each of these 1-D operations includes a multiplication by the same square root of two factor. Since the intermediate transposition involves no scaling, the result of two multiplications by the square root of two in series is that the final 2-D results will be scaled upward by a factor two. To obtain the unscaled value, the circuitry need then only divide by two. Since the values are all represented digitally, this can be accomplished easily by a simple right shift of the data.
  • the upward scaling by the square root of two in each 1-D IDCT stage and final down-scaling by 2 is accomplished by adders, multipliers and shifters all within the system's hardware, so that the system places no requirements for scaled inputs on the other devises to which the system may be connected. Because of this, the system is compatible with other conventional devises that operate according to the JPEG or MPEG standards. Normalization according to this embodiment of the present invention, therefore, eliminates the need for hardware multipliers within the IDCT semiconductor architecture for at least two square root of two multiplication operations. As is explained below in greater detail, the single additional multiplication step (upward scaling by the square root of two) of the input data in each 1-D operation leads to the elimination of still other multiplication steps that are required when using conventional methods.
  • the N-point, 1-D IDCT (see E4) can also be split into the two N/2-point, 1-D IDCTs each involving common core operations (under the summation sign) on the N/2 odd (grouped) and the N/2 even input values.
  • the expressions above yield the following simple structure for the IDCT as implemented in this embodiment:
  • g(k) operates directly on even input values to yield output values directly, whereas h(k') involves grouping of input values, as well as multiplication by the values d1, d3, d5 and d7.
  • an IDCT circuit is faced with a number of trade-offs, such as size versus speed and greater number of implemented devices versus reduced interconnection complexity. For example, it is often possible to improve the speed of computation by including additional, or more complicated devices on the silicon chip, but this obviously makes the implementation bigger or more complex. Also, what is available or desired on the IDCT chip may limit or preclude the use of sophisticated, complicated, designs such as "look-ahead" adders.
  • IDCT systems are at present measured according to a standardized method put forth by the Comite Consultatif International Circuitique et Telephonique ( ⁇ CCIT ⁇ ) in ⁇ Annex 1 of CCITT Recommendations H.261--Inverse Transform Accuracy Specification. ⁇ This test specifies that sets of 10,000 8-by-8 Blocks containing random integers be generated. These blocks are then DCT and IDCT transformed (preceded or followed by predefined rounding, clipping and arithmetic operations) using predefined precision to produce 10,000 sets of 8-by-8 ⁇ reference ⁇ IDCT output data.
  • the CCITT test blocks are used as inputs.
  • the actual IDCT transformed outputs are then compared statistically with the known ⁇ reference ⁇ IDCT output data.
  • Maximum values are specified for the IDCT in terms of peak, mean, mean square, and mean mean error of blocks as a whole and as individual elements.
  • the IDCT must produce all zeros output if the corresponding input block contains all zeros, and the IDCT must meet the same standards when the sign of all input data is changed. Implementations of the IDCT are said to have acceptable accuracy only if their maximum errors do not exceed the specified maximum values when these tests are run.
  • FIG. 9 is a simplified block diagram illustrating the data flow of the IDCT method according to one embodiment of the present invention (although the hardware structure, as is illustrated and explained below, is made more compact and efficient).
  • the inputs to the system such as Y[0] and Y[4], and the outputs from the system, such as X[3] and X[6], are shown as being conveyed on single lines.
  • each of the single-drawn lines in FIG. 9 represents several conductors in the form of data buses to convey, preferably in parallel, the several-bit wide data words to which each input and output corresponds.
  • the large open circles 225 and 226 represent two-input adders, whereby a small circle 227 at the connection point of an input with the adder indicates that the complement of the corresponding input word is used.
  • Adders with such a complementing input thus, subtract the complemented input from the non complemented input.
  • Adders with a single complementing input can, therefore, be said to be differencing components.
  • constant-coefficient multipliers are represented by solid triangles 230 in the data path.
  • FIG. 9 illustrates an important advantage of the embodiment, in accordance with the present invention.
  • the structure is divided into four main regions: a pre-common block, PREC 231, that forms the paired inputs p(k) and multiplies the input y(1) by the square root of two; a first post-common block, POSTC1 233, that includes four multipliers for the constants d1, d3, d5, d7 (see expression E12); a second post-common block, POSTC2 235, that sums the g0 to g3 terms and the h0 to h3 terms for the low order outputs, and forms the difference of the g0 to g3 terms and the h0 to h3 terms for the high-order outputs (See expressions E17 and E17); and a common block, CBLK 232, is included in both the even and odd data paths.
  • the common operations performed on the odd and even numbered inputs are
  • Units ⁇ 6 ⁇ plus ⁇ 5 ⁇ is ⁇ 1 ⁇ with a carry of ⁇ 1 ⁇ into the ⁇ tens ⁇ position--Sum: 1, Carry-in: 0, Carry-Out: 0.
  • the answer, ⁇ 1261 ⁇ is, thus, formed by adding the carry-in sum for each position to the sum for the same position, with the carry-in to each position being the carry-out of the adjacent lower-order position. (Note that this implies that the carry-in to the lowest order position is always a ⁇ 0 ⁇ ).
  • the problem is that one must wait to add the ⁇ 4 ⁇ and ⁇ 8 ⁇ in the hundreds place until one knows whether there will be a carry-in from the tens place. This illustrates a "ripple adder", which operates essentially in this way. A ripple adder, thus, achieves a ⁇ final ⁇ answer without needing extra storage elements, but it is slower than some other designs.
  • this possibility of delaying resolution is used to simplify the design and to increase the throughout of the IDCT circuitry.
  • certain bits of preselected carry words are, optionally and deliberately forced to predetermined values before resolution in order to provide greater expected accuracy of the IDCT result based on a statistical analysis of test runs of the invention on standard test data sets.
  • FIG. 10 is a block diagram that illustrates a preferred structure, in accordance with the present invention.
  • the even and odd numbered inputs are time-multiplexed and are processed separately in the common block CBLK 232.
  • the inputs may be processed in either order.
  • the notation Y[1,0], Y[5,4], Y[3,2] and Y[7,6] is used to indicate that the odd numbered inputs Y1, Y3, Y5, Y7 preferably pass through the calculation circuitry first, followed by the even numbered inputs Y0, Y2, Y4, Y6.
  • This order is not essential to the present embodiment; nonetheless, as is explained below, certain downstream arithmetic operations are performed only on the odd numbered inputs, and by entering the odd numbered input values first, these downstream operations can be processing at the same time that arithmetic operations common to all inputs are performed upstream on the even numbered inputs. This reduces the time that several arithmetic devices would otherwise remain idle.
  • X[0,7], X[1,6], X[3,4], X[2,5] is used to indicate that the low order outputs X0, X1, X2, X3 are output first, followed by the high order outputs X4, X5, X6, X7.
  • the inputs are preferably initially not grouped in ascending order, although this is not necessary since to odd numbered inputs are Y1, Y5, Y3, and Y7. Arranging the input signals in this order makes possible the simple ⁇ butterfly ⁇ data path structure shown in FIGS. 9 and 10 and greatly increases the interconnection efficiency of the implementation of the present invention in silicon semiconductor devices.
  • adders and subtractors are indicated by circles either a ⁇ + ⁇ (adder) 235, ⁇ - ⁇ (subtractor ) 236 which is an adder with one complementing input or ⁇ (resolving adder/subtractor, which is able to switch between addition and subtraction 237).
  • the left most adders and subtractors in the common block 232 of the two m-bit input words is the m-bit partial resulting parallel with the m-bit or (m-1) bit word containing the carry bits of the addition/subtraction.
  • the first additions and subtractions in the common block CBLK 232 are preferably unresolved, meaning that the addition of the carry bits is delayed until a subsequent processing stage.
  • FIG. 10 also illustrates the use of one and two input latches in the preferred embodiment of the present invention.
  • latches are illustrated as rectangles 238 and are used in both the pre-common block PREC 231 and the post-common block POSTC 233.
  • Single-input latches are used at the inputs of the multipliers D1, D3, D5 and D7, as well as to latch the inputs to the resolving adders/subtractors which are the computed g(k) and h(k) values corresponding to the respective outputs from latches g[0,7], g[1,6], g[3,4] and g[2,5] and h[0,7], h[1,6], h[3,4] and h[2,5].
  • the resolving adders/subtractors perform the addition or subtraction indicated in expressions E16 and E17 above.
  • the even-numbered inputs Y0, Y2, Y4 and Y6 do not need to be paired before being processed in the common block CBLK 232.
  • the input Y12 must also be multiplied by the square root of two in order to ensure that proper input values are presented to the common block CBLK 232.
  • One input to the 2-input mux latch is consequently tied directly to the unprocessed input values, whereas the other input is received from the resolving adders and, for the input Y 1 , the resolving square root of two multiplier.
  • the correct paired or unpaired inputs can, therefore, be easily presented to the common block CBLK 232 easily by simple switching of the multiplexing latches between their two inputs.
  • the square root of two multipliers D1, D3, D5, D7 preferably resolve their outputs, that is, they generate results in which the carry bits have been added in to generate a complete sum. This ensures that the outputs from the multipliers have the same bus width as the un-multiplied inputs in the corresponding parallel data paths.
  • the preferred embodiment of the common block 232 also includes one ⁇ dummy ⁇ subtractor 240 in the forward data path for Y[1,0] and Y[5,4], respectively.
  • These devices act to combine the two inputs (in the case of the dummy subtractor, after 2's-complementing the one input) in such a way that they are passed as parallel outputs.
  • the one input is manipulated as if it contained carry bits, which are added on in the subsequent processing stage. The corresponding addition and subtraction is, thus, performed, although it is delayed.
  • the ⁇ combiners ⁇ act as adders and subtractors and can be implemented for these devices and can be implemented either as simple conductors to the next device (for addition), or as a row of inverters (for subtraction), either of which requires little or no additional circuitry.
  • the even-numbered inputs are processed separately from the odd-numbered inputs in this preferred embodiment of the present invention. Assume, further, that the odd-numbered inputs are to be processed first. Supervisory control circuitry (not shown in FIG. 10) applies the odd-numbered input words to the pre-common block PREC, and selects the lower inputs (viewed as in FIG. 10) of the multiplexing latches C10, C54, C32, C76 which then stores the paired values p0 to p3 (see FIG. 9 and the definition of p(n) above). The latches 1h0, h1, 1h3 and 1h2 are then activated to latch the values H0, H1, H3 and H2, respectively.
  • the supervisory control circuitry latches and then selects the upper inputs of the two-input multiplexing latches C10, C54, C32 and C76 in the precommon block PREC 231 and applies the even numbered input words to these latches. Since the even-numbered inputs are used to form the values of g0 to g3, the supervisory control circuitry also opens the latches Lg0 to Lg3 in the post-common block POSTC 233, to store the g(k) values.
  • the post-common block POSTC 233 outputs the high-order signals X7, X6, X5 and X4 by switching the resolving adder subtractors to the subtraction mode.
  • the low order output signals X3, X2, X1 and X0 are then generated by switching the resolving adders/subtractors to the addition mode. Note that the output data can be presented in an arbitrary order, including natural order.
  • the preferred multiplexed implementation in accordance with the present invention, is illustrated in greatly simplified, schematic form in FIG. 10, performs the same calculations as the non-multiplexed structure illustrated in FIG. 9.
  • the number of adders, subtractors and multipliers in the common block CBLK 232 is, however, cut in half and the use of dummy adder/subtractors 240 further reduces the complexity of the costly arithmetic circuitry.
  • FIG. 11 illustrates the main components and data lines of an actual implementation of the IDCT circuit according to the embodiment of the present invention.
  • the main components include the precommon block circuit PREC 231, the common block circuit CBLK 232, and the post-common block POSTC 233.
  • the system also includes a controller CNTL 241 that either directly or indirectly applies input, timing and control signals to the precommon block PREC 231 and post-common block POSTC 233.
  • the input and output signals (Y0 to Y7 and X0 to X7, respectively) are 22 bits wide. Tests have indicated that this is the minimum width that is possible which still yields acceptable accuracy as measured by existing industry standards. As is explained in greater detail below, this minimum width in achieved in part by deliberately forcing certain carry words in selected arithmetic devices to be either a ⁇ 1 ⁇ or a ⁇ 0 ⁇ . This bit manipulation, corresponding to an adjustment of certain data words, is carried out as the result of a statistical analysis of the results of the IDCT system, in accordance with the present invention, to the after using the IDCT transformation of known input test data.
  • input data words are, therefore, preferably all conveyed serially over a single 22 bit input bus and each input word is sequentially latched at the proper input point in the data path.
  • the 22 bit input data bus is labelled T -- IN[21:0] 242.
  • the widths of multiple-bit signals are indicated in brackets with the high-order bit to the left of a colon ⁇ : ⁇ and the least significant bit (LSB) to the right of the colon.
  • the input signal T IN[21:0] 242 is 22 bits wide, with the bits being numbered from 0 to 21.
  • a single bit is identified as a single number within square brackets, thus, T -- IN[1] indicates the next to least significant bit of the signal T -- IN.
  • control signals are used to control the operation of the pre-common block PREC 231 in the preferred embodiment of the present invention.
  • the system in accordance with the present invention, preferably uses a non-overlapping two phase clock.
  • the signals IN -- CLK and OUT -- CLK are accordingly columns of latches that hold the values of input, intermediate, and output signals.
  • LATCH10, LATCH54, LATCH32, LATCH76 Preferably, one 22-bit word is input to the system at a time. On the other hand, four input signals are processed at a time. Each input signal must, therefore, be latched at its appropriate place in the architecture before being processed with three other input words. These latch signals are used to enable the respective input latches.
  • the signal LATCH54 for example, is first used to latch input signal Y5 and later to latch input signal Y4, which enters the pre-common block PREC 231 at the same point as the input signal Y5 (see FIG. 10) but during a subsequent processing stage.
  • LATCH Once the four even or odd-numbered input signals are latched into the pre-common block PREC 231, they are preferably shifted at the same time to a subsequent column of latches. The signal LATCH is used to enable a second column of input latches that hold the four input values to be operated on by the arithmetic devices in the pre-common block PREC 231.
  • the even-numbered input signals that are latched into the latches C10, C54, C32 and C76 should be those that bypass the adders and the square root of two resolving multiplier.
  • the odd-numbered input signals must first be paired to form the paired inputs p(n), and the signal Y1 must be multiplied by the square root of two.
  • the control signal SEL -- P is activated in order to select the paired input signals. Hence, these signals are used to control gates that act as multiplexers to let the correct signals pass to the output latches of the precommon block PREC 231.
  • the odd inputs are preferably applied as a group to the pre-common block first, followed by the even-numbered inputs, but any order may be used within each odd or even group, i.e., any order of inputs may be used, however, suitable latch arrangements as separately provided to process the odd-numbered inputs, or at least are provided in separate regions of the circuit.
  • the supervisory control circuitry also generates timing and control signals for the post-common block POSTC 233. These control signals are as follows:
  • EN -- BH, EN -- GH Referring again to FIG. 9, the outputs from the common block CBLK 232, after processing of the odd-numbered inputs, are shown as H0, H1, H3, and H2. These signals are then sent to the coefficient multipliers, d1, d3, d7, d5, respectively, in the first post common block POSTC1 233.
  • the signal EN -- BH is used to enable latches that hold the g0 to g3 values, as well as to enable the latches that hold the h0 to h3 values after they have been multiplied in the coefficient multipliers.
  • the embodiment includes a bank of resolving adders/subtractors that sum and difference(k) and h(k) values in order to form the low-order outputs, respectively.
  • the signals ADD, SUB are used to set the resolving adders/subtractors in the addition and subtraction modes, respectively.
  • EN -- 0 This signal is used to enable output latches that latch the results from the resolving adders/subtractors.
  • the output data from the system is preferably transmitted over a single 22-bit output bus, so that only one output value (X0 to X7) is transferred at a time.
  • These signals are activated sequentially to select which of the four latched output values is to be latched into a final output latch. Accordingly, these signals thus act as the control signals for a 4-to-1 multiplexer.
  • the output signals from the pre-common block PREC 231 are latched to form the input signals to the common block CBLK 232.
  • the output signals from the pre-common block PREC 231 are presented as the four 22-bit data words Cl10[21:0], Cl54[21:0], Cl32[21:0], Cl76[21:0], which become the input signals IN[0], IN[1], IN[3], IN[2], respectively, to the common block CBLK 232.
  • the four 22-bit results from the common block CBLK 232 are transferred in parallel as output signals OUT0[21:0], OUT1[21:0], OUT3[21:0], OUT2[21:0], which are then latched as the input signals of the post common block POSTC 233 as CO70[20:1], CO61[21:0], CO43[21:0], CO52[21:0].
  • the common block of the system's operations can be performed as pure logic operations, with no need for clock, timing or control signals. This further reduces the complexity of the device.
  • the pre-common and post-common blocks PREC 231, POSTC 233 may also be arranged to operate without clock timing or control signals.
  • FIG. 12 is a block diagram of the pre-common block PREC 231 of the present invention.
  • the notation ⁇ S1[a], S2[b], . . . , SM[Z] ⁇ indicates that the selected bits a, b, . . . , z from the signals S1, S2, . . . , SM are transferred in parallel over the same bus, with the most significant bits (MSBs) being the selected bits ⁇ a ⁇ of the signal S1, and the least significant bits (LSBs) being the selected ⁇ z ⁇ of signal SM.
  • the selected bits do not have to be individual bits, but rather, entire or partial multi-bit words may also be transmitted along with other single bits or complete or partial multi-bit words.
  • the symbol S will be replaced by the corresponding signal label.
  • a square root of two multiplier is shown as R2MUL.
  • the ⁇ save ⁇ ; or ⁇ unresolved sum ⁇ output from this non-resolving multiplier is indicated as the 21-bit word M5S[20:0]
  • the ⁇ carry ⁇ output from the multiplier R2MUL is shown as the 22-bit word M5C[20:0], which is transferred over the bus to the ⁇ b ⁇ input of a carry-save resolving adder M5A.
  • GND.M5S[20:0] the conductor corresponding to the MSB input to the adder M5A is forced to be a ⁇ 0 ⁇ by tying it to ground GND.
  • the four inputs that are processed at a given time in the pre-common block PREC 231 are transferred over the input bus T -- IN(21:0).
  • This input bus is connected to the inputs of four input latches IN10L, IN54L, IN32L, AND IN76L.
  • Each respective latch is enabled only when the input clock signal IN -- CLK and the corresponding latch selection signal LATCH10, LATCH54, LATCH32, LATCH76 are high.
  • the four inputs can, therefore, be latched into their respective input latches in four periods of the IN -- CLK signal by sequential activation of the latch enabling signals LATCH10, LATCH54, LATCH32, and LATCH76.
  • the LATCH signal should be low (or on a different phase) to enable the input latches IN10L, IN54L, IN32L, and IN76L to stabilize and latch the four input values.
  • FIG. 13 An example of the timing of the latches, in accordance with the present invention, is illustrated in FIG. 13. Once the four input signals are latched in the preferred order, they are passed to a second bank of latches L10L, L54L, L32L, L76L. These second bank of latches are enabled when the signals OUT -- CLK and LATCH are high. This signal timing is also illustrated in FIG. 13.
  • system of the present invention does not have to delay receipt of all eight input words. Once all the even or odd input words are received and latched in IN10L, IN54L, IN32L and L76L, this frees the In latches, which can then begin to receive the other four input signals without delay at the next rising edge of IN -- CLK.
  • the 2-digit suffix notation [10, 54, 32, 76] used for the various components illustrated in the Figures indicates that odd-numbered signals are processed first, followed by the even-numbered signals on a subsequent pass through the structure. As is mentioned above, this order is not required by the present invention, and it will be appreciated by one of ordinary skill in the art that additional orders may be used.
  • the corresponding values are either passed as inputs to output latches C10L, C54L, C32L and C76L on activation of the selected bypass signal SEL -- BYP, or they are passed as paired and multiplied inputs to the same output latches upon activation of the ⁇ select p ⁇ signal SEL -- P.
  • all signals are passed, both directly and indirectly, via arithmetic devices, to the output latches C10L, C54L, C32L, C76L of the pre-common block PREC 231.
  • the uppermost input value at the output of latch L10L is passed first to the square root of two-multiplier R2MUL and then to the resolving adder M5A as indicated.
  • the output from the resolving adder M5A is shown as an equivalent of the resolved multiplication of the output from the latch L10L by the square root of two.
  • the outputs from the other three latches L54L, L32L, L76L are also transferred to corresponding output latches C54L, C32L and C76L, respectively, both directly via 22-bit latch buses LCH54[21:0], LCH32[21:0] LCH76[21:0] and indirectly to the output latches via resolving adders P2A, P1A and P3A, respectively.
  • each resolving adder P2A, P1A, P1A has two inputs "a" and "b".
  • the one input is received from the latch L32L
  • the other input is received from the latch L54L.
  • the output from the adder P2A will, therefore, be equal to Y5+Y3, which, as is shown above, is equal to p(2).
  • the adders "pair" the odd-numbered inputs to form the paired input values p(1), p(2) and p(3).
  • FIG. 14 illustrates the preferred architecture of the common block CBLK 232, in accordance with the present invention. Because of the various multiplications and additions in the different system blocks, it is necessary or advantageous to scale down the input values to the common block before performing the various calculations. This ensures a uniform position for the decimal point (which is implied for integer arithmetic) for corresponding inputs to the various arithmetic devices in the system.
  • the input values IN0[21:0] AND IN1[21:0] are accordingly scaled down by a factor of four, which corresponds in digital arithmetic to a right shift of two bits.
  • the most significant bit (MSB) must then be replicated in the two most significant bits of the resulting right-shifted word; this process is known as "sign extension".
  • the input value IN0 is downshifted by two bits with sign extension to form the shifted input value indicated as IN[21], IN0[21], IN0[21:2].
  • the input value IN1[21:0] is similarly sign-extended two places.
  • the input IN2 is also shifted and extended to form IN2[21], IN2[21:1].
  • the input IN2, IN3 are those which must be multiplied by the scaled coefficients c1s and c3s.
  • Each input IN3 and IN2 must be multiplied by each of the scaled coefficients.
  • this is implemented by the four constant-coefficient carry-save multipliers MULC1S, MULNC1S, MULC3S3, and MULC2S2.
  • the bottom multiplier for IN2 is an inverting multiplier MULC1S, that is, its output corresponds to the negative of the value of the input multiplied by the constant C1S. Therefore, the value latched in C76 is subtracted from the value latched in C32 (after multiplication by C3S).
  • inverting multiplier MULNC1S By providing the inverting multiplier MULNC1S, subtraction is implemented by adding the negative of the corresponding value, which is equivalent to forming a difference. This allows the use of identical circuitry for the subsequent adders, while allowing a non-inverting multiplier may be used with a following subtractor.
  • cosine coefficient multipliers MULC1S, MULNC1S, MULC2S3, and MULC3S2 are included. If arrangements are made for signals to pass separately through the multipliers, however, the necessary multiplications can be implemented using only two multipliers one for the c1s coefficient and one for the c3s coefficient.
  • the multipliers for MULC1S, MULNC1S, MUL3S3 and MULC3S2 are preferably of the carry-save type, which means that they produce two output words, one corresponding to the result of the various rows of additions performed within a hardware multiplier, and another corresponding to the carry bits generated.
  • the outputs from the multipliers are then connected as inputs to either of two 4-input resolving adders BT2, BT3.
  • five of the inputs to the adders BT2 and BT3 are shown as being "split".
  • the "ca” input of the adder BT2 is shown as having IN3[21] over M3C[20:0] being input as the least significant 21 bits.
  • the "sa" (the “save-a” input) of the same adder is shown as being GND, GND over M3S[19:0]. This means that two zeros are appended as the two most significant bits of this input word. Such appended bits ensure that the proper 22-bit wide input words are formed with the proper sign.
  • the carry-save adders BT2 and BT3 add the carry and save words of two different 22-bit inputs to form a 22-bit output save word T3S[21:0] and a 21-bit output carry word T3C[21:1]. Accordingly, the input to each adder is thus 88 bits wide and the output from each adder is 43 bits wide.
  • FIG. 10 illustrates, the output from the latch C10 is combined with the output from the latch C54 in the upper-most data path before addition with the output from the carry-save adder BT3.
  • the "combination" is not, however, necessary until reaching the following adder in the upper data path. Consequently, as FIG. 14 shows, the shifted and sign-extended input value IN0 is connected to the upper carry input.
  • the upper carry input of adder CS0 is connected to the shifted and sign-extended input value IN0, and the shifted and sign-extended input IN1 is connected as the upper save input of the same adder. In other words, IN0 and IN1 are added later in the adder CS0.
  • dummy adder/subtractor 240 indicates which operation must be performed, although it does not necessarily have to be performed at the point indicated in FIG. 10.
  • the lower dummy subtractor 240 shown in FIG. 10 requires that the output from latch C54 be subtracted from the output from latch C10. This is the same as adding the output from C10 to the complement of the output of C54.
  • the complement of the input IN1 (corresponding to the output of latch C54 in FIG. 10) is performed by a 22-bit input inverter IN1[21:0] (which generates the logical inverse of each bit of its input, bit-for-bit).
  • the complement of IN1 value--NlN1[21:0]-- is passed to the upper "save" input of the adder CS1, with the corresponding upper "carry” input being the shifted and sign-extended IN0.
  • the upper portion of the adder CS1 therefore, performs the subtraction corresponding to IN0 minus IN1.
  • resolving subtractors are used instead of the resolving adders shown in the upper two data paths at the output of the common block CBLK 232.
  • Each resolving adder or subtractor is equivalent to a carry-save adder or subtractor followed by a resolving adder. This is shown in FIG. 14.
  • Subtractors CS2 and CS3 have as their inputs the processed values of IN0 to IN3 according to the connection structure shown in FIG. 10.
  • the 22-bit carry and save outputs from each of the adders/subtractors C20-CS3 are resolved in the resolving adders RES0-RES3.
  • resolution of carry and save outputs is well understood in the art of digital design and is, therefore, not described in greater detail here.
  • FIG. 14 illustrates, the save outputs the carry-save adders/subtractors CS0-CS3 are passed directly as 22-bit inputs to the "a"-input of the corresponding resolving adders RES0-RES3.
  • the 2's-complement of a binary number is formed by inverting each of its bits (changing all "1's” to “0's” and vice versa) and then adding "1". Note that the "1" can be added immediately after the bit inversion, or later.
  • the LSB of a carry word will always be a "0" which is implemented in the illustrated embodiment of the present invention by tying the LSB of the carry words O0C and O1C to ground GND as they are input to the resolving adders RES0 and RES1, respectively.
  • a "0" is appended as the LSB to the 21-bit carry words from the carry-save adders Cs0 and CS1 (by tying the LSB to ground GND) and the LSB of the carry words from the carry-save subtractors CS2 and CS3 is set equal to "one" by tying the corresponding data line to the supply voltage VDD.
  • FIG. 14 Two advantages of the IDCT circuitry according to the embodiment of the present invention can be seen in FIG. 14. First, no control or timing signals are required for the common block CBLK 232. Rather, the input signals to the common block are already processed in such a way they can be applied immediately to the pure-logic arithmetic devise in the common block 232. Second, by proper scaling of the data words, integer arithmetic can be used throughout (or, at least, decimal point for all values will be fixed). This avoids the complexity and slowness of floating-point devices, with no unacceptable sacrifice of precision.
  • Yet another advantage of the embodiment of the present invention is that, by ordering the inputs as shown, and by using the balanced decimated method in accordance with the present invention, similar design structures can be used at several points in the silicon implementation.
  • the constant coefficient multipliers MULC1S, MULC3S3, MULC3S2 and MULNC1S all have similar structures and receive data at the same point in the data path, so that all four multipliers can be working at the same time.
  • the carry-save adders BT2 and BT3 similarly will be able to work simultaneously, as will the following carry-save adders and subtractors. This symmetry of design and efficient simultaneous utilization of several devices is common throughout the structure according to the embodiment of the present invention.
  • FIG. 15 shows the preferred arrangement of the post-common block POSTC 233 in accordance with the present invention.
  • the primary functions of the post-common POSTC 233 are to form the h0 to h3 values by multiplying the outputs of the common block by the coefficients d1, d3, d5 and d7; to add the g(k) and h(k) values to form the low order outputs; and to subtract the h(k) values from the corresponding g(k) values to form the high-order outputs.
  • the post-common block POSTC 233 latches the corresponding outputs from the common block CBLK 232 into latches BH0L, BH1L, BH3L and BH2L when the Bh latches are enabled, the control circuitry sets the EN -- BH signal high, and the output clock signal OUTC -- CLK signal goes high.
  • the g(k), g0 to g3 values are latched into corresponding latches G0L, G1L, G3L and G2L when the control circuitry enables these latches via the signal EN -- GH and input clock signal IN -- CLK goes high.
  • the processed odd-numbered inputs that is, the values h0 to h3, are latched into latches H0L, H1L, H3L and H2L when the EN -- GH and IN -- CLK signals are high, via the constant coefficient multipliers D1MUL, D3MUL, D5MUL and D7MUL. These multipliers multiply, respectively by d1, d3, d5 and d7.
  • these constant-coefficient multipliers are preferably carry-save multipliers in order to simplify the design and to increase calculation speed.
  • the LSB of the H0 signal is preferably forced to be a "1" by tying the corresponding "save” output for H0 is set to 0 (tied to ground GND), and the second bit (corresponding to H0S[1]) is set to "1".
  • the data words from the carry and save outputs of the constant-coefficient multiplier D3MUL are similarly manipulated an input to the resolving adder H1A. The advantage of these manipulations and their input to the resolving adder H1A.
  • Bit-forcing may also be applied "within" a multiplication, for example, by selectively forcing one or more carry bits to predetermined values.
  • bit-forcing scheme need not be static, with certain bits always forced to take specified values, but rather a dynamic scheme may also be used. For example, selected bits of a data word may be forced to "1" or "0" depending on whether the word (or even some other data) is even or odd, positive or negative, or above or below a predetermined threshold, and the like.
  • the LSB's of selected data words are forced to be a "1" or a "0".
  • the CCITT test is run, and the CCITT statistics for the run are compiled.
  • the bit is then forced to the other of "1” or "0”, and the test is rerun.
  • the LSB (or LSBs) of other data words are forced to "1" or "0", and similar statistics are compiled.
  • the outputs from the constant-coefficient multipliers D1MUL, D3MUL, D5MUL and D7MUL may be resolved in the conventional manner in the resolving adders H0A-H3A.
  • these devise add their inputs when the ADD signal is high, and subtract the "b" input from the "a” input when the subtraction enable signal SUB is high.
  • the second bits of the upper two latch pairs H0L, G0L, H1L and G1L are manipulated by multiplexing arrangements in a manner described below.
  • the outputs from the resolving adder-subtractors S70A, S61A, S43A and S52A are latched into result latched R70L, R61L, R43L, R52L.
  • the input words to the adder/subtractor S70A and dS61A have the second bits of each input word manipulated.
  • the second bit of the input word to the "a"-input of the adder subtractor S70A is G0[1M], G0[1M], G0[0].
  • the second bit is set to have the value G01M.
  • the second bits of the other inputs to the adder/subtractors S70A and S61A are similarly manipulated. This bit manipulation is accomplished by four 2:1-bit multiplexers H01MUX, G01MUX, H11MUX and G11MUX (shown to the right in FIG. 15b).
  • these multiplexers are controlled by the ADD and SUB signals such that the second bit (H01M, G01M, H11M, and G11M) is set to one if the respective adder subtractor S70A, S61A is set to (ADD is high), and the second bit is set to its actual latch output value if the SUB signal is set too high.
  • Setting of individual bits in this manner is an easily implemented high-speed operation.
  • the preferred embodiment therefore, includes this bit-forcing arrangement since, as is described above, statistical analysis of a large number of tests pixel words has indicated that more accurate results are thereby obtained. It is not necessary, however, to manipulate the second bits in this manner, although it gives the advantage of smaller word width.
  • the four high or low-order results are latched in the output latches R70L, R61L, R43L and R52L.
  • the results are sequentially latched into the final output latched OUTF under the control of the multiplexing signals MUX -- OUT70, MUX -- OUT61, MUX -- OUT43, MUX -- OUT52.
  • the order in which resulting signals are output can therefore be controlled simply by changing the sequence with which they are latched into the latch.
  • FIGS. 13b and 13c The relationship between the clock and control signals in the post-common block POSTC 233 is shown in FIGS. 13b and 13c.
  • the output signals from the post-common block POSTC 233 are therefore, according to this embodiment of the present invention, first sorted in a known manner column-wise (or row-wise) in a conventional storage unit, such as a RAM memory circuit (not shown), and are then read from the storage unit row-wise (column-wise) so as to be passed as inputs to a subsequent pre-common block and for processing as described above in this block, and in a common block CBLK 232, and a post-common block POSTC 233.
  • a conventional storage unit such as a RAM memory circuit (not shown)
  • Storing by row (column) and reading out by column (row) performs the required operation of transposing the data before the second 1-D IDCT.
  • the output from the second POSTC 233 will be the desire, 2-D IDCT results and can be scaled in a conventional manner by shifting to offset the scaling shifts carried out in the various processing blocks. In particular, a right shift by one position will perform the division by 2 necessary to offset the two square root of two multiplications performed in the 1-D IDCT operations.
  • this second IDCT structure (which is preferably identical to that shown FIG. 11) is preferably a separate semiconductor implementation. This avoids the decrease in speed that would arise if the same circuits were used for both transforms, although separate 1-D transform implementations are not necessary if the pixel-clock rate is now sufficient such that a single implementation of the circuit will be able to handle two passes in real time.
  • a second preferred embodiment uses a single one-dimensional transform. This embodiment does not require a lowering of the pixel-clock rate as discussed previously.
  • the existing "resolving-adders" in the first preferred embodiment have been changed to "fast-resolving-adders". As seen in FIG. 38, these have been titled, “Fast Resolving Adders”. This change has the effect of allowing more time for each datapath arithmetic block to act on its data inputs.
  • the existing "latches” in the first preferred embodiment have been changed to 2-phase “flip-flops" or "registers".
  • the latching memory elements located on the front and end of the existing 1D IDCT datapath pipelines have been combined into single blocks, as shown particularly in particular in FIG. 18. Additionally, the amount of memory elements present at the input and the output of the second preferred embodiment has been increased to allow variable amounts of T2 data to be buffered.
  • stream “T1” raw unoperated upon data
  • stream “T2” data which has been through the 1D IDCT once and has been transposed in the TRAM
  • each stream takes its turn to introduce a group of data items into the datapath pipeline.
  • the data streams are "interleaved” as they pass sequentially down the datapath pipeline and are "de-interleaved” at the datapath output, as shown in FIGS. 17, 18 and 33.
  • a group can vary in number, but in this example, they are eight bits.
  • Ti must not be stalled. If T2 arrives at the point of interleaving with T1, but the input buffer should not introduce its data into the pipeline because this would clash with the T1 stream, then stream T2 provides an extra buffering so that T2 does not stall the data stream, but instead will buffer up data from its input stream until such a time as it may safely interleave with stream T1.
  • FIGS. 19 and 33 where the data from stream T1 is being loaded into the first transform in latches 0-7, using signals, "Latch 1(0) ⁇ through ⁇ Latch 1(7)". Additionally, data from T2 is being loaded in "Latch 2(0) ⁇ through ⁇ Latch 2(15)", as shown in FIG. 19, using signals shown in FIG. 33.
  • the interleaving is controlled by "T1 OK2 insert” and "T2 OK2 insert” signals. Under normal operation, the interleaving will occur when the signals go high. However, if the appropriate amount of data in the latch for T2 has not yet been reached when "T2 OK insert” goes high, then the latch will miss its opportunity and must continue buffering data until the next opportunity to insert data occurs.
  • T2 slips when it misses its data insertion point and has to continue buffering in the latches shown in FIG. 19. If T2 slipped and did not introduce data into the pipeline there will be a corresponding gap in the T2 stream output at the datapath output. This gap may be removed or "swallowed up” by use of the extra buffering at the T2 output.
  • This process may be thought of as having a "fixed" T1-1D IDCT transform with a variable T2-1D IDCT, where the data streams are interleaved in a time multiplex fashion such that they may use the same piece of arithmetic datapath pipeline.
  • Non-data is a data type that bypasses the IDCT and is shown as a data spike in "Latch 2 [ ⁇ ]" of FIG. 34. This eventually makes its way to T2 input, which allows the T2 buffering to fill up at the output.
  • Recovery is shown in FIG. 33 and FIG. 25 when the "T2 dout” signal and the "out” signal are gapped by a number of cycles. The gap is used as a reference to fix the data stream. It should be noted that the gap in cycles between these two signals is the same as the gap of buffering when the latch for T2 was waiting to insert its data.
  • the interleaved stream is de-interleaved into "T2 out", as shown in FIGS. 18 and 23.
  • the "T2 out” data stream has slip gaps in the data as described above.
  • the T2 out [143: ⁇ ], shown in FIG. 17, enters a 16 to 1 multiplexor block, shown as block "IDDPMUX" in FIG. 17.
  • This multiplexor block will select data from one of 16 positions in the output buffer block, as shown in FIG. 25. This position is selected by the control logic, shown in FIG. 29, which uses the gap by which T2 "buffered-up" at its input. This gap is used as a reference.
  • the output stream, T2DOUT, from the multiplexer block is the "fixed" data stream.
  • the method and system, in accordance with the present invention can be varied in numerous ways.
  • the structures used to resolve additions or multiplications may be altered using any known technology.
  • it is possible to use resolving adders of subtractors where the preferred embodiment uses carry-save devices with separate resolving adders.
  • the preferred embodiment of the present invention uses down-scaling at various points to ensure that all values remain within their acceptable ranges. Down-scaling is not necessary, however, because other precautions may be taken to avoid overflow or underflow.
  • certain bits of various data words were manipulated to reduce the required word width within the system.
  • the various intermediate values may, of course, be passed without bit manipulation.
  • data words were bit-manipulated in the illustrated example of the present invention, it is also possible to manipulate the bits of constant coefficients as well and evaluate the results under the CCITT standard. If a comparison of the results showed that it would be advantageous to force a particular bit to a given value, in some cases, on might then be able to increase the number of "zeros" in the binary representation of these coefficients in order to decrease further the silicon area required to implement the corresponding multiplier.
  • bit manipulation is not necessary.
  • an apparatus for transforming data having a first latch defining a first data stream source and a second latch defining a second data stream source.
  • the first and second latches are in communication with a single arithmetic unit.
  • the arithmetic unit communicates data to a transpose RAM, the transpose RAM transposes the data and communicates it to the second latch.
  • the second latch is adjustable and can be varied in size to accommodate variable rates of data being received and transmitted.
  • the second latch and first latch communicate 1st and 2nd data stream to the arithmetic unit sequentially, however, the sequential communication of the second latch does not interrupt the communication from the first latch.
  • common arithmetic unit is used for a first and second data stream. Furthermore, a process for transforming data using a common arithmetic unit having the following steps is described. First, loading the data into a first latch and, upon reaching a predefined number of cycles transmitting the data to an arithmetic unit and loading a first marker bit into a control shift register. Next, loading data into a second latch, the second latch is adjustable and can be varied in size to accommodate variable rate of data being received and transmitted at different rates. The next step is to transmit the data in the second latch to the arithmetic unit when the first control shift register reaches a predetermined state and the second latch is filled with a predetermined amount of data. Next, preventing transmission of data from the second latch, if the second latch is not filled with a predetermined amount of data and then recovering the second latch when the first latch is receiving non data.
  • Clock references are used to inform the decoder what number is used to represent the time "now”. This is used to initialize a counter that is incremented at regular intervals so that the decoder always knows what the current time is.
  • Time stamps are carried in some of the streams of data that are used to make up the programme (typically video and audio).
  • a time stamp is associated with a picture and tells the decoder at what "time” (defined by the counter that was initialized by the clock reference) a picture should be displayed.
  • PCRs Program Clock References
  • SCRs System Clock References
  • PCRs and SCRs have timing information to a resolution of 90 kHz with a further field extending the resolution to 27 MHz (or 1/27 ⁇ 10e6 in seconds).
  • Clock references are included in the data stream fairly often in order that "system time” may be reinitialized after a random access or channel change.
  • timestamps refer to a hypothetical model of a decoder that can decode pictures instantly.
  • any real decoder cannot do this and must take steps to modify the theoretical time in which pictures should be displayed.
  • time stamps and the clock references are used to determine display time and errors in display time. This modification depends upon the details of the architecture of the particular decoder. Clearly any delay introduced by the video decoder must be matched by an equivalent delay in the audio decoder.
  • FIG. 39 shows the demultiplexing of the MPEG systems stream into elementary streams 250.
  • Each elementary stream will typically carries either video or audio data although, in general, any form of data may be transported.
  • Each elementary stream is divided into a series of access units. In the case of video, the access unit is a picture. In the case of audio, it is a fixed number of samples of audio data.
  • time stamps 251 associated with each elementary stream.
  • the time stamps specify the "system time” at which the next access unit for the respective elementary stream is to be presented. These time stamps are referred to as presentation time stamps, "PTS”.
  • a second type of time stamp is also defined is referred to as a decode time stamp, "DTS". Since the DTS is only present when a PTS is also present and there is a simple relationship between them, the detailed differences between these two types of timestamps can be ignored since PTS/DTS differences have no bearing on the present invention.
  • the decode time stamps define the time at which an access unit (picture in the case of video) is to be decoded.
  • the presentation time stamps (PTS) define the time at which an access unit is to be presented (displayed).
  • the timing model used is a hypothetical model in which the decoder is infinitely fast. In this case, the DTS and PTS would be identical to one another.
  • the time synchronizing circuitry is placed at a point in the decoding pipeline when the pictures occur in their decoded order. Accordingly, this embodiment uses the DTS.
  • the information derived from the timestamps is transported through the various circuits by means of tokens.
  • Tokens consist of a series of one or more words of information.
  • the first word of the token contains a code which identifies the type of token and, hence, the type of information carried by that token.
  • an extension bit which is set to one to indicate that there are more words in the current token. Therefore, the last word of a token is indicated by the extension bit being zero.
  • the code in the first word indicating the type of token may be of a variable number of bits so that some codes use a small number of bits (allowing the remainder of the bits in the first word to be used to represent other information) while other codes use a larger number of bits.
  • Tokens may be characterized as being either control or DATA tokens.
  • the coded video data is viewed as data and is carried in a DATA token (e.g., the token called DATA) while the synchronization time is viewed as control information and is carried in a control token (called SYNC -- TIME).
  • Additional control tokens may also be used from time to time in the present invention. For example, a FLUSH token that behaves in a manner similar to a reset signal may be required to initialize the video decoding circuitry before attempting to restart decoding because of an error.
  • time synchronize two circuits it is an object of one preferred embodiment to time synchronize two circuits and, more particularly, to time synchronize two circuits without directly communicating system time from the first to the second circuit.
  • time synchronization of two circuits is accomplished without passing system time directly to the second circuit by providing synchronized time counters in each circuit.
  • the present invention also enables the system to time synchronize two circuits without communicating system time from the first to the second circuit by providing an elementary stream time counter in each circuit.
  • another object of the present invention is to time synchronize two circuits and to determine the presentation time error, if any, of the object being presented by using time stamp information, system time, and elementary stream time from the first circuit to generate synchronization time passed to the second circuit and compared to a copy of elementary stream time in the second circuit which is synchronized with the elementary stream time in the first circuit.
  • the system of the present invention can time synchronize a system decoder and a video decoder without directly communicating system time from the system decoder to the video decoder, without passing system time directly to the video decoder by providing synchronized time counters in each circuit and without communicating system time from the system decoder to the video decoder by providing a video counter in each circuit.
  • the invention also enables the system to time synchronize a system decoder and a video decoder and to determine the display time error, if any, of the picture being displayed by using video time stamp information, system time, and video decoding time from the system decoder to generate synchronization time which is then passed to the video decoder and compared to a copy of video decoding time in the video decoder which is synchronized with the video decoding time in the system decoder.
  • information derived from the timestamps can be transported through the system using a control token as previously described.
  • FIG. 40 shows a first preferred embodiment implementing elementary stream timestamp management, in accordance with the present invention.
  • the clock references 253, which represent system time, are decoded by the system demultiplexer 254 and placed initially, and then as needed, into a time counter 255 within the system decoder 256, and are incremented at 90 kHz.
  • a second copy of the clock reference 253 is simultaneously loaded into the time counter 258 that is inside the elementary stream decoder 257, incremented also at 90 kHz, and synchronized to the time counter 255 in the system decoder 256.
  • the time stamps 251 flow from the system demux 254 through the elementary stream buffer 260 so that they are delayed by the same amount as the incoming data.
  • the time stamps 251 may also have a correction added to compensate for the non-zero decode time of the elementary stream decoder 257.
  • the corrected time stamps 251 are then compared with the copy of the time stored in the time counter 258 inside the elementary stream decoder 257 to determine whether the decoded information is presented too early or too late.
  • the above embodiment is better than merely passing system time directly to the elementary stream decoder 257 from the time counter 255 in the system decoder 256 because the counter in the system decoder changes 90,000 times a second. Therefore, system time would, in all essence, need to be continually passed to the elementary stream decoder 257. Passing system time continually would require dedicated pins or the like.
  • system time can be passed in the form of clock references 253 a few times a second.
  • FIG. 41 Another embodiment is shown in FIG. 41.
  • the embodiment shown in FIG. 41 avoids the need for the clock references 253 to be passed to the elementary stream decoder 257. This is achieved by using a second counter "es -- time" 262, containing information on elementary stream time, which is maintained in both the system decoder 256 and the elementary stream decoder 257.
  • the two es -- time counters 262 and 263 are reset at power on, and at other times such as channel change, and then they free run from there on. Since this embodiment depends on the two es -- time counters 262 and 263 staying in step, it will be appreciated that it is necessary to take measures to ensure the es -- time counters do not get out of step.
  • One way to ensure the es -- time counters 262 and 263 stay in step is to use carry out of the es -- time counter in the system decoder to reset the es -- time counter in the elementary stream decoder 257 as shown in FIG. 41.
  • the clock references 253, which represent system time, are decoded by the system demultiplexer 254 and placed into a time counter 255 within the system decoder 256 and incremented at 90 kHz.
  • the es -- time counter 262 in the system decoder 256 of the present invention and the es -- time counter 263 in the elementary stream decoder 257 of the present invention are synchronized with each other and incremented at 90 kHz.
  • Elementary stream time stamps are also decoded by the system demultiplexer 254. Accordingly, a synchronization value X is computed using the elementary stream timestamp, the system time contained in the time counter and the elementary stream time contained in the es -- time counter 262 contained in the system decoder 256 according to the equations 3-1.
  • Equation 3-1 (a-d) is the equation required for time synchronization. Since it is undesirable to pass system time directly to the elementary stream decoder circuit 257, as shown in FIG. 41, a synchronization time representation X is generated, using Equation 3-1 (b-d), by the system decoder 256 and this value is passed to the elementary stream decoder. Synchronization time X is then compared to the elementary stream time contained within the es -- time counter 263 located within the elementary stream decoder 257. Hence, the compared result is used to determine whether the decoded information is presented too early or too late and then is further used in time synchronizing the system.
  • Time Synchronization (Elementary stream timestamp-system time)
  • the synchronization time, X may have a correction added to compensate for the non-zero decode time of the elementary stream decoder 257.
  • the corrected synchronization time is then compared with the elementary stream time contained in the es -- time counter 263 located inside the elementary stream decoder 257 to determine whether the decoded information is presented too early or too late and is further used to time synchronize the system.
  • the time correction could be subtracted from elementary stream time contained in the es -- time counter 263 located inside the elementary stream decoder 257 instead of added to synchronization time X for the same result.
  • the above embodiment is an example of a solution for generating synchronization time X and determining whether the information is presented early or late. It will be apparent to those skilled in the art that there are many other equivalent solutions for accomplishing the above.
  • FIG. 42 shows an alternative method for determining the synchronization time, X, in accordance with the present invention.
  • the system decoder 256 does not maintain an elementary stream time. Instead it records, in the register initial -- time 265, the value of system time at the instant that the elementary stream time counter, es -- time 263, located in the elementary stream decoder 257 is reset to zero.
  • the value in es -- time 263 can be computed by the system decoder 256 because it will be the difference between the current system time and the value recorded in initial -- time.
  • Equation 3-2 (a-c) is the equation representing the value of the elementary stream time stored in es -- time 263 located in the elementary stream decoder 257. This is substituted into equation 3-1 (d) to give equation 3-2 (b) which is simplified to derive equation 3-2 (c) providing the synchronization time, X, as a function of the system time and the value stored in the initial -- time register 265.
  • Equations 3-2
  • FIG. 43 shows another embodiment of the present invention implementing video timestamp management.
  • the clock references 253, which represent system time, are decoded by the system demultiplexer 254 and placed initially, and then as needed, into a time counter 255 within the system decoder 256 and are incremented at 90 kHz.
  • a second copy of the clock references 253 are simultaneously loaded into the time counter 258 that is inside the video decoder 270 and incremented at 90 kHz, and synchronized to the time counter 255 in the system decoder 256.
  • the video time stamps flow from the system demux 254 through the video decoding buffer 271 so that they are delayed by the same amount as the incoming video data.
  • the video time stamps may have a correction added to compensate for the non-zero decode time of the video decoder 270.
  • the corrected video time stamps are than compared with the copy of the time in the time counter 258 inside the video decoder 270 to determine whether the decoded picture is displayed too early or too late.
  • the embodiment shown in FIG. 43 is an improvement over the process of merely passing system time directly to the video decoder from the time counter in the system decoder because the counter in the system decoder changes 90,000 times a second. Therefore, system time would in all essence need to be continually passed to the video decoder. Passing system time continually would require dedicated pins or the like. By using a time counter located in the system decoder and a time counter located in the video decoder system time can be passed in the form of clock references a few times a second.
  • the clock references which represent system time, are decoded by the system demultiplexer 254 and placed into a time counter 255 within the system decoder 256 and incremented at 90 kHz.
  • the vid -- time counter 272 in the system decoder 256 and the vid -- time counter 273 in the video decoder 270 are synchronized with each other and incremented at 90 kHz.
  • Video time stamps are also decoded by the system demultiplexer 254. Accordingly, a synchronization value X is computed using a video timestamp, the system time contained in the time counter 273 and the video decoding time contained in the vid -- time counter 272 contained in the system decoder 256 according to the equations 3-3.
  • Equation 3-3 (a-d) is illustrative of one method in accordance with the present invention, for time synchronization which avoids passing the clock reference 253 to the video decoder 270.
  • Equation 3-3 (a) is the equation required for time synchronization. Since it is undesirable to pass system time directly to the video decoder circuit 270 as shown in FIG. 44, a synchronization time representation X is generated, using Equation 3-3 (b-d), by the system decoder 256 and passed to the video decoder 270. Synchronization time, X, is then compared to the video decoding time contained within the vid -- time counter 273 located within the video decoder 270. The compared result is used to determine whether the decoded picture is displayed too early or too late and then further used in time synchronizing the system.
  • the synchronization time, X may have a correction added to compensate for the non-zero decode time of the video decoder.
  • the corrected synchronization time is then compared with the video decoding time contained in the vid -- time counter 273 located inside the video decoder 270 to determine whether the decoded picture is displayed too early or too late and is also used to time synchronize the system.
  • the time correction can be subtracted from the video decoding time contained in the vid -- time counter 273 located inside the video decoder 270 instead of added to synchronization time X for the same result.
  • the above embodiment of the present invention is another example of a solution for generating synchronization time X and determining whether the picture is displayed early or late.
  • Another nice feature, in accordance with the present invention, is that there is no need to deal with the full 33 bit time stamp number or 42 bit clock reference number.
  • the present invention restricts the counters to 16 bits to allow 16 bit handling on the video decoder 270.
  • 16 bits cannot represent a sufficient number range at a resolution of 90 kHz (only 2/3 second to be used).
  • the time control on the video decoder 270 is only accurate to a field time (since the video timing generator VTG free-runs or is gen-locked to something that has nothing to do with the MPEG stream being decoded) and, therefore, it is not related to timestamps or presentation time in any way.
  • the synchronization time X and the vid -- time counter 273 within the video decoder 270 use only sixteen bits. This is made possible by two factors. First, the difference between system time and the timestamp (used to derive the synchronization time; see Equation 3-3) should always be small, thus allowing the more significant bits to be discarded. Second, it is only possible to control the presentation of video to the nearest frametime. Therefore, the less significant bits are not required and are discarded by shifting right by four bits.
  • the sixteen bits of time information used in the present invention are able to deal with timing errors of up to about 11.5 seconds with an accuracy of about 180 ⁇ s (about 1% of a field time).
  • a PAL or SECAM European 625 line TV system is, thus, 112.5 ticks of the 5625 Hz clock; a NTSC 525 line TV system is 93.84 ticks.
  • using 16 bits allows timing calculations with an accuracy of about 1% of a field time.
  • FIG. 46 shows the preferred process, in accordance with the present invention, of the moving the time stamp through the hardware.
  • the preferred method for communicating information in this hardware is Tokens, but it will be appreciated that alternative methods may also be employed.
  • the hardware is divided into two modules. The first module is added just after the Start Code Detector 201. This module is responsible for generating a token, SYNC -- TIME containing the synchronization time X, as discussed above, and this occurs just before an associated PICTURE -- START token.
  • the time stamp is carried in a packet header and refers to the first picture in the packet of data. Since the packets do not line up with the video data, there will, in general, be the end of the previous picture before the start of the picture to which the time stamp refers.
  • the synchronization time information may be supplied to the present invention either via a microprocessor interface or by using a Token. In either case, the synchronization time date (16 bits) is stored in the synchronization time register (divided into two parts to allow access to each byte individually), as further detailed in Table 12.
  • a flag, ts -- waiting is set to indicate the fact that valid synchronization time information is in the timestamp register. If the data was supplied using the SYNC -- TIME token, then that token is removed from the stream of tokens.
  • the flag that indicates the status of the synchronization time register is examined. If the flag is not set, then no action is taken and the PICTURE -- START token and all subsequent data is unaffected. If, however, the flag is set, indicating that valid synchronization time information is available in the register, then a SYNC -- TIME token is generated and placed in the data stream before the PICTURE -- START token. The flag is then cleared and the synchronization time register is made available for the next time-stamp that occurs.
  • the second module as shown in FIG. 46 consists of a prescaler clocked at 27 MHz and a vid -- time counter clocked by the prescaler 278 which are associated with the microprogrammable state machine, (MSM) 218.
  • MSM microprogrammable state machine
  • prescaler 278 that divides the clock by 4800, as shown in FIG. 44 and FIG. 46.
  • 4800 is 300 (27 MHz/90 kHz) times 16.
  • the 4804.8 option shown in FIG. 45 and FIG. 46 is discussed below.
  • the frame rate is not 30 Hz but is, in fact, approximately 29.94 Hz (precisely 30000/1001 Hz).
  • 29.94 Hz precise 30000/1001 Hz.
  • One possible solution is to "bend" the clock rate at the decoder so that instead of producing a 27 MHz clock, a 27.027 MHz clock is generated.
  • This clock is generated using the MPEG clock references with a divider of 300.3 (rather than 300) to yield the 90 kHz clock.
  • This 27.027 MHz clock when clocking the identical video timing circuitry that provides a 29.94 Hz frame rate from 27 MHz will give a precise 30 Hz rate.
  • the Vid -- time counter 273 (discussed above) contains the video decoding time and is incremented each time that the prescaler reaches its terminal count.
  • the vid -- time counter 273 is reset by the reset-time pin.
  • the prescaler and vid -- time counter of the present invention can be implemented with fully clocked feed-back flip-flops which are much more resistant to ⁇ -particle corruption than the resistive-feedback or weak-feedback latches used elsewhere.
  • Using clocked feedback flip-flops for time counters will help ensure that the time counter in the video decoder stays in step with the time counter in the system decoder.
  • FIG. 47 illustrates the process the MSM 218 performs when it receives the SYNC -- TIME token.
  • the MSM 218 is able to read the current time indicated by the video time counter and to then compare it with the value supplied by the video SYNC -- TIME token. It can, therefore, determine whether it is early or late, as compared to the time at which it should be decoding the pictures.
  • a 16 bit signed timestamp correction is added to the synchronization time X (discussed above) that was carried by the video SYNC -- TIME token. This correction is reset to zero by the MSM 218 at chip-reset, and if no action is taken, the synchronization time remains be unaltered.
  • the controlling microprocessor can always write value into ts -- correction to modify the synchronization time and, therefore, compensate for differential delays through the video and audio decoders.
  • the current video decoding time contained in the vid -- time counter 273 is subtracted from the corrected synchronization time.
  • the sign of value gives the direction of the error (and determines the error code, if any, generated by the MSM 218).
  • the absolute value of the difference is then taken and the result is compared to a threshold value to determine whether the timing error is within acceptable limits. Since, at present, the video timing can only be controlled to an accuracy of plus or minus a frame time from the nominal time (because the VTG 333 free-runs) this threshold is set at one frame time.
  • the MSM 218 of the present invention can correct the situation itself if the decoding is too early since the MSM can simply delay the decoding until the appropriate time. However, if the decoding is later than the intended time, then time correction is more difficult because it is not possible to discard pictures reliably at the output of the coded data buffer. Essentially, the decoding of the sequence is broken and the most reliable way to correct the situation is to restart the decoding process in a manner similar to random-access or channel change. In order to facilitate this process, the control register of the MSM 218 may be programmed to discard all data until a suitable start code or FLUSH token is encountered. In addition, the error "ERR -- TOO -- EARLY" is not generated during start-up, irrespective of the setting of disable -- too -- early, because at start up, the first picture is expected to be early.
  • Table 13 is illustrative of how the MSM 218 registers work and details some of the actions and error messages information placed in the registers can generate.
  • ERR -- TOO -- EARLY is generated if the decoding is taking place earlier than the time indicated by the time -- stamp. ERR -- TOO -- EARLY may be suppressed, but ERR -- TOO -- LATE will always be generated unless all errors are masked out.
  • the present invention includes: an apparatus for synchronizing time having, a timestamp for determining presentation time, a clock reference for initializing system time in a first circuit, a first time counter in communication with the clock reference for keeping system time in a first circuit and a second time counter initialized by the clock reference in a second circuit synchronized with the first time counter, for keeping a local copy of the system time and for determining the presentation timing error between the local copy of system time and system time by comparing the timestamp to the second time counter.
  • a still another embodiment includes an apparatus for synchronizing a first circuit and a second circuit using a clock reference for initializing system time in the first circuit, a first circuit having a time counter in communication with the clock reference for keeping system time, a first elementary stream time counter in the first circuit for providing elementary stream time.
  • the first circuit is adapted to receive a time stamp, and the first circuit generates synchronization time by adding elementary stream time to the time stamp and subtracting system time.
  • the second circuit is adapted to receive synchronization time from the first circuit and has a second elementary stream time counter in synchronization with the first elementary stream time counter for providing a local copy of the elementary stream time and for determining a timing error between the system time and the time stamp by comparing synchronization time to the local copy of elementary stream time.
  • an apparatus for synchronizing a first circuit and a second circuit has a clock reference for initializing system time in the first circuit.
  • the first circuit has a time counter in communication with the clock reference for keeping system time, and a first video time counter for providing video decoding time.
  • the first circuit is adapted to receive a video time stamp and generates synchronization time by adding video decoding time to the video time stamp and subtracting system time.
  • the second circuit is adapted to receive synchronization time from the first circuit and has a second video time counter in synchronization with the first video time counter for providing a local copy of video decoding time and for determining a timing error between system time and the video time stamp by comparing synchronization time to the local copy of video decoding time. Accordingly, the clock reference signal does not have to be passed directly to the second circuit in order to determine the timing error.
  • the present invention also includes a method for providing timing information by providing a video data stream having a time stamp carried in packet header wherein the time stamp refers to the first picture in the packet of data. In the next step a register is provided having a flag used to indicate valid time stamp information which is taken from the packet header and placed into the register.
  • the timestamp is removed from the video data stream and placed in the register.
  • the method encounters a picture start and subsequently examines the status of the register to determine if valid time stamp information is contained in the register by checking the flag status.
  • a time stamp is generated in response to the picture start if the flag indicates valid time stamp information is contained in the register and then the timestamp is inserted back into the data stream.
  • Another embodiment includes an apparatus described above wherein the elementary stream time counters are restricted to 16 bits.
  • the second elementary stream time counter located in the elementary stream decoder is restricted to 16 bits.
  • the synchronization time is restricted to 16 bits for controlling the elementary stream decode.
  • the present invention also has a process for decoding video and for determining display time errors against a threshold value. It then parses video data into tokens for further processing, determining if a time stamp token is indicated, comparing the time stamp token to a video time, and generates a compared value to determine an indicative of timing error. Next, it determines whether the compared value, when compared against a threshold value, is within acceptable parameters when a timing error is indicated and indicates when the compared value is outside acceptable parameters.
  • An alternative embodiment includes an apparatus for using a system decoder and a video decoder.
  • the system decoder is adapted to accept MPEG system streams and demultiplexing video data and the video time stamp from the stream.
  • the system decoder has a first time counter representative of system time.
  • the video decoder accepts the video data and the video time stamp, and has a second time counter in synchronization with the first time counter.
  • the video decoder also has a video decoder buffer for accepting the video data at a substantially constant rate and outputting the video data at a varying rate and for passing a video time stamp.
  • the video decoder while decoding a picture from the video data also compares the video time stamp for the decoded picture with the second time counter to determine the appropriate display time.
  • a method for determining a time error between a first circuit and a second circuit by providing the first circuit with a system time (SY), a time stamp (TS), and an elementary stream time (ET), obtaining synchronization time (X) by using the elementary stream time (ET), the time stamp (TS) and the system time (SY), in accordance with the equation;
  • X ET+TS-SY, providing synchronization time (X) to the second circuit and generating a synchronized elementary stream time (ET2) and obtaining a time error by using synchronized time (X) and in accordance with the equation ET2-X; hence, the first circuit can be time synchronized with the second circuit without passing system time to the second circuit.
  • the first circuit can be time synchronized with the second circuit without passing system time to the second circuit.
  • asynchronous swing buffering For asynchronous swing buffering, in accordance with the present invention, two buffers are operated asynchronously; one is written while the other is read. Accordingly, this allows for a data stream having a first rate of through-put to be resynchronized to another rate, while still maintaining a desired rate.
  • the write control and read control both have state indicators for communicating which buffer they are using and whether the controls are waiting for access or are, in fact, accessing that buffer. Each side communicates to the other side a single bit to indicate which buffer it is using. This is the only signal that must be synchronized between the two sides of asynchronous circuitry.
  • control circuit read or write
  • the invention will allow control to pass to the other circuit. If, after the control has swung, and two control circuits are trying to use the same buffer, then the later control circuit will begin waiting. The control circuit will wait until each side is using alternate buffers, i.e., the other side has swung. If, after it has swung, it finds that it is now using the alternate buffer to the other side, it will not wait, but immediately commence accessing.
  • This system of arbitration between the buffers is started up by both buffers using the same buffer, buffer 0, in this case.
  • the read side starts up by waiting, while the write side is accessing, since there is nothing valid to read out of either buffer.
  • the swing buffers are two discrete RAMS having all signals, such as enabling strobes, addresses and data multiplexed from either the read or write side, dependent on which buffer is being accessed by each side. This structure has been shown to use a lot of area in the busing of a large number of signals between the two buffers.
  • This structure contains twice as many rows of cells as one of the discrete RAMs found in the first embodiment of the present invention.
  • the second embodiment must have two pairs of bit lines since the read and write to the discrete buffers is happening simultaneously and asynchronously. Each row will be of its original width (i.e., have the same number of cells) since accesses are the same width as for the discrete RAMS. Each pair of rows are accessed as if at the same address, but from different buffers, so they connect to a different pair of bitlines.
  • these pair of rows can be readily accessed by one row decoder connected to the read address and one row decoder connected to the write address. Again, the read and write control never access the same buffer at the same time so there is no conflict as to which pair is accessed by which row decoder.
  • both the read and write circuitry within the structure of the present invention connect to each pair of bitlines, one pair from each buffer.
  • the read and writes are then multiplexed into each of the buffers and, for the same reasons explained above, there will not be conflict.
  • a swing unit 1 includes swing buffers 10 with RAM 12 and 14 in accordance with the present invention.
  • the swing unit 1 also includes a write control circuit and a read control circuit, which control the data into and out of the RAM 12 and 14.
  • the read control circuit and the write control circuit accomplish this by use of strobes, data and address control lines, 8.
  • Lines 7 and 9 are control lines to indicate the RAM used by the write control circuit and the RAM used by the read control circuit.
  • Line 7 functions to control the write control circuitry, i.e., when the read control circuitry is using, RAM 12 if low, RAM 14 if high.
  • Line 9 functions to inform the read control circuitry that the write control circuitry is using RAM 12 if low, RAM 14 if high.
  • swing buffer 10 has two RAM arrays, 12 and 14.
  • Swing Buffer 10 is capable of asynchronous, alternative reading and writing to the RAM area which enables the apparatus to achieve the necessary band width for high speed accessing of the memory.
  • the RAMs 12 and 14 require the following signals: write address 16, read address 18, data in 20, data out 22; and a read and write enable signal (not shown). See also FIG. 49.
  • the write address and read address signals are multiplexed by multiplexers 24.
  • the RAM array 12 and 14 operate with the write circuitry, row decoder and read circuitry in a conventional sense.
  • RAM 12 will be written to until the control circuitry switches a write enable single to RAM 14.
  • the RAM array 12 Once the RAM array 12 has been written, it falls under the control of the read circuitry 4, to be read. During this time, the RAM array 14 is also being written. It is important to note when the RAM array falls under the control of the read array control 2, or the write control circuit 4, the control is established until reading or writing is completed and then control is turned over. In the situation where the read control circuit 4 is accessing the RAM array, such as 12, and the write control circuitry 2 needs to access the same RAM array 12, then the write control circuit will begin waiting.
  • two control events are created.
  • a write control circuit or a read control circuit swings to a different RAM, it will either begin immediately accessing the RAM since the RAM is free and not under control of the alternative circuit, or it will begin to wait.
  • the read side defers to the write side, since there is nothing valid to be read out of either buffer.
  • An integrated swing buffer 30 includes a RAM array 32 having the logical size of RAM array 12 combined with RAM array 14. In other words, there is the same amount of RAM in both the first and second embodiments, however, it is combined in the second embodiment. Accordingly, the integrated swing buffer has the advantage of saving much of the busing area while still performing the same swing buffer function.
  • the write circuit and read circuit 34 and 36 respectively are similar to those used in the swing buffer 10. However, these circuits now include selectors which choose from the pairs of bit lines described hereinafter. Likewise, the read access row decoder 38 and the write access row decoder 40 are similar to those contained in swing buffer 10, however, they are able to access a pair of rows as described hereinafter in FIG. 51.
  • the particular structure of the integrated swing buffer 30, in accordance with the present invention is detailed.
  • Individual cells 42 are contained in rows 44.
  • the read row decoder 38 and write row decoder 40 access the rows 44 in pairs. A pair of rows have the same address provided by the address lines 16 and 18.
  • the read buffer line 52 and write buffer line 54 provide the control information for selecting one of the paired rows 44.
  • the buffer 0 bitlines 48 and buffer 1 bitlines 50 connect to alternative rows of cells and to the read and write circuitry 34 and 36.
  • the lighter shading illustrates the read row decoder 38 accessing a row in buffer 0.
  • the darker shading illustrates the write row decoder 40 accessing a row in buffer 1.
  • the present invention includes a swing buffer apparatus having at least two RAM arrays, a write control circuit in communication with the RAM arrays for controlling data input into the RAM array, and a read control circuit in communication with the RAM arrays for controlling data output from the RAM arrays. Furthermore, the write control circuit and read control circuit are in communication with one another to allow a synchronized control of the RAM arrays.
  • a swing buffer apparatus having a RAM array, a write control circuit in communication with the RAM array through a pair of bit lines, a read control circuit in communication with the RAM array through another pair of bit lines and a read row decoder and a write row decoder for addressing the RAM through a pair of rows so that individual cells are read.
  • the present invention also provides a method of asynchronously addressing RAM, by decoding at least a pair of cells in the RAM, using a row decoder to decode at least a pair of rows and selecting one of the rows to be assessed, using at least two pairs of bitlines connected to read a circuit and a write circuit and selecting the pair of bitlines to be used.
  • Video decompression systems contains three basic parts used to decode and display picture information.
  • the three main parts of a video decompression system are the spatial decoder, temporal decoder and the video formatter.
  • the present invention involves the temporal decoder and video formatter and the way in which the temporal decoder and video formatter manage their respective picture buffers, hereinafter the frame store buffer.
  • the temporal decoder contains two frame store buffers and the video formatter contains two frame store buffers.
  • MPEG uses three different picture types: Intra (I), Predicted (P) and Bidirectionally interpolated (B).
  • B pictures are based on predictions from two other pictures; one picture is from the future and one from the past.
  • the I pictures require no further decoding by the temporal decoder, but must be stored in one of the two frame store buffers for later use in decoding P and B pictures.
  • Decoding a P picture requires forming predictions from a previously decoded P or I picture.
  • the decoded P picture is stored in a frame store buffer for use in decoding further P and B pictures.
  • B pictures can require predictions from both of the frame store buffers. However, B pictures are not stored in the frame store buffers.
  • I and P pictures are not output from the temporal decoder as they are decoded. Instead, I and P pictures are written into one of the frame store buffers, and they are read out only when a subsequent I or P picture arrives for decoding. In other words, the temporal decoder relies on subsequent P or I pictures to flush previous pictures out of the two picture buffers. Accordingly, the spatial decoder of the present invention can provide a fake I or P picture when it is necessary to flush the temporal decoder's two frame store buffers. In turn, this fake picture is flushed when a subsequent video sequence begins.
  • the picture frames are displayed in numerical order.
  • the frames are transmitted in a different order. It is useful to begin the analysis from an intra frame (I frame).
  • the I frame is transmitted in the order it is to be displayed.
  • the next predicted frame (P frame), P4 is then transmitted.
  • any bidirectionally interpolated frames (B frames) to be displayed between the I frame and P4 frame are transmitted, represented by Be and B3. This allows the transmitted B frames to reference a previous frame (forward prediction) or a future frame (backward prediction).
  • the P7 frame After transmitting all the B frames to be displayed between I and P4, the P7 frame is transmitted. Next, all the B frames to be displayed between the P4 and P7 frames are transmitted, i.e., corresponding to B5 and B6. Then, the next I frame, 110, is transmitted. Finally, all the B frames to be displayed between the P7 and 110 frames are transmitted, corresponding to B8 and B9.
  • This ordering of transmitted frames requires only 2 frames to be kept in memory by the temporal decoder at any one time, and does not require the decoder to wait for the transmission of the next P frame or I frame to display an interjecting B frame.
  • the temporal decoder of the present invention can be configured to provide MPEG picture reordering. With this picture reordering, the output of P and I pictures is delayed until the next P or I picture in the data stream starts to be decoded by the temporal decoder.
  • the video formatter of the present invention stores two framestores or pictures.
  • three pictures or framestores are used to accommodate such features as repeating or skipping pictures.
  • the video formatter's off-chip DRAM holds three framestores.
  • the use of three framestores here allows frames to be either repeated or skipped in situations where the frame rates of the decoded video and the display are different.
  • All I, B and P frames are stored in the framestores of the video formatter. At any one time, there may be one frame store from which data is being displayed, one frame store into which data is being written, and in video formatters with three framestores, one other frame may be being stored in the third frame store.
  • the present embodiment performs all the prediction, reordering and block-to-raster tasks MPEG normally handles by using a temporal decoder with two framestores and a video formatter with two framestores, i.e., for a total of four framestores. This is accomplished in the present invention by using a frame store sharing scheme that only uses three framestores. The present embodiment cannot, however, handle the repeat and skip frame tasks of a video formatter with only the three framestores.
  • the present invention stores I pictures in a first frame store and P pictures in a second frame store. Because of the need to perform the block-to-raster conversion, B frames are stored in the manner detailed below in a third frame store. In order to minimize the amount of external DRAM required, a scheme is used where successive B frames share the same third frame store.
  • a B frame When a B frame is decoded, it may refer to the two previously decoded I or P frames occupying the first and second framestores.
  • the decoded B frame is written into the third frame store.
  • the present embodiment allows the raster to commence prior to a frame store being completely filled. The raster is allowed to start before the frame store is filled so that the next B frame can be written into the same frame store to occupy the space vacated by the raster at the top of the previous frame.
  • each frame store is split into sectors.
  • each frame store is first split into two field stores, each of which comprises N sectors, where N is the number of block rows in the field.
  • Frames coded as field pictures are straightforward. Each successive macroblock row occupies two sectors in a field store. Once the write back has progressed far enough down the frame, the raster starts reading out each sector from the top. Once the write back of the first frame has been completed, the start of the next frame is written into the space left by the raster. Checks on the status of each of the sectors ensures that the sector to be rastered is indeed full, and that for write back, the two sectors required are empty.
  • Frames coded as frame pictures are more difficult. Unlike field pictures, the macroblock rows of data are not written to the DRAM in the same order as they are to be rastered. The field stores are written to in parallel, whereas the fields are rastered in turn.
  • Field store 0 consists of 8 sectors numbered 0 to 7, each of which contains one row of blocks (i.e., each 8 pixels deep by the width of the picture).
  • Field store 1 consists of 8 sectors, numbered 8 to 15, each of which contains one row of blocks (i.e., each 8 pixels deep by the width of the picture).
  • the first macroblock row is written back into sector 0 in field store 0 and to sector 8 in field store 1.
  • the field stores continue to be filed in parallel. At some point, the raster beings displaying sectors from field store 0, that point being chosen so that the raster of field store does not catch up with the write back. However, the second frame cannot be written back in the same manner as the first. Because the sectors are written and read in a different order, waiting for the same two sectors to be free at the start of a frame would mean that write and read could not run continuously. This must be achieved in order to maintain the display and to maintain decoding at the necessary rate.
  • the second frame must be written into sectors of the frame store already freed by the raster.
  • This is implemented by dividing the framestores in two.
  • Sectors 4-7 become the upper part of the second field store and sectors 8-11 become the lower part of the first field store, i.e., they swap over.
  • the first macroblock row is written to sectors 0 and 4, once they are freed, with subsequent rows written to 1 and 5, then 2 and 6, and then 3 and 7.
  • the next row is written to sectors 8 and 12, and so on through to 11 and 15. This reallocation to the memory is sufficient to allow the write back and raster to continue at the appropriate rate.
  • the FIRST picture is rastered from--Sector 0, Then 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15.
  • the SECOND frame is written to--Sectors 0 and 4, Then 1 and 5, 2 and 6, 3 and 7, 8 and 12, 9 and 13, 10 and 14, 11 and 15.
  • the SECOND frame is rastered from--Sector 0, Then 1, 2, 3, 8, 9, 10, 11, 4, 5, 6, 7, 12, 13, 14, 15.
  • the second frame, the first macroblock row is not written into sectors 0 and 1, which are, after all, the first two sectors to be freed by the raster. Instead, it waits for sector 4 to clear. This is done for two reasons: First, waiting for sector 4 to clear does not affect the system's ability to maintain continuous decoding and display, even in the situation of worst-case coded data, and it is easier to implement. Secondly, with picture sizes which divide into a number of sectors that are not a power of two, the sequence for writing to and reading from sectors of memory does not repeat often (for example, NTSC format has 30 sectors per field and the sequence would repeat every 58 frames). This makes testability and recovery difficult.
  • each half field store is effectively implemented as a fifo, with pointers to the next location to be written and to be read.
  • each fifo being full or empty causes write back and raster, respectively to be disabled. This makes use of the knowledge that each half field store is itself written and read only one way, just like a fifo.
  • the present invention provides method for storing video information by providing video information in the form of an I Frame, a P Frame, a B 1 Frame and a B2 Frame, storing the I Frame in a first Frame store, storing the P frame in a second frame store; providing a third Frame store having a first and second field store, the first and second field store being split into at least two memory areas respectively, storing the B 1 Frame in the third register, reading the B 1 Frame from a selected portion of the memory area in the first or second field store; writing a portion of the B2 Frame into the selected portion of the memory area from which the B 1 Frame was read; whereby a reduced amount of memory can be used to store video information.
  • the Parallel Huffman Decoder block will decode Huffman coded Variable Length Codes (VLCs) and Fixed Length Codes (FLCs), and pass through tokens under the control of the parser microprogrammable state machine (MSM).
  • VLCs Variable Length Codes
  • FLCs Fixed Length Codes
  • MSM microprogrammable state machine
  • This embodiment of the present invention handles both MPEG-2 as well as MPEG-1 Huffman codes.
  • An important aspect of this embodiment of the invention is that it can sustain a high through-put due to the fact that it is a parallel decoder rather than a serial one.
  • This embodiment of the present invention uses a code lookup technique to decode Huffman codes. This is done to achieve the performance requirements and also to handle the second MPEG-2 transform coefficient table which is irregular or non-canonical in nature.
  • this embodiment of the invention has some features that allow it to decode certain more complex components from the stream in a single cycle without the assistance of an external controller.
  • complex components are Escape-coded coefficients, Intra-DC values and Motion Vector deltas, all of which are present in the stream as combined VLC/FLC components.
  • VLCs variable length codes
  • FLCs require a bypass mechanism which uses the selector 301 output to generate data and an input field to specify the length of the FLC.
  • the ROM 302 is not required at all during FLC decoding.
  • the "table select" input is used to select between the various different Huffman code tables required by MPEG.
  • the core of the implementation of the present invention, used to decode all the VLCs is a special ROM 302 whose addresses are controlled with a selector/shifter 301 as shown in FIGS. 52 and 53.
  • the ROM 302 has the job of performing a VLC table index calculation, followed by the index-to-data operation that yields decoded data.
  • the index calculation can be thought of as a content addressable memory (CAM) operation with "don't care" matching implemented to handle the Huffman codes which form the presented data. Since all the VLC code tables are fixed, a CAM-ROM will suffice and this is the job of the ROM AND-plane shown in FIGS. 54 through 57. Since the index generation is performed in a look-up manner (rather than algorithmically) there is no restriction to handling tables which are canonical.
  • CAM content addressable memory
  • the ROM Or-plane converts the "index" (an activated word-line) into the decoded data and the size (or length) of the code.
  • the data forms the decoded output (subject to error checking) and the size information is fed back to allow a calculation to be performed which controls the selector and, thus, presents the decoder ROM 302 with the correct data to perform the decoding of the next VLC in the subsequent cycle.
  • the ROM 302 address of the present invention is in two fields.
  • the larger field is the bit-pattern to be decoded and the smaller field selects which Huffman code table is to be examined.
  • the bit-pattern which must be examined is quite long, 16 bits, corresponding to the longest VLC code and there is an additional 4 bits of table select.
  • the AND-plane In order to decode VLCs, the AND-plane must be able to decode "don't care" bits in the VLC bit-pattern. This is because all VLCs which are shorter than the maximum 18 bits will be followed by additional bits which form no part of the decoding of that VLC. Because of the wide address, the AND-plane is predecoded (2 ⁇ 4), and the ROM 302 must combine "don't care" handling with this predecode. Furthermore, in addition to the complete MPEG code tables, the ROM 302 also has entries to identify illegal VLC patterns, which exist for some code tables.
  • Normal transform coefficients are also "complex" symbols, in the sense that they consist of a VLC followed by a 1-bit FLC which gives the sign of the level value and are handled in a similar manner to the other complex symbols (e.g. motion Vectors, Intra DC and Escape coded coefficients). Peak throughput cannot be achieved if coefficients are decoded as a VLC followed by an FLC (in separate cycles) and the alternative of allowing the ROM 302 to decode the sign bit would double the size of the two largest tables in the ROM. Thus in the present invention, special handling is used for various symbols so that a single cycle can produce the "final" required result.
  • FLC handling The basis of FLC handling is to control the selector with the required length of the FLC and to bypass the ROM 302 and simply output the correctly selected FLC.
  • simple FLCs are handled fairly naturally by the decoder, without significant extra hardware.
  • tokens are not manipulated, but simply passed directly to the output of the decoder.
  • the implementation includes the arrangement of registers with the counter 303 and selector 301, as shown in FIG. 52, and the actual code ROM.
  • the schematic of FIG. 53 shows how the core components are interconnected to implement the main Huffman decoding core section of the present invention.
  • the registers ms[31:0] and ls[31:0] are MSReg and LSReg, respectively, and the block phselect is the selector.
  • the counter logic is contained in the block phcclog (together with various other logic) and the count latch is called cntl[4:0].
  • the other logic on this schematic deals with handling commands, data and command dynamics, tokens, and the manipulation of the more "complex" symbols (performed in block phcop).
  • FIG. 54 illustrates a very small sample ROM design of the type used to implement the Huffman code ROM 302 in accordance with the present invention.
  • the unusual features of this ROM 302 lie in the AND-plane where predecode and "don't care" handling are used to implement a method of decoding variable length Huffman codes.
  • each address line (a[3], a[2], a[1] and a[0]) is driven across the AND-plane in both its true and inverted directions.
  • a transistor is connected to either the true or inverted address line in the conventional manner.
  • a transistor is not connected to either the true or the inverted line.
  • FIGS. 56 and 57 show alternative embodiments that utilize pre-decoding to reduce worst-case number of series transistors in the decoding logic.
  • two address bits are combined together in predecoding such that one of four lines is driven high for each of the four possible numbers that can be represented with the two address bits. It will be appreciated by one of ordinary skill in the art that the present invention would work equally well with higher levels of predecoding in which more than two bits are combined together. If the two address bits that are grouped together in the predecoding have defined values (either 1 or zero, but not the "don't care") then a transistor is connected to the appropriate predecoded address line in the conventional manner. Similarly, if both of the address bits have a "don't care", then no transistor is used as before.
  • the decoding requires that the wordline driven across the Or-plane be selected when either of two of the predecoded address lines is active. In the embodiment shown in FIG. 56, this is achieved by placing two transistors, one on each of the relevant predecoded address lines, in parallel as shown in the case for the code; 001x. In the embodiment shown in FIG. 57 the required decoding is achieved without using a parallel connection of transistors. In this case, two separate decodes are performed both of which must be selected. They are combined together using a NOR gate in the wordline driver such that the wordline is only activated if both of the selects are active.
  • the aforedescribed pipeline system of the present invention satisfies a long existing need for further improvements in various aspects of video decoding systems, including an MPEG video decompression method and apparatus utilizing a plurality of stages interconnected by a two-wire interface arranged as a pipeline processing machine.
  • Control tokens and DATA Tokens pass over the single two-wire interface for carrying both control and data in token format.
  • a token decode circuit is positioned in certain of the stages for recognizing certain of the tokens as control tokens pertinent to that stage and for passing unrecognized control tokens along the pipeline.
  • Reconfiguration processing circuits are positioned in selected stages and are responsive to a recognized control token for reconfiguring such stage to handle an identified DATA Token.
  • a wide variety of unique supporting subsystem circuitry and processing techniques are disclosed for implementing the system, including memory addressing, transforming data using a common processing block, time synchronization, asynchronous swing buffering, storing of video information, a parallel Huffman decoder, and the like.

Abstract

A method of asynchronously accessing a random access memory having a plurality of rows and columns, where each row has a wordline connected to read and write row decoders and each column is connected to bitlines. A row address is assigned to a first and second row, i.e. a pair of rows. A read address is provided to the read row decoder and a write address is provided to the write row decoder. The read and write addresses are decoded by the read and write row decoders, respectively, and one of the first or second rows is selected for reading. Asynchronous with selecting one of the first or second rows for reading, one of the first or second rows is selected for writing. Data is then read from the row selected for reading and asynchronously data is written into the row selected for writing. Signals are provided which coordinate the reading and writing of data, where in the event reading or writing is being performed, another of the reading or writing is deferred until completion of the first reading or writing. The read row decoder and the write decoder are unable to select the same row simultaneously and a read control circuit and a write control circuit are unable to select the same bit lines simultaneously.

Description

CROSS REFERENCE TO RELATED APPLICATIONS
This Application is a division of application Ser. No. 08/991,234, filed Dec. 16, 1997, which is a continuation of application Ser. No. 08/475,729, filed Jun. 7, 1995 (now abandoned), which is a division of application Ser. No. 08/473,813, filed Jun. 7, 1995, (U.S. Pat. No. 5,821,885), and a continuation-in-part of application Ser. No. 08/400,201, filed Mar. 7, 1995 (now U.S. Pat. No. 5,603,012), which is a division of application Ser. No.08/400,397, filed Mar. 7, 1995, now abandoned, which is a continuation-in-part of U.S. application Ser. No. 08/382,958, filed Feb. 2, 1995 (now abandoned), which is a continuation of U.S. application Ser. No. 08/082,291, filed Jun. 24, 1993 (now abandoned).
The following U.S. Patent application have subject matter related to this Application: application Ser. Nos. 08/382,958, filed Feb. 2, 1995, 08/400,397, filed Mar. 7, 1995; 08/399,851 filed Mar. 7, 1995; 08/482,296, filed Jun. 7, 1995; 08/486,396, filed Jun. 7, 1995; 08/484,730, filed Jun. 7, 1995 (now U.S. Pat. No. 5,677,648); 08/479,279, filed Jun. 7, 1995 (now U.S. Pat. No. 5,805,914); 08/483,020, filed Jun. 7, 1995; 08/487,224, filed Jun. 7, 1995 (now U.S. Pat. No. 5,835,740); 08/400,722, filed Mar. 7, 1995 (now U.S. Pat. No. 5,596,517); 08/400,723, filed Mar. 7, 1995 (now U.S. Pat. No. 5,594,678); 08/404,067, filed Mar. 14, 1995 (now U.S. Pat. No. 5,590,067); 08/567,555, filed Dec.5, 1995 (now U.S. Pat. No. 5,617,458); 08/396,834, filed Mar. 1, 1995 (now U.S. Pat. No. 5,677,648); 08/473,813, filed Jun. 7, 1995 (now U.S. Pat. No. 5,821,885); 08/484,456, filed Jun. 7, 1995; 08/476,814, filed Jun. 7, 1995 (now U.S. Pat. No. 5,798,719); 08/481,561, filed Jun. 7, 1995 (now U.S. Pat. No. 5,801,973); 08/482,381, filed Jun. 7, 1995 (now U.S. Pat. No. 5,828,907); 08/479,910, filed Jun. 7, 1995 (now U.S. Pat. No. 5,768,629); 08/475,729, filed Jun. 7, 1995 (abandoned); 08/484,578, filed Jun. 7, 1995 (now U.S. Pat. No. 5,878,273); 08/473,615, filed Jun. 7, 1995 (abandoned); 08/487,356, filed Jun. 7, 1995; 08/487,134, filed Jun. 7, 1995 (now U.S. Pat. No. 5,835,792); 08/481,772, filed Jun. 7, 1995 (now U.S. Pat. No. 5,740,460); 08/481,785, filed Jun. 7, 1995 (now U.S. Pat. No. 5,703,793); 08/486,034, filed Jun. 7, 1995 (abandoned); 08/486,908, filed Jun. 7, 1995 (now U.S. Pat. No. 5,820,007); 08/488,348, filed Jun. 7, 1995 (now U.S. Pat. No. 5,984,512); 08/484,170, filed Jun. 7, 1995 (now U.S. Pat. No. 5,963,154); 08/516,038, filed Aug. 17, 1995 (abandoned); 08/399,810, filed Mar. 7, 1995 (now U.S. Pat. No. 5,625,571); 08/400,201, filed Mar. 7, 1995 (now U.S. Pat. No. 5,603,012); 08/400,215, filed Mar. 7, 1995, 08/400,072, filed Mar. 7, 1995 (now U.S. Pat. No. 5,784,631); 08/402,602, filed Mar. 7, 1995; 08/400,206, filed Mar. 7, 1995 (abandoned); 08/400,151, filed Mar. 7, 1995; 08/400,202, filed Mar. 7, 1995; 08/400,398, filed Mar. 7, 1995; 08/400,161, filed Mar. 7, 1995; 08/400,141, filed Mar. 7, 1995; 08/400,211, filed Mar. 7, 1995 (now U.S. Pat. No. 5,842,033); 08/400,331, filed Mar. 7, 1995; 08/400,207, filed Mar. 7, 1995 (abandoned); 08/399,898, filed Mar. 7, 1995 (now U.S. Pat. No. 5,768,561); 08/399,665, filed Mar. 7, 1995 (abandoned); 08/400,058, filed Mar. 7, 1995 (abandoned); 08/399,800, filed Mar. 7, 1995 (abandoned); 08/399,801, filed Mar. 7, 1995; 08/399,799, filed Mar. 7, 1995 (abandoned); 08/474,222, filed Jun. 7, 1995 (abandoned); 08/486,481, filed Jun. 7, 1995 (abandoned); 08/474,231, filed Jun. 7, 1995; 08/474,830, filed Jun. 7, 1995 (abandoned); 08/474,220, filed Jun. 7, 1995 (now U.S. Pat. No. 5,699,544); 08/473,868, filed Jun. 7, 1995 (now U.S. Pat. No. 5,761,741); 08/474,603, filed Jun. 7, 1995 (now U.S. Pat. No. 5,861,894); 08/485,242, filed Jun. 7, 1995 (now U.S. Pat. No. 5,689,313); 08/477,048, filed Jun. 7, 1995 (abandoned); 08/485,744, filed Jun. 7, 1995; 08/850,125, filed May 1, 1997 (now U.S. Pat. No. 5,956,519); 08/812,820, filed Mar. 6, 1997 (now U.S. Pat. No. 5,724,537); 08/804,620, filed Feb. 24, 1997 (now U.S. Pat. No. 5,907,692); 08/876,720, filed Jun. 16, 1997; 08/903,969, filed Jul. 31, 1997; 08/947,727, filed Sep. 25, 1997 (now U.S. Pat. No. 5,809,270); 08/937,143, filed Sep. 24, 1997; 08/946,754, filed Oct. 7, 1997; 08/947,646, filed Oct. 8, 1997; 08/950,892, filed Oct. 15, 1997 (now U.S. Pat. No. 5,956,741); 08/955,476, filed Oct. 21, 1997; 08/967,515, filed Nov. 11, 1997; 08/992,859, filed Dec. 10, 1997; and 08/487,740, filed Jun. 7, 1995 (abandoned).
INTRODUCTION
The present invention relates generally to a new and improved system for decoding a plurality of audio and video signals and, more particularly, to a new and improved system for decoding a plurality of MPEG audio and video signals.
A serial pipeline processing system of the present invention comprises a single two-wire bus used for carrying unique and specialized interactive interfacing tokens, in the form of control tokens and data tokens, to a plurality of adaptive decompression circuits and the like positioned as a reconfigurable pipeline processor.
PRIOR ART
U.S. Pat. No. 5,111,292 discloses an apparatus for encoding/decoding a HDTV signal for e.g. terrestrial transmission includes a priority selection processor for parsing compressed video codewords between high and low priority channels for transmission. A compression circuit responsive to high definition video source signals provides hierarchically layered codewords CW representing compressed video data and associated codewords T defining the types of data represented by codewords CW. The priority selection processor, responsive to the codewords CW and T, counts the number of bits in predetermined blocks of data and determines the number of bits in each block to be allocated to the respective channels. Thereafter the processor parses the codewords CW into high and low priority codeword sequences wherein the high and low priority codeword sequences correspond to compressed video data of relatively greater and lesser importance to image reproduction respectively.
One prior art system is described in U.S. Pat. No. 5,216,724. The apparatus comprises a plurality of compute modules, in a preferred embodiment, for a total of four compute modules coupled in parallel. Each of the compute modules has a processor, dual port memory, scratch-pad memory, and an arbitration mechanism. A first bus couples the compute modules and a host processor. The device comprises a shared memory which is coupled to the host processor and to the compute modules with a second bus.
U.S. Pat. No. 4,785,349 discloses a full motion color digital video signal that is compressed, formatted for transmission, recorded on compact disc media and decoded at conventional video frame rates. During compression, regions of a frame are individually analyzed to select optimum fill coding methods specific to each region. Region decoding time estimates are made to optimize compression thresholds. Region descriptive codes conveying the size and locations of the regions are grouped together in a first segment of a data stream. Region fill codes conveying pixel amplitude indications for the regions are grouped together according to fill code type and placed in other segments of the data stream. The data stream segments are individually variable length coded according to their respective statistical distributions and formatted to form data frames. The number of bytes per frame is withered by the addition of auxiliary data determined by a reverse frame sequence analysis to provide an average number selected to minimize pauses of the compact disc during playback, thereby avoiding unpredictable seek mode latency periods characteristic of compact discs. A decoder includes a variable length decoder responsive to statistical information in the code stream for separately variable length decoding individual segments of the data stream. Region location data is derived from region descriptive data and applied with region fill codes to a plurality of region specific decoders selected by detection of the fill code type (e.g., relative, absolute, dyad and DPCM) and decoded region pixels are stored in a bit map for subsequent display.
U.S. Pat. No. 4,922,341 discloses a method for scene-model-assisted reduction of image data for digital television signals, whereby a picture signal supplied at time is to be coded, whereby a predecessor frame from a scene already coded at time t-1 is present in an image store as a reference, and whereby the frame-to-frame information is composed of an amplification factor, a shift factor, and an adaptively acquired quad-tree division structure. Upon initialization of the system, a uniform, prescribed gray scale value or picture half-tone expressed as a defined luminance value is written into the image store of a coder at the transmitter and in the image store of a decoder at the receiver store, in the same way for all picture elements (pixels). Both the image store in the coder as well as the image store in the decoder are each operated with feed back to themselves in a manner such that the content of the image store in the coder and decoder can be read out in blocks of variable size, can be amplified with a factor greater than or less than 1 of the luminance and can be written back into the image store with shifted addresses, whereby the blocks of variable size are organized according to a known quad tree data structure.
U.S. Pat. No. 5,122,875 discloses an apparatus for encoding/decoding an HDTV signal. The apparatus includes a compression circuit responsive to high definition video source signals for providing hierarchically layered codewords CW representing compressed video data and associated codewords T, defining the types of data represented by the codewords CW. A priority selection circuit, responsive to the codewords CW and T, parses the codewords CW into high and low priority codeword sequences wherein the high and low priority codeword sequences correspond to compressed video data of relatively greater and lesser importance to image reproduction respectively. A transport processor, responsive to the high and low priority codeword sequences, forms high and low priority transport blocks of high and low priority codewords, respectively. Each transport block includes a header, codewords CW and error detection check bits. The respective transport blocks are applied to a forward error check circuit for applying additional error check data. Thereafter, the high and low priority data are applied to a modem wherein quadrature amplitude modulates respective carriers for transmission.
U.S. Pat. No. 5,146,325 discloses a video decompression system for decompressing compressed image data wherein odd and even fields of the video signal are independently compressed in sequences of intraframe and interframe compression modes and then interleaved for transmission. The odd and even fields are independently decompressed. During intervals when valid decompressed odd/even field data is not available, even/odd field data is substituted for the unavailable odd/even field data. Independently decompressing the even and odd fields of data and substituting the opposite field of data for unavailable data may be used to advantage to reduce image display latency during system start-up and channel changes.
U.S. Pat. No. 5,168,356 discloses a video signal encoding system that includes apparatus for segmenting encoded video data into transport blocks for signal transmission. The transport block format enhances signal recovery at the receiver by virtue of providing header data from which a receiver can determine re-entry points into the data stream on the occurrence of a loss or corruption of transmitted data. The re-entry points are maximized by providing secondary transport headers embedded within encoded video data in respective transport blocks.
U.S. Pat. No. 5,168,375 discloses a method for processing a field of image data samples to provide for one or more of the functions of decimation, interpolation, and sharpening. This is accomplished by an array transform processor such as that employed in a JPEG compression system. Blocks of data samples are transformed by the discrete even cosine transform (DECT) in both the decimation and interpolation processes, after which the number of frequency terms is altered. In the case of decimation, the number of frequency terms is reduced, this being followed by inverse transformation to produce a reduced-size matrix of sample points representing the original block of data. In the case of interpolation, additional frequency components of zero value are inserted into the array of frequency components after which inverse transformation produces an enlarged data sampling set without an increase in spectral bandwidth. In the case of sharpening, accomplished by a convolution or filtering operation involving multiplication of transforms of data and filter kernel in the frequency domain, there is provided an inverse transformation resulting in a set of blocks of processed data samples. The blocks are overlapped followed by a savings of designated samples, and a discarding of excess samples from regions of overlap. The spatial representation of the kernel is modified by reduction of the number of components, for a linear-phase filter, and zero-padded to equal the number of samples of a data block, this being followed by forming the discrete odd cosine transform (DOCT) of the padded kernel matrix.
U.S. Pat. No. 5,175,617 discloses a system and method for transmitting logmap video images through telephone line band-limited analog channels. The pixel organization in the logmap image is designed to match the sensor geometry of the human eye with a greater concentration of pixels at the center. The transmitter divides the frequency band into channels, and assigns one or two pixels to each channel, for example a 3 KHz voice quality telephone line is divided into 768 channels spaced about 3.9 Hz apart. Each channel consists of two carrier waves in quadrature, so each channel can carry two pixels. Some channels are reserved for special calibration signals enabling the receiver to detect both the phase and magnitude of the received signal. If the sensor and pixels are connected directly to a bank of oscillators and the receiver can continuously receive each channel, then the receiver need not be synchronized with the transmitter. An FFT algorithm implements a fast discrete approximation to the continuous case in which the receiver synchronizes to the first frame and then acquires subsequent frames every frame period. The frame period is relatively low compared with the sampling period so the receiver is unlikely to lose frame synchrony once the first frame is detected. An experimental video telephone transmitted 4 frames per second, applied quadrature coding to 1440 pixel logmap images and obtained an effective data transfer rate in excess of 40,000 bits per second.
U.S. Pat. No. 5,185,819 discloses a video compression system having odd and even fields of video signal that are independently compressed in sequences of intraframe and interframe compression modes. The odd and even fields of independently compressed data are interleaved for transmission such that the intraframe even field compressed data occurs midway between successive fields of intraframe odd field compressed data. The interleaved sequence provides receivers with twice the number of entry points into the signal for decoding without increasing the amount of data transmitted.
U.S. Pat. No. 5,212,742 discloses an apparatus and method for processing video data for compression/decompression in real-time. The apparatus comprises a plurality of compute modules, in a preferred embodiment, for a total of four compute modules coupled in parallel. Each of the compute modules has a processor, dual port memory, scratch-pad memory, and an arbitration mechanism. A first bus couples the compute modules and host processor. Lastly, the device comprises a shared memory which is coupled to the host processor and to the compute modules with a second bus. The method handles assigning portions of the image for each of the processors to operate upon.
U.S. Pat. No. 5,231,484 discloses a system and method for implementing an encoder suitable for use with the proposed ISO/IEC MPEG standards. Included are three cooperating components or subsystems that operate to variously adaptively pre-process the incoming digital motion video sequences, allocate bits to the pictures in a sequence, and adaptively quantize transform coefficients in different regions of a picture in a video sequence so as to provide optimal visual quality given the number of bits allocated to that picture.
U.S. Pat. No. 5,267,334 discloses a method of removing frame redundancy in a computer system for a sequence of moving images. The method comprises detecting a first scene change in the sequence of moving images and generating a first keyframe containing complete scene information for a first image. The first keyframe is known, in a preferred embodiment, as a "forward-facing" keyframe or intraframe, and it is normally present in CCITT compressed video data. The process then comprises generating at least one intermediate compressed frame, the at least one intermediate compressed frame containing difference information from the first image for at least one image following the first image in time in the sequence of moving images. This at least one frame being known as an interframe. Finally, detecting a second scene change in the sequence of moving images and generating a second keyframe containing complete scene information for an image displayed at the time just prior to the second scene change, known as a "backward-facing" keyframe. The first keyframe and the at least one intermediate compressed frame are linked for forward play, and the second keyframe and the intermediate compressed frames are linked in reverse for reverse play. The intraframe may also be used for generation of complete scene information when the images are played in the forward direction. When this sequence is played in reverse, the backward-facing keyframe is used for the generation of complete scene information.
U.S. Pat. No. 5,276,513 discloses a first circuit apparatus, comprising a given number of prior-art image-pyramid stages, together with a second circuit apparatus, comprising the same given number of novel motion-vector stages, perform cost-effective hierarchical motion analysis (HMA) in real-time, with minimum system processing delay and/or employing minimum system processing delay and/or employing minimum hardware structure. Specifically, the first and second circuit apparatus, in response to relatively high-resolution image data from an ongoing input series of successive given pixel-density image-data frames that occur at a relatively high frame rate (e.g., 30 frames per second), derives, after a certain processing-system delay, an ongoing output series of successive given pixel-density vector-data frames that occur at the same given frame rate. Each vector-data frame is indicative of image motion occurring between each pair of successive image frames.
U.S. Pat. No. 5,283,646 discloses a method and apparatus for enabling a real-time video encoding system to accurately deliver the desired number of bits per frame, while coding the image only once, updates the anqutization step size used to quantize coefficients which describe, for example, an image to be transmitted over a communications channel. The data is divided into sectors, each sector including a plurality of blocks. The blocks are encoded, for example, using DCT coding, to generate a sequence of coefficients for each block. The coefficients can be quantized, and depending upon the quantization step, the number of bits required to describe the data will vary significantly. At the end of the transmission of each sector of data, the accumulated actual number of bits expended is compared with the accumulated desired number of bits expended, for a selected number of sectors associated with the particular group of data. The system then readjusts the quantization step size to target a final desired number of data bits for a plurality of sectors, for example describing an image. Various methods are described for updating the quantization step size and determining desired bit allocations.
U.S. Pat. No. 5,287,420 discloses a method and apparatus for image compression suitable for personal computer applications, which compresses and stores data in two steps. An image is captured in realtime and compressed using an efficient method and stored to a hard-disk. At some later time, the data is further compressed in non-realtime using a computationally more intense algorithm that results in a higher compression ratio. The two-step approach allows the storage reduction benefits of a highly sophisticated compression algorithm to be achieved without requiring the computational resources to perform this algorithm in realtime. A compression algorithm suitable for performing the first compression step on a host processor in a personal computer is also described. The first compression step accepts 4:2:2 YCrCb data from the video digitizer. The two chrominance components are averaged and a pseudo-random number is added to all components. The resulting values are quantized and packed into a single 32-bit word representing a 2×2 array of pixels. The seed value for the pseudo-random number is remembered so that the pseudo-random noise can be removed before performing the second compression step
U.S. Pat. No. 5,289,577 discloses a method and apparatus for a sequential process-pipeline which has a first processing stage coupled to a CODEC through a plurality of buffers, including an image data input buffer, an image data output buffer and an address buffer. The address buffer stores addresses, each of which identifies an initial address of a block of addresses within an image memory. Each block of addresses in the image memory stores a block of decompressed image data. A local controller is responsive to the writing of an address into the address buffer to initiate the operation of the CODEC to execute a Discrete Cosine Transformation Process and a Discrete Cosine Transformation Quantization Process.
The article, Chong, Yong M., A Data-Flow Architecture for Digital Image Processing, Wescon Technical Papers: No. 2 Oct./Nov. 1984, discloses a real-time signal processing system specifically designed for image processing. More particularly, a token based data-flow architecture is disclosed wherein the tokens are of a fixed one word width having a fixed width address field. The system contains a plurality of identical flow processors connected in a ring fashion. The tokens contain a data field, a control field and a tag. The tag field of the token is further broken down into a processor address field and an identifier field. The processor address field is used to direct the tokens to the correct data-flow processor, and the identifier field is used to label the data such that the data-flow processor knows what to do with the data. In this way, the identifier field acts as an instruction for the data-flow processor. The system directs each token to a specific data-flow processor using a module number (MN). If the MN matches the MN of the particular stage, then the appropriate operations are performed upon the data. If unrecognized, the token is directed to an output data bus.
The article, Kimori, S. et al. An Elastic Pipeline Mechanism by Self-Timed Circuits, IEEE J. of Solid-State Circuits, Vol. 23, No. 1, February 1988, discloses an elastic pipeline having self-timed circuits. The asynchronous pipeline comprises a plurality of pipeline stages. Each of the pipeline stages consists of a group of input data latches followed by a combinatorial logic circuit that carries out logic operations specific to the pipeline stages. The data latches are simultaneously supplied with a triggering signal generated by a data-transfer control circuit associated with that stage. The data-transfer control circuits are interconnected to form a chain through which send and acknowledge signal lines control a hand-shake mode of data transfer between the successive pipeline stages. Furthermore, a decoder is generally provided in each stage to select operations to be done on the operands in the present stage. It is also possible to locate the decoder in the preceding stage in order to pre-decode complex decoding processing and to alleviate critical path problems in the logic circuit. The elastic nature of the pipeline eliminates any centralized control since all the interworkings between the submodules are determined by a completely localized decision and, in addition, each submodule can autonomously perform data buffering and self-timed data-transfer control at the same time. Finally, to increase the elasticity of the pipeline, empty stages are interleaved between the occupied stages in order to ensure reliable data transfer between the stages.
Accordingly, those skilled in the art have recognized a long felt need for a new and improved video decompression system obviating the deficiencies of the prior art systems. The present invention clearly fulfills this need.
SUMMARY OF INVENTION
The present invention embodies a method of asynchronously accessing a random access memory having a plurality of rows and columns, where each row has a wordline connected to read and write row decoders and each column is connected to bitlines. A row address is assigned to a first and second row, i.e. a pair of rows. A read address is provided to the read row decoder and a write address is provided to the write row decoder. The read and write addresses are decoded by the read and write row decoders, respectively, and one of the first or second rows is selected for reading. Asynchronous with selecting one of the first or second rows for reading, one of the first or second rows is selected for writing. Data is then read from the row selected for reading and asynchronously data is written into the row selected for writing. Signals are provided which coordinate the reading and writing of data, where in the event reading or writing is being performed, another of the reading or writing is deferred until completion of the first reading or writing. The read row decoder and the write decoder are unable to select the same row simultaneously and a read control circuit and a write control circuit are unable to select the same bit lines simultaneously.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates data flow through a preferred embodiment in the present invention;
FIG. 2 shows an example of a 13 bit word used to address 8 bit data in a 64×32 RAM;
FIG. 3 is a functional block diagram of a Register file in the present invention;
FIG. 4 illustrates data flow in a register file as shown in FIG. 3;
FIG. 5 is a block diagram illustrating register file address decoding, in accordance with the present invention;
FIG. 6 is a block diagram of a Microcodable State Machine, in accordance with the present invention;
FIG. 7 shows a fixed width word, in accordance with the present invention, used for addressing and having an address field, a substitution field and a substitution header;
FIG. 8 is a block diagram of one example of an Arithmetic Core in accordance with the present invention;
FIG. 9 illustrates the basis steps in a method, in accordance with the present invention, for performing an IDCT on input data;
FIG. 10 is a block diagram illustrating the combined, simplified, two-stage architecture of an IDCT system, in accordance with the present invention;
Figure 11 is a simplified block diagram of an integrated circuit that comprises the main system components of the IDCT shown in FIG. 10;
FIG. 12a and FIG. 12b taken together are a block diagram of a pre-processing circuit corresponding to one of the main system component; for ease of explanation, these figures are referred collectively as FIG. 12;
FIG. 13a, FIG. 13b and FIG. 13c depict timing diagrams which illustrate the relationships between timing and control signals in the IDCT system of a preferred embodiment;
FIG. 14a and FIG. 14b taken together are a block diagram of a common processing circuit in the IDCT system for ease of explanation, these figures are referred to collectively as FIG. 14;
FIG. 15a, FIG. 15b, FIG. 15c and FIG. 15d taken together are a block diagram of a post-processing circuit which corresponds to another main component of the system and are referred collectively as FIG. 15;
FIGS. 16a and 16b are a block diagram, in accordance with the present invention illustrating an IDCT having a twin data stream, a transpose RAM and an improved buffer and are collectively FIG. 16.
FIGS. 17a, 17b, 17c, 17d, 17e, 17f are a block diagram showing in further detail the 1-D IDCT system shown in FIGS. 16a and 16b and are collectively FIG. 17;
FIGS. 18a and 18b are a block diagram showing greater detail of the transform system as shown in FIGS. 17a-17f and are collectively FIG. 18;
FIGS. 19a and 19b are a block diagram showing in greater detail the input buffer shown in FIGS. 18a and 18b and are collectively FIG. 19;
FIGS. 20a and 20b are a simplified block diagram of a pre-processing circuit "PREC", in accordance with the present invention and are collectively FIG. 20;
FIGS. 21a and 21b are a block diagram illustrating a common processing circuit "CBLK" found in the IDCT and are collectively FIG. 21;
FIGS. 22a and 22b are a block diagram of a post-processing circuit "POSTC" and are collectively FIG. 22;
FIGS. 23a, 23b, 23c, 23d are another illustration of the post-processing circuit shown in FIGS. 22a and 22b and are collectively FIG. 23;
FIG. 24 is a block diagram depicting a round and saturate block, in accordance with the present invention;
FIGS. 25a and 25b are a block diagram of an output buffer in the present invention and are collectively FIG. 25;
FIGS. 26a and 26b (collectively FIG. 26) are a block diagram of a control shift register, in accordance with the present invention;
FIGS. 27a, 27b, 27c (collectively FIG. 27) are a block diagram of a control shift register decode in the present invention;
FIGS. 28a, 28b, 28c (collectively FIG. 28) are depict a control shift register and an input control buffer;
FIGS. 29a-1, 29a-2, 29b, 29c, 29d, 29e, 29f (collectively FIG. 29) illustrate a control circuit for a T2 data stream;
FIGS. 30a, 30b, 30c, 30d (collectively FIG. 30) show data in a counter for a T1 data stream;
FIGS. 31a, 31b, 31c, 31d, 31e (collectively FIG. 31) depict data in a counter for a T2 data stream in the present invention;
FIG. 32 is a timing diagram showing the initialization of the IDCT and associated circuitry
FIG. 33 is a timing diagram showing the interleaving of T1 and T2 data;
FIG. 34 is a timing diagram illustrating slippage and recovery of T2 data;
FIG. 35 is a timing diagram depicting a flushing operation of the IDCT and associated circuitry in the present invention;
FIG. 36 illustrates start-up of the system, in accordance with the present invention;
FIG. 37 depicts slippage and recovery in the early stages of interleaving T1 and T2 data;
FIG. 38 illustrates another preferred embodiment of the IDCT system shown in FIGS. 16a through 37;
FIG. 39 shows MPEG information streams being demultiplexed, in accordance with the present invention, into elementary streams containing data and timestamp information;
FIG. 40 depicts a first embodiment of an elementary stream timestamp error determination and time synchronization system, in accordance with the present invention;
FIG. 41 illustrates a second embodiment of an elementary stream timestamp error determination and time synchronization system, in accordance with the present invention;
FIG. 42 shows a third embodiment of an elementary stream timestamp error determination and time synchronization system, in accordance with the present invention;
FIG. 43 depicts a first embodiment of a video timestamp error determination and time synchronization system, in accordance with the present invention;
FIG. 44 illustrates a second embodiment of a video timestamp error determination and time synchronization system, in accordance with the present invention;
FIG. 45 shows the second embodiment of a video timestamp error determination and time synchronization system as shown in FIG. 44 and operating at 30 Hz;
FIG. 46 shows timestamp information flow through the system of the present invention;
FIG. 47 is a block diagram illustrating synchronization time information being processed by a microprogrammable state machine;
FIG. 48 is a block diagram illustrating a first preferred embodiment of the present invention;
FIG. 49 is another block diagram illustrating the first preferred embodiment of the present invention;
FIG. 50 depicts a second preferred embodiment of the present invention;
FIG. 51 illustrates a detailed method of addressing used by the second preferred embodiment, in accordance with the present invention;
FIG. 52 is a block diagram showing an apparatus for decoding Huffman VLCs, in accordance with the present invention;
FIGS. 53a, 53b, 53c, 53d are collectively FIG. 53 and are a schematic diagram showing the overall structure of the parallel huffman decoder of the present invention;
FIGS. 54a and 54b are collectively FIG. 54 and are a schematic diagram illustrating a ROM adapted for decoding parallel huffman codes;
FIG. 55 illustrates a first embodiment of a ROM adapted for decoding parallel huffman codes;
FIG. 56 illustrates a second embodiment of a ROM adapted for decoding parallel huffman codes;
FIG. 57 depicts a third embodiment of a ROM adapted for decoding parallel huffman codes;
FIG. 58 is a block diagram illustrating the primary system component of one embodiment of the present invention;
FIG. 59 is a block diagram depicting the start code detector of the present invention;
FIG. 60 is a block diagram showing the parser of the present invention;
FIG. 61 is a block diagram depicting the primary components of the spatial processing circuitry of the present invention;
FIG. 62 is a block diagram illustrating the display circuitry, in accordance with the present invention;
FIG. 63 illustrates one embodiment of timestamp management, in accordance with the present invention;
FIG. 64 shows another embodiment of timestamp management in the present invention;
FIG. 65 is a block diagram depicting the hardware components of the system of the present invention;
FIG. 66 is a block diagram providing an overview of the system components of the microcontroller of the present invention;
FIG. 67 is a simplified diagram illustrating the Arithmetic core of the present invention;
FIG. 68 illustrates the ALU of the present invention;
FIG. 69 depicts a register file, in accordance with the present invention;
FIG. 70 illustrates the writing to independent bus registers in the present invention;
FIG. 71 illustrates frame-based prediction wherein vector[1]=0 and vector[0]=0;
FIG. 72 depicts frame-based prediction wherein vector[1]=0 and vector[0]=1;
FIG. 73 shows frame-based prediction wherein vector[1]=1 and vector[0]=0;
FIG. 74 illustrates frame-based prediction wherein vector[1]=0 and vector[0]=1;
FIG. 75 depicts field-based prediction wherein motion-- vertical-- field-- select=0 and vector[0]=0;
FIG. 76 illustrates field-based prediction wherein motion-- vertical-- field-- select=0 and vector[0]=1;
FIG. 77 similarly illustrates field-based prediction wherein motion-- vertical-- field-- select=1 and vector[0]=0;
FIG. 78 shows field-based prediction wherein motion-- vertical-- field-- select=1 and vector[0]=1;
FIG. 79 shows field-based prediction in frame pictures wherein motion-- vertical-- field-- select=0 and vector[0]=0;
FIG. 80 illustrates the prediction of FIG. 79 wherein motion-- vertical-- field-- select=0 and vector[0]=1;
FIG. 81 shows the prediction mode of FIG. 79 wherein motion-- vertical-- field-- select=1 and vector[0]=0;
FIG. 82 shows the prediction mode of FIG. 79 wherein both motion-- vertical-- field-- select and vector[0]=1;
FIG. 83 illustrates an additional mode of prediction filtering;
FIG. 84 shows still another prediction mode;
FIG. 85 illustrates yet another prediction mode, in accordance with the present invention;
FIG. 86 shows another prediction mode of the present invention;
FIG. 87 is a block diagram illustrating the organization of the various system components of the display system of the present invention;
FIG. 88 depicts a 4:3 filtering operation;
FIG. 89 depicts a 3:2 filtering operation;
FIG. 90 illustrates a 2:1 filtering operation of the present invention;
FIG. 91 shows a three tap filter used in the present invention;
FIG. 92 illustrates the repetition of erroneous pels;
FIG. 93 depicts the filed-- id signal of the present invention;
FIG. 94 shows the horizontal timing points (cycles), in accordance with the present invention;
FIG. 95 illustrates the PAL vertical timing at 625 lines per field, in accordance with the present invention;
FIG. 96 illustrates the NTSCV vertical timing at 525 lines per field, in accordance with the present invention;
FIG. 97 shows a horizontal counting machine, in accordance with the present invention;
FIG. 98 illustrates border generation in the present invention;
FIG. 99 depicts picture cropping, in accordance with the present invention;
FIG. 100 is a block diagram illustrating the present invention as a chip;
FIG. 101 illustrates the sysclock requirements of the present invention;
FIG. 102 depicts the two-wire protocol on a coded data interface, in accordance with the present invention;
FIG. 103 shows a DATA token of the present invention;
FIG. 104 shows a FLUSH token of the present invention;
FIG. 105 illustrates the timing of the coded data interface;
FIG. 106 depicts using non-even mark-space ratio CDCLOCK, in accordance with the present invention;
FIG. 107 shows output timing in 16 bit mode in the present invention;
FIG. 108 illustrates output timing in 8 bit mode in the present invention;
FIG. 109 shows the timing of the video output interface in the present invention;
FIG. 110 depicts video output mode signals, in accordance with the present invention;
FIG. 111 shows horizontal timing in the present invention;
FIGS. 112a and 112b (collectively FIG. 112) show the vertical timing for a 525 line system;
FIGS. 113a and 113b (collectively FIG. 113) depict the vertical timing for a 625 line system;
FIG. 114 illustrates the sync and blanking signals for a 525 line system, in accordance with the present invention;
FIG. 115 shows the sync and blanking signals for a 625 line system, in accordance with the present invention;
FIG. 116 illustrates a zero SDRAM connection configuration in the present invention;
FIG. 117 shows one SDRAM connection configuration in the present invention;
FIG. 118 depicts a two SDRAM connection configuration, in accordance with the present invention;
FIG. 119 illustrates a three SDRAM connection configuration
FIG. 120 is a flow chart depicting the flag-- picture-- end operation, in accordance with the present invention;
FIG. 121 is a flow chart showing the start-- code-- search operation, in accordance with the present invention;
FIG. 122 shows timestamp modification, in accordance with the present invention
FIG. 123 illustrates the read timing for the microprocessor interface; and
FIG. 124 shows the write timing for the microprocessor interface.
In the ensuing description of the practice of the invention, the following terms are frequently used and are generally defined by the following glossary:
GLOSSARY
BLOCK: An 8-row by column matrix of pels, or 64 DCT coefficients (source, quantized or dequantized).
CHROMINANCE (COMPONENT): A matrix, block or single pel representing one of the two color difference signals related to the primary colors in the manner defined in the bit stream. The symbols used for the color difference signals are Cr and Cb.
CODED REPRESENTATION: A data element as represented in its encoded form.
CODED VIDEO BIT STREAM: A coded representation of a series of one or more pictures as defined in this specification.
CODED ORDER: The order in which the pictures are transmitted and decoded. This order is not necessarily the same as the display order.
COMPONENT: A matrix, block or single pel from one of the three matrices (luminance and two chrominance) that make up a picture.
COMPRESSION: Reduction in the number of bits used to represent an item of data.
DECODER: An embodiment of a decoding process.
DECODING (PROCESS): The process defined in this specification that reads an input coded bitstream and produces decoded pictures or audio samples.
DISPLAY ORDER: The order in which the decoded pictures are displayed. Typically, this is the same order in which they were presented at the input of the encoder.
ENCODING (PROCESS): A process, not specified in this specification, that reads a stream of input pictures or audio samples and produces a valid coded bitstream as defined in this specification.
INTRA CODING: Coding of a macroblock or picture that uses information only from that macroblock or picture.
LUMINANCE (COMPONENT): A matrix, block or single pel representing a monochrome representation of the signal and related to the primary colors in the manner defined in the bit stream. The symbol used for luminance is Y.
MACROBLOCK: The four 8 by 8 blocks of luminance data and the two (for 4:2:0 chroma format) four (for 4:2:2 chroma format) or eight (for 4:4:4 chroma format) corresponding 8 by 8 blocks of chrominance data coming from a 16 by 16 section of the luminance component of the picture. Macroblock is sometimes used to refer to the pel data and sometimes to the coded representation of the pet values and other data elements defined in the macroblock header of the syntax defined in this part of this specification. To one of ordinary skill in the art, the usage is clear from the context.
MOTION COMPENSATION: The use of motion vectors to improve the efficiency of the prediction of pet values. The prediction uses motion vectors to provide offsets into the past and/or future reference pictures containing previously decoded pel values that are used to form the prediction error signal.
MOTION VECTOR: A two-dimensional vector used for motion compensation that provides an offset from the coordinate position in the current picture to the coordinates in a reference picture.
NON-INTRA CODING: Coding of a macroblock or picture that uses information both from itself and from macroblocks and pictures occurring at other times.
PEL: Picture element.
PICTURE: Source, coded or reconstructed image data. A source or reconstructed picture consists of three rectangular matrices of 8-bit numbers representing the luminance and two chrominance signals. For progressive video, a picture is identical to a frame, while for interlaced video, a picture can refer to a frame, or the top field or the bottom field of the frame depending on the context.
PREDICTION: The use of a predictor to provide an estimate of the pel value or data element currently being decoded.
RECONFIGURABLE PROCESS STAGE (RPS): A stage, which in response to a recognized token, reconfigures itself to perform various operations.
SLICE: A series of macroblocks.
TOKEN: A universal adaptation unit in the form of an interactive interfacing messenger package for control and/or data functions.
START CODES [SYSTEM AND VIDEO]: 32-bit codes embedded in a coded bitstream that are unique. They are used for several purposes including identifying some of the structures in the coding syntax.
VARIABLE LENGTH CODING; VLC: A reversible procedure for coding that assigns shorter code-words to frequent events and longer code-words to less frequent events.
VIDEO SEQUENCE: A series of one or more pictures.
DETAILED DESCRIPTIONS
The forthcoming "Detailed Description of the Invention" contains the following Sections:
1) Detailed Description of the Invention for Memory Addressing
Variable Length Fields Within a Fixed Width Word
Using Fixed Width Word with Variable Length Fields to Perform Address Substitution
Addressing Variable Width Data with a Fixed Width Word
Microcodable State Machine Structure
Arithmetic Core
2) Detailed Description of the Invention for Transforming Data using a Common Processing Block
Theoretical Background of the Invention
3) Detailed Description of Invention for Time Synchronization
4) Detailed Description of the Invention for Asynchronous Swing Buffering
5) Detailed Description of the Invention for Storing Video Information
6) Detailed Description of the Invention for a Parallel Huffman Decoder
The Huffman Code ROM
Maximizing Throughput
FLCs and Tokens
Implementation
7) MORE DETAILED DESCRIPTION
DETAILED DESCRIPTION OF THE INVENTION
As an introduction to the illustrative embodiment(s) of the most general features of the invention, and referring more particularly to FIG. 1 of the drawings, the data flow through the preferred embodiment 200 of the invention is shown. The embodiment of the present invention is preferably implemented using a two-wire pipeline system having various control and DATA tokens. The major elements of the system are a Start Code Detector 201, a Video Parser 202 incorporating a Huffman Decoder 203 and a Microprogrammable State Machine (MSM) 204, an Inverse Discrete Cosine Transform (IDCT) 205, a synchronous DRAM controller 206 with an associated address generation unit 207, appropriate prediction circuitry 208 and display circuitry 209 which includes upsampling 210 and 211 and video timing generation 212.
This application relates to similar subject matter disclosed in British Patent Application number 9405914.4 entitled "Video Decompression" filed on Mar. 24, 1994, by Discovision Associates, and the latter application is specifically incorporated by reference in this application.
In accordance with the above, specific aspects, features and subsystem areas of the present invention will be referred to in greater detail below. In the drawings, like reference numerals denote like or corresponding parts throughout the various drawings and figures.
Detailed Description of the Invention for Memory Addressing
In accordance with the present invention, a method and apparatus for addressing memory is described herein. In particular, the present invention provides for deferring variable width bit fields with fixed width words. More particularly, the present invention provides a method of addressing variable width data with a fixed width word. In various forms of the embodiment, variable bit field is used to specify bits to be substituted into the word or to specify an unused portion of the word in addressing variable width data with a fixed width word. In addition, the system of the present invention includes a microcodable state machine having an arithmetic core.
The microcodable state machine is intended to be used for solving design problems where there is a need for versatile and/or complicated calculations. Examples of such designs include address generation, stream parsing and decoding, and filter tap coefficient calculations. In this regard, the addressing must cope with two different features: (1) variable length addresses to access varying width portions of words and (2) address substitution. In the present invention, a RAM having a 64×32 bit configuration can be addressed in partial words having 64×32 bit, 128×16 bit, 256×8 bit, 512×4 bit, 1024×2 bit, or 2048×1 bit formats.
Variable Length Fields Within a Fixed Width Word
In many applications, it is useful to define variable portions of a word (to be known as fields) for actions such as substitution, variable width data addressing, or the constriction of other parts of the word. The conventional method for defining variable portions of words is to have an additional word (or words) which specify the width of the field (or fields) within the word. In accordance with the present invention, a method for encoding this information within the word itself is described. The present method has the advantages of savings bits in the overall definition of the word, simplifying decoding of the encoded word and providing a more intuitive view of what has been encoded. Furthermore, this encoding method is applicable if the variable width fields are most or least significant bit justified within the word.
Accordingly, Table 1 shows two examples of variable width fields (marked "F") that are least significant bit justified defined within an eight bit word. A "w" marks other potential fields of these words.
              TABLE 1                                                     
______________________________________                                    
Bit number (hex)                                                          
           7      6     5    4   3    2    1    0                         
______________________________________                                    
Fixed word w      w     w    F   F    F    F    F                         
   w w w w w w F F                                                        
______________________________________
Table 2 shows the conventional method of encoding the fields shown in Table 1 using sufficient additional bits to specify the maximum width of the field in binary. (Bits marked "x" "don't care", i.e., their value is of no consequence. This method is clearly inefficient in its use of bits and, furthermore, provides a less intuitive form than that described in the present invention.
              TABLE 2                                                     
______________________________________                                    
Bit number (hex)                                                          
           7     6     5   4   3   2   1   0   Field Define               
______________________________________                                    
Fixed word w     w     w   x   x   x   x   x   1   0   1                  
   w w w w w w x x 0 1 0                                                  
______________________________________
The new method, in accordance with the present invention, defines the field within the word. This method defines the field by using a continuation marker and a termination marker. The field is specified, from one end of the field, as a series of continuation markers followed by a termination marker. In the case of a zero length field, however, only a termination marker is provided at the end of the word. Both the continuation marker and the termination marker are single bits, and they must be complementary. In addition, the field must be justified to either end of the word. Accordingly, the method of the present invention for encoding fields requires a width of only one bit extra over the original word width.
As shown in Table 3, the encoding of the fields shown in the Table 1, in accordance with the new method, is depicted. In this example, the continuation marker is "1" and the termination marker is "0". The field in this example is least significant bit justified.
              TABLE 3                                                     
______________________________________                                    
Bit number (hex)                                                          
              7     6     5   4   3    2   1   0                          
______________________________________                                    
Fixed word    w     w     w   0   1    1   1   1   1                      
  Continuation marker = 1; w w w w w w 0 1 1                              
  Termination marker = 0.                                                 
______________________________________
Therefore, the advantages of the encoding method, in accordance with the present invention, are:
1. A reduction in the number of bits needed in the encoding.
2. A simplification in the decoding process is required since the need for a "x to 1 of "decode of the "field define" shown in Table 1-2 that would normally be required is inherent in the encoding which is already in the form of 1 of 2x ; and
3. The encoding is in a more intuitive form allowing the field defined to be more easily identified.
Furthermore, the use of this encoding method of the present invention can also be used such that the termination marker and the continuation marker are inverted to provide that the encoding of Table 3 resembles that of Table 4. Hence, the use of "1" or "0" is used interchangeably throughout this application.
              TABLE 4                                                     
______________________________________                                    
Bit number (hex)                                                          
              7     6     5   4   3    2   1   0                          
______________________________________                                    
Fixed word    w     w     w   1   0    0   0   0   0                      
  Continuation marker = 1: w w w w w w 1 0 0                              
  Termination marker = 0.                                                 
______________________________________
As previously identified, the field encoded must be justified to either end of the word. Table 5 illustrates most significant justified fields, i.e., these are encoded in a similar way to least significant bit justified fields except that the field reaches from the most significant bit (hereinafter MSB) towards the least significant bit (hereinafter "LSB") up to and including the first termination marker. The encoding of the fields shown in Table 5 are shown in Table 6.
              TABLE 5                                                     
______________________________________                                    
Bit number (hex)                                                          
           7      6     5    4   3    2    1    0                         
______________________________________                                    
Fixed word F      F     F    F   F    w    w    w                         
   F F w w w w w w                                                        
______________________________________                                    

              TABLE 6                                                     
______________________________________                                    
Bit number (hex)    7     6   5   4    3   2   1   0                      
______________________________________                                    
Fixed word    1     1     1   1   1    0   w   w   w                      
  Continuation marker = 1; 1 1 0 w w w w w w                              
  Termination marker = 0.                                                 
______________________________________
Moreover, fields may be encoded from the least significant and most significant ends of the word simultaneously. For example, the two fields shown in Table 7 may be encoded as in Table 8, with the addition of just one bit for each field as described previously.
              TABLE 7                                                     
______________________________________                                    
Bit number (hex)                                                          
           7      6     5    4   3   2     1    0                         
______________________________________                                    
Fixed word F      F     F    F   w    w    F    F                         
   w w w w F F F F                                                        
______________________________________                                    

              TABLE 8                                                     
______________________________________                                    
Bit number (hex)   7     6   5   4   3   2   1   0                        
______________________________________                                    
Fixed word   1     1     1   1   0   w   w   0   1   1                    
  Continuation marker = 1; 0 w w w w 0 1 1 1 1                            
  Termination marker = 0.                                                 
______________________________________
Using a Fixed Width Word with Variable Length Fields to Perform Address Substitution
There are situations in which it is useful to substitute part of a memory address by another value. In this way it is possible to construct a data dependent address. The encoding method of the present invention can be applied to the addresses of a memory to specify what portion of the address is to be substituted. If a least significant bit justified variable length field is used in the address, a substitution field can be defined. For example, a 12 bit address 0baaaaaaaaaaaa encoded to have its five least significant bit substituted by the 12 bit value 0bcccccccccccc would be 0baaaaaaa011111 and produce the address 0baaaaaaaaccccc. Table 9 shows the encoding for substitution into a 12 bit address.
              TABLE 9                                                     
______________________________________                                    
Address substitution                                                      
  No. Bits                                                                
  substituted B A 9 8 7 6 5 4 3 2 1 0                                     
______________________________________                                    
0      a     a     a   a   a   a   a   a   a   a   a                      
                           a   1                                          
  1 a a a a a a a a a a a 0 1                                             
  2 a a a a a a a a a a b 1 1                                             
  3 a a a a a a a a a 0 1 1 1                                             
  4 a a a a a a a a 0 1 1 1 1                                             
  5 a a a a a a a 0 1 1 1 1 1                                             
  6 a a a a a a 0 1 1 1 1 1 1                                             
  7 a a a a a 0 1 1 1 1 1 1 1                                             
  8 a a a a 0 1 1 1 1 1 1 1 1                                             
  9 a a a 0 1 1 1 1 1 1 1 1 1                                             
  10 a a 0 1 1 1 1 1 1 1 1 1 1                                            
  11 a 0 1 1 1 1 1 1 1 1 1 1 1                                            
  12 0 1 1 1 1 1 1 1 1 1 1 1 1                                            
______________________________________
Addressing Variable Width Data with a Fixed Width Word
One embodiment of the present invention is for addressing a memory which can be accessed at its full width or in 2n widths up to its full width (these smaller words are called partial words). Hence, it will be shown how the variable field encoding of the present invention can be used to address this memory and to index those addresses into the memory.
To access a 64×32 bit Register file in widths of 32, 16, 8, 4, 2 and 1 bit requires different lengths of address, i.e., the implementation of this embodiment is a 64×32 bit memory which can be accessed as 64×32 bits, 128×16 bits, 256×8 bits, 512×4 bits, 1024×2 bits, or 2048×1 bit. It is seen that 5 bits are required to address one of the 64×32 bit locations, while 12 bits are required to address one of the 2048×1 bit locations. Hence, the addresses can be of variable length and, in fact, the width of the address specifies the address format of the memory. Accordingly, the address can be defined within a fixed word width by using a most significant justified variable width field which constricts the address and defines its width. This is illustrated in Table 10.
              TABLE 10                                                    
______________________________________                                    
Variable width addressing                                                 
  Data Width       A   9   8   7   6   5   4   3   2                      
                           1   0                                          
______________________________________                                    
1        1     a     a   a   a   a   a   a   a   a   a                    
                             a                                            
  2 0 1 a a a a a a a a a a                                               
  4 0 0 1 a a a a a a a a a                                               
  8 0 0 0 1 a a a a a a a a                                               
  16 0 0 0 0 1 a a a a a a a                                              
  32 0 0 0 0 0 1 a a a a a a                                              
______________________________________
To allow indexing of the address, a portion of it can be substituted using the same method described previously for address substitution. The substitution portion (or field) of the address can be defined by a least significant bit justified variable length field (The continuation marker "1"; termination marker "0") that is superimposed on top of those shown in Table 10. Using an address of an eight bit word, as an example, Table 11 shows how to define the number of the least significant bits to be substituted. The least significant bit added is the substitution indicator (marked "w"). The general case of a Fixed width word for substitution is shown in FIG. 2.
              TABLE 11                                                    
______________________________________                                    
Address substitution                                                      
  Bits to be                                                              
  substituted  A 9 8 7 6 5 4 3 2 1 0 w                                    
______________________________________                                    
0      0     0     0   1   a   a   a   a   a   as  a                      
                           a   0                                          
  1 0 0 0 1 a a a a a a a 0 1                                             
  2 0 0 0 1 a a a a a a 0 1 1                                             
  3 0 0 0 1 a a a a a 0 1 1 1                                             
  4 0 0 0 1 a a a a 0 1 1 1 1                                             
  5 0 0 0 1 a a a 0 1 1 1 1 1                                             
  6 0 0 0 1 a a 0 1 1 1 1 1 1                                             
  7 0 0 0 1 a 0 1 1 1 1 1 1 1                                             
  8 0 0 0 1 0 1 1 1 1 1 1 1 1                                             
______________________________________
In effect the substitute code is superimposed on top of the address that is already coded. From this coding, it can be seen that there are illegal addresses, most obviously 0x0000 and 0x3fff. In this case, a "0" must be in the bottom 9 bits to prevent substituting more than 8 bits and a "1" in the top 6 bits specifies an allowable access width. If one of these errors is detected, the access is undefined, but the Register file contents will not be affected.
In accordance with the present invention, the system for addressing and for accessing partial words in a register file is discussed below.
The conventional memory circuitry dictates that the memory must always be accessed at it full width. To achieve variable width accesses, a full (32 bit) width word is read. This full word is rotated until the partial word accessed is justified in the LSB. The upper parts of the word are extended to the full width and then output. Extending may encompass padding with zeros or ones, sign extending, using the sign bit of a sign-magnitude number as the new MSB or any similar conventional method. Extending is dependent on the mode of operation. When the partial word is input to and written back into the memory, it is multiplexed back into the rotated full word, which is then rotated back and written into the array. FIG. 3 shows these steps for the access of a 4 bit partial word in the fourth four bit word of the 32 bit word.
To access or read partial words, such as the highlighted four bit word shown in row "1" 213 of FIG. 3, the full width word must be rotated to place the partial word at the LSB, as shown in row "2" 214. As shown in row "3" 215, the four bit word is extended to create a full 32 bit word: This word can now be accessed.
As shown in FIG. 3, a full width word that has been selected to be written back is truncated to the width of the original partial word which is multiplexed into the word shown in row "2" 214. At the LSB position, this is shown in row "4" 216. The resulting word is rotated back in its original significance in the read word, this is shown in row "5" 217. This full word can now be written back into the register file.
The following list, therefore, summarizes the steps numbered in FIG. 3:
1. Full word read from memory;
2. 12 bit rotated right puts partial word into the LSB;
3. Extended to full word, then passed to output;
4. The inputted partial word is multiplexed into rotated full word from (2); and
5. 12 bit rotated left puts full word back to original state to be written.
The above accesses suggests the data flow structure of the memory that is shown in FIG. 4. The numbers in the structure refer to the above text and to FIG. 3.
The memory address must be decoded to control the above structure. It should be recognized that the MSB of any width of address is at the same significance with reference to the memory. The top six bits of a decoded address are a 32 bit word address, whereas the remainder is a bit address. Therefore, the stage of decoding (in parallel with the substitution) is to decode the address width defining variable field by detecting the position of the most significant termination marker. This allows the address to be MSB justified (shifting in zeros at the LSB). The top six bits can be used directly as a 32 bit word row address of the memory. The bottom five bits can be used to directly control both barrel shifters (as seen in FIG. 4), because, for example, an original 32 bit address will always have a shift of 0b00000 (these having been shifted when the address was MSB justified). Similarly, a 16 bit address can have a shift of 0bx0000, i.e., 0 or 16 bit shift and a 1 bit address can have a shift of 0bxx, i.e., 0 to 31 bit shifts. The extender and input multiplexer are controlled by the access width decode to mask out the output words and multiplex the input words to an appropriate significance, respectively. The block diagram of the decode is shown in FIG. 5. It can be seen that the decode of the two variable width fields for width and substitution can be done in parallel and independently.
FIG. 2 illustrates an example of a fixed width word 13 bits long for addressing variable width data and substitution as shown in the bottom two rows. For these examples, an eight bit word would have been addressed at location 0b110lssss, where "ssss" is substituted from another address source.
Microcodable State Machine Structure
In accordance with the present invention, the substitution into a memory address and the variable width accessing of a memory have been brought together in the implementation of a microcodable state machine the structure of which is shown in FIG. 6. The structure is one of a state machine 218 providing control of an arithmetic core 219 by way of a wide word of control signals called a microcode instruction. The arithmetic core 219, in turn, passes status flags and some data to the state machine 218.
The state machine 218, in accordance with the present invention, includes a memory containing a list of the microcode instructions. As with conventional microcodable state machines, it is capable of either proceeding through the list of microcode instructions contiguously or a jump can occur from one instruction to another. The jump address is in the form shown in FIG. 7. The substituted value comes from the Arithmetic core 219 as shown in FIGS. 6 and 8. This allows the construction of "jump tables" within the microcode programs. Thus, if a jump is made with 3 bits substituted, for example, there are eight possible contiguous locations that may be jumped to, each dependent on the value from the arithmetic core, i.e., it has so become a programmable jump.
Arithmetic Core
The arithmetic core 219, as shown in FIG. 8, includes a memory called a register file 221, an Arithmetic and Logic unit (ALU) 222, an input port 223 and an output port 224. These components are connected via buses and multiplexers. As previously stated, these components, and the multiplexers defining their connections, are entirely controlled by the microcode instruction issued by the state machine 218. The ALU 222 and the ports 223 and 224 are conventional, however, the register file 221 is a memory which allows variable width indexed accesses. The addresses to the register file 221 is coded directly into the microcode instruction.
There are many advantages of using this method of addressing to the register file. First, many locations in an application do not need to be the full width of the memory (32 bits in this case). Whilst it will cause no effect on the operation of the device to use a full width location, it is very wasteful of memory locations. Minimizing the number of memory locations will minimize the amount of space used by the memory and, therefore, minimize the capacitive loading in the register file. This maximizes the speed of the register file. Second, the indexing combined with the variable width of memory accessing allows the stepping through of locations of variable width. In the one bit case this allows an elegant implementation of long division and multiplication.
In summary, therefore, there is described a procedure for addressing memory having the following steps: (1) providing a fixed width word having a predetermined fixed number of bits to be used for addressing variable width data; (2) defining the fixed width word with a width defining field and an address field providing the width defining field with at least one bit to serve as a termination marker; (3) defining the address field with a plurality of bits defining the address of the data; and (4) varying the size of bits in the address field in inverse relation to the size of the variable width data varying the number of bits in the width defining field in direct relation to the size of the variable width data and maintaining a fixed width word for addressing variable width data while varying the width of the width defining field and the address field. In addition, a procedure for addressing memory having the following steps is described: (1) providing a fixed width word having a predetermined fixed number of bits to be used for addressing data; (2) defining the fixed width word with an address field and a substitution field; (3) defining the address field with a plurality of bits defining the address of the data; (4) defining a variable width substitution field with at least one substitution bit; (5) the substitution field has at least one bit to serve as a termination marker between the address field and the substitution field; and (6) using the substitution field to indicate substituted bits from a separate addressing source and maintaining a fixed width word for addressing variable width data while inversely varying the width of the address field and the width of the substitution field. In addition, a process for addressing variable width data in a memory is described as having the following steps: (1) providing a memory having words of predetermined width and composed of partial words; (2) rotating the partial word to be accessed to a least significant bit justification; (3) extending the remaining part of the word so that the accessed word will be recognized as the partial word; and (4) restoring the remaining part of the word and rotating the word until the partial word is restored to its original position.
Detailed Description of the Invention for Transforming Data Using a Common Processing Block
This present embodiment, in accordance with the present invention, relates to a method for the transformation of signals from a frequency to a time representation, as well as a digital circuit arrangement for implementing the transformation.
It is a common goal in the area of telecommunications to increase both information content and transmission speed. Each communications medium, however, imposes a limitation on transmission speed, as does the hardware at the transmitting and receiving end that must process the transmitted signals. A telegraph wire is, for example, typically a much faster medium for transmitting information than the mail is, even though it might be faster to type and read a mailed document than to tap out a telegraph key.
The method of encoding transmitted information also limits the speed at which information can be conveyed. A long-winded telegraph message will, for example, take longer to convey than a succinct message with the same information content. The greatest transmission and reception speed can therefore be obtained by compressing the data to be transmitted as much as possible, and then, using a high-speed transmission medium, to process the data at both ends as fast as possible, which often means the reduction or elimination of `bottlenecks` in the system.
One application in which it is essential to provide high-speed transmission of large amounts of data is in the field of digital television. Whereas conventional television systems use analog radio and electrical signals to control the luminance and color of picture elements (`pixels`) in lines displayed on a television screen, a digital television transmission system generates a digital representation of an image by conveying analog signals into binary `numbers` corresponding to luminance and color values for the pixels. Modem digital encoding schemes and hardware structures typically enable much higher information transmission rates than do conventional analog transmission systems. As such, digital televisions are able to achieve much higher resolution and much more life-like images than their conventional analog counterparts. It is anticipated that digital television systems including so-called High-Definition TV (HDTV) systems, will replace conventional analog television technology within the next decade in much of in the industrialized world. The conversion from analog to digital imaging, for both transmission and storage will, thus, be similar to the change-over from analog audio records to the now ubiquitous compact discs (CD's).
In order to increase the general usefulness of digital image technology, standardized schemes for encoding digital images have been adopted. Once such standardized scheme is known as the JPEG standard and is used for still pictures. For moving pictures, there are at present two standards, MPEG and H.261, both of which carry out JPEG-like procedures on each of the sequential frames of the moving picture. To gain advantage over using JPEG repeatedly, MPEG and H.261 operate on the differences between subsequent frames, taking advantage of the well-known fact that the difference, that is, the movement between frames, is small. It, therefore, takes less time or space to transmit or store the information corresponding to the changes rather than to transmit or store equivalent still-picture information as if each frame in the sequence were completely unlike the frames closest to it in the sequence.
For convenience, all the current standards operate by breaking an image or picture into tiles or blocks, each block consisting of a piece of the picture eight pixels wide by eight pixels high. Each pixel is then represented by three (or more) digital numbers known as `components` of that pixel. There are many different ways of breaking a colored pixel into components, for example, using standard notation, e.g., YUV, YCr, Cb, RGB, etc. All the conventional JPEG-like methods operate on each component separately.
It is well known that the eye is insensitive to high-frequency components (or edges) in a picture. Information concerning the highest frequencies can usually be omitted altogether without the human viewer noticing any significant reduction in image quality. In order to achieve this ability to reduce the information content in a picture by eliminating high-frequency information without the eye detecting any loss of information, the 8-by-8 pixel block containing spatial information (for example, the actual values for luminance) must be transformed in some manner to obtain frequency information. The JPEG, MPEG and H.261 standards all use the known Discrete Cosine Transform to operate on the 8-by-8 spatial matrix to obtain an 8-by-8 frequency matrix.
As described above, the input data represents a square area of the picture. In transforming the input data into the frequency representation, the transform that is applied must be two-dimensional, but such two-dimensional transforms are difficult to compute efficiently. The known, two-dimensional Discrete Cosine Transform (DCT) and the associated inverse DCT (IDCT), however, have the property of being "separable". This means that rather than having to operate on all 64 pixels in the eight-by-eight pixel block at one time, the block can first be transformed row-by-row into intermediate values, which are then transformed column-by-column into the final transformed frequency values.
A one-dimensional DCT of order N is mathematically equivalent to multiplying two N-by-N matrices. In order to perform the necessary matrix multiplication for an eight-by-eight pixel block, 512 multiplications and 448 additions are required, so that 1,024 multiplications and 896 additions are needed to perform the full 2 dimensional DCT on the 8-by-8 pixel block. These arithmetic operations, and especially multiplication, are complex and slow and, therefore, limit the achievable transmission rate. They also require considerable space on the silicon chip used to implement the DCT.
The DCT procedure can be rearranged to reduce the amount of computation required. There are, at present, two main methods used for reducing the computation required for the DCT, both of which use "binary decimation". The term "binary decimation" means than an N-by-N transform can be computed by using two N2-by-N2 transformations, plus some computational overhead whilst arranging this. Whereas the eight-by-eight transform requires 512 multiplications and 448 additions, a four-by-four transform requires only 64 multiplications and 48 additions. Binary decimation, thus, saves 284 multiplications and 352 additions and the overhead incurred in performing the decimation is typically insignificant compared to the reduction in computation.
At present, the two main methods for binary decimation were developed by Eong Gi Lee (`A New Algorithm to Compute the DCT`) IEEE Transactions on Acoustics, Speech and Signal Processing, Vol. Assp 32, No. 6, p 1243 December 1984) and Wen-Hsiung Chen (`A Fast Computational Algorithm for the DCT`, Wen-Hsiung Chen, C. Harrison Smith, S. C. Pralick, IEEE Transactions on Communications, Col. Com 25, No. 9 1004, September 1977). Lee's method makes use of the symmetry inherent in the definition of the inverse DCT and, by using simple cosine identities, it defines a method for recursive binary decimation. The Lee approach is only suitable for the IDCT.
The Chen method uses a recursive matrix identity that reduces the matrices into diagonals only. This method provides easy binary decimation of the DCT using known identities for diagonal matrices.
A serious disadvantage of the Lee and Chen methods is that they are unbalanced in respect of when multiplications and additions must be performed. Essentially, both of these methods require that many additions be followed by many multiplications, or vice versa. When implementing the Lee or Chen methods in hardware, it is, therefore, not possible to have parallel operation of adders and multipliers. This reduces their speed and efficiency since the best utilization of hardware is when all adders and multipliers are used all the time.
An additional disadvantage of such known methods and devises for performing DCT and IDCT operations is that it is usually difficult to handle the so-called normalization coefficient, and known architectures require adding an additional multiplication time when all the multipliers are being used.
Certain known methods for applying the forward and inverse DCT to video data are very simple and highly efficient for a software designer who need not be concerned with the layout of the semiconductor devices which perform the calculations. Such methods, however, often are far too slow or are too complex in semiconductor architecture and hardware interconnections to perform satisfactorily at the transmission rate desired for digital video.
Yet another shortcoming of existing methods and hardware structures for performing DCT and IDCT operations on video data is that they require floating-point internal representation of numerical values. To illustrate this disadvantage, assume that one has a calculator that is only able to deal with three-digit numbers, including digits to the right of the decimal point (if any). Assume further that the calculator is to add the numbers 12.3 and 4.56 (Notice that the decimal point is not fixed relative to the position of the digits in these two numbers. In other words, the decimal point is allowed to `float`). Since the calculator is not able to store the four digits required to fully represent the answer 16.86, the calculator must reduce the answer to three digits either by truncating the answer by dropping the right-most `6`, yielding an answer of 16.8, or it must have the necessary hardware to round the answer up to the closest three-digit approximation 16.9.
As this very simple example illustrates, if floating-point arithmetic is required, one must either accept a loss of precision or include highly complicated and space-wasting circuitry to minimize rounding error. Even with efficient rounding circuitry, however, the accumulation and propagation of rounding or truncation errors may lead to unacceptable distortion in the video signals. This problem is even greater when the methods for processing the video signals require several multiplications, since floating point rounding and truncation errors are typically greater for multiplication than for addition.
A much more efficient DCT/IDCT method and hardware structure would ensure that the numbers used in the method could be represented with a fixed decimal point, but in such a way that the full dynamic range of each number could be used. In such a system, truncation and rounding errors would either be eliminated or, at least, greatly reduced.
In the above example, if the hardware can handle four digits, no number greater than 99.99 were ever needed, and every number had the decimal point between the second and third places, then the presence of the decimal point would not affect calculations at all. Accordingly, the arithmetic could be carried out just as if every number were an integer, e.g., the answer 1230+0456=1686 would be just as clear as 12.30+4.56=16.86, since one would always know that the `1686` should have a decimal point between the middle `6` and `8`. Alternatively, if numbers (constant or otherwise) are selectively scaled or adjusted so that they all fall within the same range, each number in the range couldis ao be accurately and unambiguously represented as a set of integers.
One way of reducing the number of multipliers needed is simply to have a single multiplier that is able to accept input data from different sources. In other words, certain architectures use a single multiplier to perform the multiplications required in different steps of the DCT or IDCT calculations. Although such "crossbar switching" may reduce the number of multipliers required, it means that large complicated multiplexer structures must be included instead to select the inputs to the multiplier, to isolate others from the multiplier, and to switch the appropriate signals from the selected sources to the inputs of the multiplier. Additional large-scale multiplexers are also required to switch the large number of outputs from the shared multipliers to the appropriate subsequent circuitry. Crossbar switching or multiplexing is, therefore, complex, is generally slow (because of the extra storage needed) and costs are significant in a final semiconductor implementation.
Still another drawback of existing architectures, including the "crossbar switching" is that they require general purpose multipliers. In other words, existing systems require multipliers for which both inputs are variable. As is well known, implementations of digital multipliers typically include rows of adders and shifters such that, if the current bit of a multiplier word is a `one` the value of the multiplicand is added into the partial result, but not if the current bit is a `zero`. Since a general purpose multiplier must be able to deal with the case in which every bit is a `1`, a row of adders must be provided for every bit of the multiplier word.
By way of example, assume that data words are 8 bits wide and that one wishes to multiply single inputs by 5. An 9-bit representation of the number 5 is 00000101. In other words, digital multiplication by 5 requires only that the input value be shifted to the left two places (corresponding to multiplication by 4) and then added to its up-shifted value. The other six positions of the coefficients have bit values of `0`, so they would not require any shifting or additional steps.
A fixed-coefficient multiplier, that is, in this case, a multiplier capable of multiplying only by five, would require only a single shifter and a single adder in order to perform the multiplication (disregarding circuitry needed to handle carry bits). A general purpose multiplier, in contrast, would require shifters and adders for each of the eight positions, even though six of them would never need to be used. As the example illustrates, fixed coefficients can simplify the multipliers since they allow the designer to eliminate rows of adders that correspond to zeros in the coefficient, thus saving silicon area.
In an IDCT method, in accordance with the present invention, a one-dimensional IDCT for each N-row and N-column of N-by-N pixel blocks is decimated and a 1-D IDCT is performed separately on the N-2 even-numbered pixel input words and the N-2 odd-numbered pixel input words.
In a preferred embodiment, N=8 according to the JPEG standard. The two-dimensional IDCT result is then obtained by performing two one-dimensional IDCT operations in sequence (with an intermediate reordering-transposition-of data).
In a common processing step, for N=8, a first pair of input values is passed without need for multiplication to output adders and subtractors. Each of a second pair of input values is multiplied by each of two constant-efficient values corresponding to two scaled cosine values. No other multiplications and only one subtraction and one addition are required in the common processing step. The second pair is then added or differenced pairwise with the first pair of input values to form even or odd resultant values.
In a pre-common processing stage, the lowest order odd input word is pre-multiplied by the square root of two and the odd input words are summed pairwise before processing in the common processing block In a post-common processing stage, intermediate values corresponding to the processed odd input words are multiplied by predetermined constant coefficients to form odd resultant values.
After calculation of the even and odd resultant values, the N/2 high-order outputs are formed by simple subtraction of the odd resultant values from the even resultant values, and the N/2 low-order outputs are formed by simple addition of the odd resultant values and the even resultant values.
For both the DCT (at the transmission end of a video processing system) and the IDCT (at the receiving end, which incorporates one or more of the various aspects of the present invention), the values are preferably and deliberately scaled downward by a factor of two by a simple binary right shift. This deliberate, balanced, upward scaling eliminates several multiplication steps that are required according to conventional methods.
According to another aspect of the method, in accordance with the present invention, selected bits of constant coefficient or intermediate resulting data words are rounded or adjusted by predetermined setting of selected bits to either `1` or `0`.
Two-dimensional transformation of pixel data is carried out by a second, identical 1-D operation on the output values from the first 1-D IDCT processing steps.
An IDCT system, according to yet another aspect of the present invention, includes a pre-common processing circuit, and a common processing circuit, in which the pre-common, common, and post-common processing calculations are performed on input data words. A supervisory controller generates control signals to control the loading of various system latches; preferably, to serially time-multiplex the application of the N/2 even and N/2 odd-numbered input words to input latches of the pre-common block to direct addition of the even and odd resultant values to form and latch low order output signals and to direct subtraction of the odd resultant values from the even resultant values to form and latch the high-order output signals and to sequentially control internal multiplexers.
In the present invention, even and odd input words are preferably processed in separate passes through the same processing blocks. Input data words are preferably (but not necessarily) latched, not in strictly ascending or descending order, but rather in an order enabling an efficient `butterfly` structure for the data path.
Furthermore, at least the common processing circuit may be configured as a pre-logic circuit, with no clock or control signals required for its proper operation, as may be other processing blocks, depending on the particular application.
No general-purpose multipliers (with two variable inputs) are required. Rather, constant coefficient multipliers are included throughout the preferred embodiment. Furthermore, fixed-point integer arithmetic devices are included in the preferred embodiment of the invention and can be so designed as to provide a method and system for performing IDCT transformation of video data with one or more of the following features:
1. Constant use of all costly arithmetic operations;
2. In order to reduce the silicon area needed to implement the IDCT, there are a small number of storage elements (such as latches), preferably no more than required for efficient pipelining of the architecture, coupled with a small number of constant coefficient multipliers rather than general purpose multipliers that require extra storage elements;
3. Operations are arranged so that each arithmetic operation does not need to use sophisticated designs, for example, if known `ripple adders` are used, these would allow sufficient time to `resolve` (see below) or produce their answers; if operations are arranged in such a way that other devises precede the rearranging operations so as to avoid delay and to allow greater throughput and efficiency;
4. One is able to generate results in a natural order;
5. No costly, complex, crossbar switching is required;
6. The architecture is able to support much faster operations; and
7. The circuitry used to control the flow of data through the transform hardware can be small in area.
Theoretical Background of the Invention
In order to understand the purpose and function of the various components and the advantages of the signal processing method used in the IDCT system according to the present invention, it is helpful to understand the system's theoretical basis.
Separability of a Two-Dimensional IDCT
The mathematical definition of a two-dimensional forward discrete cosine transforms (DCT) for an N×N block of pixels is as follows, where U(j,k) are the pixel frequency values corresponding to the pixel absolute values X(m,n) ##EQU1##
The terms 2N govern the dc level of the transform, and the coefficients c(j), c(k) are known normalization factors.
The expression for the corresponding inverse discrete cosine transform, that is for the IDCT, is as follows: ##EQU2##
The forward DCT is used to transform spatial values (whether representing characteristics such as luminance directly, or representing differences, such as in the MPEG standard) into their frequency representation. The inverse DCT, as its name implies, operates the other `direction`, that is, the IDCT transforms the frequency values back into spatial values.
In the expression, Equation 2, (E2), note that the cosine functions each depend on only one of the summation indices.
The expression E2 can therefore be rewritten as: ##EQU3##
This is the equivalent of a first one-dimensional IDCT performed on the product of all terms that depend on k and n, followed, after a straightforward standard data transposition by a second one-dimensional IDCT using as inputs the outputs of the first IDCT operation.
Definition of the 1-D IDCT
A 1-dimensional N-point IDCT (where n is an even number) is defined by the following expression. ##EQU4## and where y(n) are the N inputs to the inverse transformation function and x(k) are its N outputs. As in the 2-D case, the formula for the DCT has the same structure under the summation sign, but with the normalization constant outside the summation sign and with the x and y vectors switching places in the equation.
Resolution of a 1-D IDCT
As is shown above, the 2-D IDCT can be calculated using a sequence of 1-D IDCT operations separated by a transpose. In accordance to one embodiment, each of these 1-D operations is, in turn, broken down into sub-procedures that are then exploited to reduce even further the required size and complexity of the semiconductor implementation.
Normalization of Coefficients
As is discussed above, an important design goal for IDCT hardware is the reduction of the required number of multipliers that must be included in the circuitry. Most methods for calculating the DCT of IDCT, therefore, attempt to reduce the number of multiplications needed. According to this embodiment, however, all the input values are deliberately scaled upward by a factor of the square root of two. In other words, using the method according to this embodiment of the present invention, the right-hand side of the IDCT expression (E) is deliberately multiplied by the square root of two.
According to this embodiment, two 1-D IDCT operations are performed in series (with an intermediate transpose) to yield the final 2-D IDCT result. Each of these 1-D operations includes a multiplication by the same square root of two factor. Since the intermediate transposition involves no scaling, the result of two multiplications by the square root of two in series is that the final 2-D results will be scaled upward by a factor two. To obtain the unscaled value, the circuitry need then only divide by two. Since the values are all represented digitally, this can be accomplished easily by a simple right shift of the data. As is made clearer below, the upward scaling by the square root of two in each 1-D IDCT stage and final down-scaling by 2 is accomplished by adders, multipliers and shifters all within the system's hardware, so that the system places no requirements for scaled inputs on the other devises to which the system may be connected. Because of this, the system is compatible with other conventional devises that operate according to the JPEG or MPEG standards. Normalization according to this embodiment of the present invention, therefore, eliminates the need for hardware multipliers within the IDCT semiconductor architecture for at least two square root of two multiplication operations. As is explained below in greater detail, the single additional multiplication step (upward scaling by the square root of two) of the input data in each 1-D operation leads to the elimination of still other multiplication steps that are required when using conventional methods.
Separation of the 1-D IDCT into High and Low-Order Outputs
Expression E can now be evaluated separately for the N/2 low-order outputs (k=0, 1, . . . , N/2-1) and the N/2 high order outputs (k=N/2, k=N/2+1, . . . N). For N=8, this means that one can first transform the inputs to calculate y(0), y(1), y(2) and y(3), and then transform the inputs to calculate y(4), y(5), y(6) and y(7).
Introduce the variable k'=(N-1-k) for the high-order outputs (k=N/2+1, . . . , N), so that k' varies from (N/2-1) to N as k varies fm ro(N/2+1) to N. For N=8, this means that k'=(3,2,1,0) for k=(4,5,6,7). It can then be shown that expression E can be divided into the following two subexpressions E5 (which is the same as E except for the interval of summation) and E6:
Low order outputs: ##EQU5##
High-order outputs: ##EQU6##
Note that both E5 and E6 have the same structure under the summation sign except that the term (-1)n changes the sign of the product under the summation sign for the odd-numbered inputs (n odd) for the upper N2 output values and except that the y term will be multiplied by c(O)=1/√2.
Separation of the IDCT into Even and Odd Inputs
Observe that the single sum in the 1-D IDCT expression E4 can also be separated into two sums: one for the even-numbered inputs (for N=8 y(0), y(2), y(4), and y(6) and one for the odd-numbered inputs (for n=8, y(1), y(3), y(5), and y(7). Let g(k) represent the partial sum for the even-numbered inputs and h(k) represent the partial sum for the odd-numbered inputs.
Thus: ##EQU7##
Now recall the known cosine identity:
2.cos A.cos B=cos(A+B)+cos(A-B), and set A=π(2k+1)/2N and B=π(2k+1)(2N+1)/2N.
One can then multiply both sides of the expression E8 by:
2.cos A=1/{2cos[π(2k+1)/2N]}=Ck.
Note that, since Ck does not depend on the summation index n, it can be moved within the summation sign. Assume then by definition that y(-1)=0, and note that the cosine function for the input y(7) is equal to zero. The expression for h(k) can then be rewritten in the following form: ##EQU8## Note that the inputs [y(2n+1)=y(2n-1)] imply that in calculating h(k), the odd input terms are paired to form N/2 paired inputs p(n)=[y(2n+1)=y(2n-1)].
For N=8 the values of p(n) are as follows:
______________________________________                                    
n          p(n)                                                           
______________________________________                                    
0          y(-1) + Y(1) = Y(1) Y(-1) = 0 by definition                    
  1 y(1) + y(3)                                                           
  2 y(3) + y(5)                                                           
  3 y(5) + y(7)                                                           
______________________________________
Expression E9 for h(k) can then be represented by the following: ##EQU9##
Observe now that the cosine term under the summation sign is the same for both g(k) and h(k) and that both have the structure of a 1-D IDCT (compared with expression E5). The result of the IDCT for the odd k terms, that is, for h(k), however is multiplied by the factor Ck=1/{2.cos [π(2k+1)/2N.
In other words, g(k) is an n/2-point IDCT operating on even inputs y(2n) and h(k) is an n/2-point IDCT operating on [y(2n+1)=y(2n=1)] where y(-1)=0 by definition.
Now introduce the following identities:
yn=y(n);
c1=cos(n8);
c2=cos(2n8)=cos(n4)=1.√2;
c3=cos(3n8);
d1=1[2.cos(n1610)];
d3=1[2.cos(3π/116)];
d5=1[2.cos(5π/116)]; and
d7=1/[2.cos(97π/16)].
Further introduce scaled cosincoe efficients as follows:
c1s=√2.cos(π/8);
c3s=√2.cos(3π8);
Using the evenness (cos(-φ)=cos(φ)) and periodicity (cos(-φ))π(-φ)=-cos(φ) of the cosine function, expressions E7 and E8 can then be expanded for N=8 to yield (recall also (O) is 1/√2; ##EQU10##
Now, recall that according to this embodiment of the present invention, all values are scaled upward by a factor of 2 for both the DCT and IDCT operations. In other words, according to the embodiment, both h(k) and g(k) are multiplied by this scaling factor. The g(k) and h(k) expressions, erthefore, become: ##EQU11##
Notice that since c2=cos (π/4)=1/√2, multiplication by √2, gives a scaled c2 value=1. By scaling the expressions (corresponding to upward scaling of the values of the video absolute and frequency values) according to this embodiment, it is, therefore, possible to eliminate the need to multiply and c3s, both of which are constant coefficients so that general utility multipliers are not needed. This, in turn, eliminates the need for the corresponding hardware multiplier in the semiconductor implementation of the IDCT operations.
The similarity in structure of g(k) and h(k) can be illustrated by expressing these sets of equations in matrix form. Let C be the 4×4 cosine coefficient matrix defined as follows: ##EQU12##
Where D=diag[d1, d3, d5, d7]=the 4×4 matrix with d1, d3, d5, and d7 along the diagonal and with other elements equal to zero. As E14 and E15 show, the procedures for operating on even-numbered inputs to get g(k) and for operating on the odd-numbered inputs to get h(k) both have the common step of multiplication by the cosine coefficient matrix C. To get h(k), however, the inputs must first be pairwise summed (recalling that y(-1)=0 by definition), y(1) must be premultiplied by 2, and the result of the multiplication by C must be multiplied by D.
As the expressions above also indicate, the N-point, 1-D IDCT (see E4) can also be split into the two N/2-point, 1-D IDCTs each involving common core operations (under the summation sign) on the N/2 odd (grouped) and the N/2 even input values. The expressions above yield the following simple structure for the IDCT as implemented in this embodiment:
Low-order outputs for (N=8, outputs k={0,1,2,3}):
u(k)=g(k)+h(k)                                             Equation 16
High-order outputs (for N=8, outputs k={4,5,6,7}):
y(k)=y(N-1-k')=g(k')-h(k')                                 Equation 17
Note that g(k) operates directly on even input values to yield output values directly, whereas h(k') involves grouping of input values, as well as multiplication by the values d1, d3, d5 and d7.
As always, the designer of an IDCT circuit is faced with a number of trade-offs, such as size versus speed and greater number of implemented devices versus reduced interconnection complexity. For example, it is often possible to improve the speed of computation by including additional, or more complicated devices on the silicon chip, but this obviously makes the implementation bigger or more complex. Also, what is available or desired on the IDCT chip may limit or preclude the use of sophisticated, complicated, designs such as "look-ahead" adders.
Standards of Accuracy
Assuming infinite precision and accuracy of all calculations, and, thus, unlimited storage space and calculation time, the image recreated by performing the IDCT and DCT-transformed image data would reproduce the original image perfectly. Of course, such perfection is not to be had using existing technology.
In order to achieve some standardization, however, IDCT systems are at present measured according to a standardized method put forth by the Comite Consultatif International Telegraphique et Telephonique (`CCIT`) in `Annex 1 of CCITT Recommendations H.261--Inverse Transform Accuracy Specification.` This test specifies that sets of 10,000 8-by-8 Blocks containing random integers be generated. These blocks are then DCT and IDCT transformed (preceded or followed by predefined rounding, clipping and arithmetic operations) using predefined precision to produce 10,000 sets of 8-by-8 `reference` IDCT output data.
When testing an IDCT implementation, the CCITT test blocks are used as inputs. The actual IDCT transformed outputs are then compared statistically with the known `reference` IDCT output data. Maximum values are specified for the IDCT in terms of peak, mean, mean square, and mean mean error of blocks as a whole and as individual elements. Furthermore, the IDCT must produce all zeros output if the corresponding input block contains all zeros, and the IDCT must meet the same standards when the sign of all input data is changed. Implementations of the IDCT are said to have acceptable accuracy only if their maximum errors do not exceed the specified maximum values when these tests are run.
Other known standards are those of the Institute of Electrical and Electronic Engineers (`IEEE`), in `IEEE Draft Standard Specification for the Implementation of 8 by 8 Discrete Cosine Transform`, P1180/D2, Jul. 18, 1990; and Annex A of `8 by 8 Inverse Discrete Cosine Transform`, ISO committee Draft CD 11172-2. These standards are essentially identical to the CCITT standard described above.
Hardware Implementation
FIG. 9 is a simplified block diagram illustrating the data flow of the IDCT method according to one embodiment of the present invention (although the hardware structure, as is illustrated and explained below, is made more compact and efficient). In FIG. 9, the inputs to the system such as Y[0] and Y[4], and the outputs from the system, such as X[3] and X[6], are shown as being conveyed on single lines. It is to be understood that each of the single-drawn lines in FIG. 9 represents several conductors in the form of data buses to convey, preferably in parallel, the several-bit wide data words to which each input and output corresponds.
In FIG. 9, the large open circles 225 and 226 represent two-input adders, whereby a small circle 227 at the connection point of an input with the adder indicates that the complement of the corresponding input word is used. Adders with such a complementing input, thus, subtract the complemented input from the non complemented input. For example, although the output T0 from the upper left adder will be equal to Y[0]+Y[4] that its T0=Y0+Y4, the adder with the output T1 forms the value Y0+(-1),* Y4=Y0-Y4. Adders with a single complementing input can, therefore, be said to be differencing components.
Also in FIG. 9, constant-coefficient multipliers are represented by solid triangles 230 in the data path. For example, the input Y1 passes through a square root of two multiplier before entering the adder to form B0. Consequently, the intermediate value T3=Y2. T3=Y2.c1S+Y6.c3s, and the intermediate value B2=p1.c3s-p1.c1s=(Y1+Y3).c3s-(Y5+Y7).c1s. By performing the indicated additions, subtractions, and multiplications, one will see that the illustrated structure implements the expressions E11 and E12 for g(0) to g(3) and h(0) to h(3).
FIG. 9 illustrates an important advantage of the embodiment, in accordance with the present invention. As FIG. 9 shows, the structure is divided into four main regions: a pre-common block, PREC 231, that forms the paired inputs p(k) and multiplies the input y(1) by the square root of two; a first post-common block, POSTC1 233, that includes four multipliers for the constants d1, d3, d5, d7 (see expression E12); a second post-common block, POSTC2 235, that sums the g0 to g3 terms and the h0 to h3 terms for the low order outputs, and forms the difference of the g0 to g3 terms and the h0 to h3 terms for the high-order outputs (See expressions E17 and E17); and a common block, CBLK 232, is included in both the even and odd data paths. In the processing circuitry according to the embodiment of the present invention, the common operations performed on the odd and even numbered inputs are carried out by a single structure, rather than duplicated structure as illustrated in FIG. 9.
To understand the method of operation and the advantages of certain digital structures used in the embodiment, it is helpful to understand what "carry bits". As a simple example, note that the addition of two binary numbers is such that 1+1=0, with a carry of "1", which must be added into the next higher order bit to produce the correct result "`10`" (the binary representation of the decimal number "2"). In other words, 01+01=00 (the "sum" without carry)+10 (the carry word); adding the "sum" to the "carry word" one gets the correct answer 00+10=10.
As a decimal example, assume that one needs to add the numbers `436` and `825`. The common procedure for adding two numbers by hand typically proceeds as follows:
1. Units `6` plus `5` is `1` with a carry of `1` into the `tens` position--Sum: 1, Carry-in: 0, Carry-Out: 0.
2. Tens: `3` plus `2` is `6`, plus the `1` carried from the preceding step, gives `6` with no carry--Sum: 5, Carry-In: 0, Carry-Out:0.
3. Hundreds: `4` plus `8` is `2` with a carry of 1 into the thousands, but with no carry to be added in from the previous step; Sum: 2, Carry-in:), Carry-Out:1
4. Thousands: `0` plus `0`, plus the `1` carried from the hundreds gives, `1` Sum: 0, Carry-In: 1, Carry-Out:0.
The answer, `1261`, is, thus, formed by adding the carry-in sum for each position to the sum for the same position, with the carry-in to each position being the carry-out of the adjacent lower-order position. (Note that this implies that the carry-in to the lowest order position is always a `0`). The problem, of course, is that one must wait to add the `4` and `8` in the hundreds place until one knows whether there will be a carry-in from the tens place. This illustrates a "ripple adder", which operates essentially in this way. A ripple adder, thus, achieves a `final` answer without needing extra storage elements, but it is slower than some other designs.
One such alternative design is known as `carry-save`, in which the sum of two numbers for each position is formed by storing a partial sum or result word (in this example, 0251) and the carry values in a different word (here, 1010). The full answer is then obtained by `resolving` the sum and carry words in a following addition step, thus, 0251+1010=1261. Note that one can perform the addition for every position at the same time, without having to wait to determine whether a carry word can be added to the partial result at any time as long as it is saved.
Since the resolving operations typically require the largest proportion of the time required in each calculation stage, speeding up these operations has a significant effect on the overall operating speed while requiring only a relatively small increase in the size of the transform. Carry-save multipliers, therefore, are usually faster than those that use ripple adders in each row. However, this gain in time comes at the cost of greater complexity, since the carry word for each addition in the multiplier must be either stored or passed down to the next addition. Furthermore, in order to obtain the final product of a multiplication, the final partial sum and final carry word will have to be resolved, normally by addition in a ripple adder. Note, however, that only one ripple adder will be needed, so that the time savings are normally proportional to the size of the multiplication that must be performed. Furthermore, note that a carry word may be treated as any other number to be added in and as long as it is added in at some time before the final multiplication answer is needed, the actual addition can be delayed.
In this embodiment of the present invention, this possibility of delaying resolution is used to simplify the design and to increase the throughout of the IDCT circuitry. Also, certain bits of preselected carry words are, optionally and deliberately forced to predetermined values before resolution in order to provide greater expected accuracy of the IDCT result based on a statistical analysis of test runs of the invention on standard test data sets.
FIG. 10 is a block diagram that illustrates a preferred structure, in accordance with the present invention. In this preferred embodiment of the present invention, the even and odd numbered inputs are time-multiplexed and are processed separately in the common block CBLK 232. The inputs may be processed in either order.
In FIG. 10, the notation Y[1,0], Y[5,4], Y[3,2] and Y[7,6] is used to indicate that the odd numbered inputs Y1, Y3, Y5, Y7 preferably pass through the calculation circuitry first, followed by the even numbered inputs Y0, Y2, Y4, Y6. This order is not essential to the present embodiment; nonetheless, as is explained below, certain downstream arithmetic operations are performed only on the odd numbered inputs, and by entering the odd numbered input values first, these downstream operations can be processing at the same time that arithmetic operations common to all inputs are performed upstream on the even numbered inputs. This reduces the time that several arithmetic devices would otherwise remain idle.
Similarly, the notation X[0,7], X[1,6], X[3,4], X[2,5] is used to indicate that the low order outputs X0, X1, X2, X3 are output first, followed by the high order outputs X4, X5, X6, X7.
As FIGS. 9 and 10 illustrate, the inputs are preferably initially not grouped in ascending order, although this is not necessary since to odd numbered inputs are Y1, Y5, Y3, and Y7. Arranging the input signals in this order makes possible the simple `butterfly` data path structure shown in FIGS. 9 and 10 and greatly increases the interconnection efficiency of the implementation of the present invention in silicon semiconductor devices.
As shown in FIG. 10, adders and subtractors are indicated by circles either a `+` (adder) 235, `-` (subtractor ) 236 which is an adder with one complementing input or `±` (resolving adder/subtractor, which is able to switch between addition and subtraction 237). The left most adders and subtractors in the common block 232 of the two m-bit input words is the m-bit partial resulting parallel with the m-bit or (m-1) bit word containing the carry bits of the addition/subtraction. In other words, the first additions and subtractions in the common block CBLK 232 are preferably unresolved, meaning that the addition of the carry bits is delayed until a subsequent processing stage. The advantage of this step is that such carry-save adder/subtractors since they do not need to perform the final addition of the carry-bit word to the result. Resolving adders may, however, also be used in order to reduce the bus width at the outputs of the adders.
FIG. 10 also illustrates the use of one and two input latches in the preferred embodiment of the present invention. In FIG. 10, latches are illustrated as rectangles 238 and are used in both the pre-common block PREC 231 and the post-common block POSTC 233. Single-input latches are used at the inputs of the multipliers D1, D3, D5 and D7, as well as to latch the inputs to the resolving adders/subtractors which are the computed g(k) and h(k) values corresponding to the respective outputs from latches g[0,7], g[1,6], g[3,4] and g[2,5] and h[0,7], h[1,6], h[3,4] and h[2,5]. As such, the resolving adders/subtractors perform the addition or subtraction indicated in expressions E16 and E17 above.
As described previously, the even-numbered inputs Y0, Y2, Y4 and Y6 do not need to be paired before being processed in the common block CBLK 232. However, not only do the odd-numbered inputs require such pairing, but the input Y12 must also be multiplied by the square root of two in order to ensure that proper input values are presented to the common block CBLK 232. The pre-common block PREC 231, therefore, includes a 2-input multiplexing (`mux`) latch C10, C54, C32 and C76 for each input value. One input to the 2-input mux latch is consequently tied directly to the unprocessed input values, whereas the other input is received from the resolving adders and, for the input Y1, the resolving square root of two multiplier. The correct paired or unpaired inputs can, therefore, be easily presented to the common block CBLK 232 easily by simple switching of the multiplexing latches between their two inputs.
As FIG. 10 illustrates, the square root of two multipliers D1, D3, D5, D7 preferably resolve their outputs, that is, they generate results in which the carry bits have been added in to generate a complete sum. This ensures that the outputs from the multipliers have the same bus width as the un-multiplied inputs in the corresponding parallel data paths.
The preferred embodiment of the common block 232, in accordance with the present invention, also includes one `dummy` subtractor 240 in the forward data path for Y[1,0] and Y[5,4], respectively. These devices act to combine the two inputs (in the case of the dummy subtractor, after 2's-complementing the one input) in such a way that they are passed as parallel outputs. In each case, the one input is manipulated as if it contained carry bits, which are added on in the subsequent processing stage. The corresponding addition and subtraction is, thus, performed, although it is delayed.
This technique reduces the resources required in the upper two data paths since a full-scale adder/subtractor need not be implemented for these devices. Therefore, the `combiners` act as adders and subtractors and can be implemented for these devices and can be implemented either as simple conductors to the next device (for addition), or as a row of inverters (for subtraction), either of which requires little or no additional circuitry.
The use of such combiners also means that the outputs from the initial adders and subtractors in the common block CBLK 232 will all have the same width and will be compatible with the outputs of the carry-save adder/subtractor found in the bottom two data paths, with which they form inputs to the subsequent resolving adders and subtractors in the common block CBLK.
As described previously, the even-numbered inputs are processed separately from the odd-numbered inputs in this preferred embodiment of the present invention. Assume, further, that the odd-numbered inputs are to be processed first. Supervisory control circuitry (not shown in FIG. 10) applies the odd-numbered input words to the pre-common block PREC, and selects the lower inputs (viewed as in FIG. 10) of the multiplexing latches C10, C54, C32, C76 which then stores the paired values p0 to p3 (see FIG. 9 and the definition of p(n) above). The latches 1h0, h1, 1h3 and 1h2 are then activated to latch the values H0, H1, H3 and H2, respectively.
The supervisory control circuitry latches and then selects the upper inputs of the two-input multiplexing latches C10, C54, C32 and C76 in the precommon block PREC 231 and applies the even numbered input words to these latches. Since the even-numbered inputs are used to form the values of g0 to g3, the supervisory control circuitry also opens the latches Lg0 to Lg3 in the post-common block POSTC 233, to store the g(k) values.
Once the g(k) and h(k) values are latched, the post-common block POSTC 233 outputs the high-order signals X7, X6, X5 and X4 by switching the resolving adder subtractors to the subtraction mode. The low order output signals X3, X2, X1 and X0 are then generated by switching the resolving adders/subtractors to the addition mode. Note that the output data can be presented in an arbitrary order, including natural order.
The preferred multiplexed implementation, in accordance with the present invention, is illustrated in greatly simplified, schematic form in FIG. 10, performs the same calculations as the non-multiplexed structure illustrated in FIG. 9. The number of adders, subtractors and multipliers in the common block CBLK 232 is, however, cut in half and the use of dummy adder/subtractors 240 further reduces the complexity of the costly arithmetic circuitry.
FIG. 11 illustrates the main components and data lines of an actual implementation of the IDCT circuit according to the embodiment of the present invention. The main components include the precommon block circuit PREC 231, the common block circuit CBLK 232, and the post-common block POSTC 233. The system also includes a controller CNTL 241 that either directly or indirectly applies input, timing and control signals to the precommon block PREC 231 and post-common block POSTC 233.
In the preferred embodiment of the present invention, the input and output signals (Y0 to Y7 and X0 to X7, respectively) are 22 bits wide. Tests have indicated that this is the minimum width that is possible which still yields acceptable accuracy as measured by existing industry standards. As is explained in greater detail below, this minimum width in achieved in part by deliberately forcing certain carry words in selected arithmetic devices to be either a `1` or a `0`. This bit manipulation, corresponding to an adjustment of certain data words, is carried out as the result of a statistical analysis of the results of the IDCT system, in accordance with the present invention, to the after using the IDCT transformation of known input test data. By forcing certain bits to predetermined values, it was discovered that the effects of rounding and truncation errors could be reduced, so that the spatial output data from the IDCT system could be made to deviate less from the known `correct` spatial data. The present invention is equally applicable, however, to other data word lengths since the components used in the circuit according to the present embodiment can all be adapted to different bus widths using known methods.
Although all four inputs that are processed together could be input simultaneously to the pre-common block PREC along 88 parallel conductors (4×22), pixel words are typically converted one at a time from the transmission data. According to the present embodiment, input data words are, therefore, preferably all conveyed serially over a single 22 bit input bus and each input word is sequentially latched at the proper input point in the data path. As shown in FIG. 11, the 22 bit input data bus is labelled T-- IN[21:0] 242.
In the Figures and in the discussion below, the widths of multiple-bit signals are indicated in brackets with the high-order bit to the left of a colon `:` and the least significant bit (LSB) to the right of the colon. For example, the input signal T IN[21:0] 242 is 22 bits wide, with the bits being numbered from 0 to 21. A single bit is identified as a single number within square brackets, thus, T-- IN[1] indicates the next to least significant bit of the signal T-- IN.
The following control signals are used to control the operation of the pre-common block PREC 231 in the preferred embodiment of the present invention.
IN-- CLK-- OUT CLK The system, in accordance with the present invention, preferably uses a non-overlapping two phase clock. The signals IN-- CLK and OUT-- CLK are accordingly columns of latches that hold the values of input, intermediate, and output signals.
LATCH10, LATCH54, LATCH32, LATCH76: Preferably, one 22-bit word is input to the system at a time. On the other hand, four input signals are processed at a time. Each input signal must, therefore, be latched at its appropriate place in the architecture before being processed with three other input words. These latch signals are used to enable the respective input latches. The signal LATCH54, for example, is first used to latch input signal Y5 and later to latch input signal Y4, which enters the pre-common block PREC 231 at the same point as the input signal Y5 (see FIG. 10) but during a subsequent processing stage.
LATCH: Once the four even or odd-numbered input signals are latched into the pre-common block PREC 231, they are preferably shifted at the same time to a subsequent column of latches. The signal LATCH is used to enable a second column of input latches that hold the four input values to be operated on by the arithmetic devices in the pre-common block PREC 231.
SEL-- BYP, SEL-- P: As FIG. 10 illustrates, the even-numbered input signals that are latched into the latches C10, C54, C32 and C76 should be those that bypass the adders and the square root of two resolving multiplier. The odd-numbered input signals, however, must first be paired to form the paired inputs p(n), and the signal Y1 must be multiplied by the square root of two. The control signal SEL-- P is activated in order to select the paired input signals. Hence, these signals are used to control gates that act as multiplexers to let the correct signals pass to the output latches of the precommon block PREC 231.
As discussed previously, not having to arrange the inputs in strictly ascending order leads to a simplified `butterfly` bus structure with high interconnection efficiency. As also described, the odd inputs are preferably applied as a group to the pre-common block first, followed by the even-numbered inputs, but any order may be used within each odd or even group, i.e., any order of inputs may be used, however, suitable latch arrangements as separately provided to process the odd-numbered inputs, or at least are provided in separate regions of the circuit.
The supervisory control circuitry also generates timing and control signals for the post-common block POSTC 233. These control signals are as follows:
EN-- BH, EN-- GH: Referring again to FIG. 9, the outputs from the common block CBLK 232, after processing of the odd-numbered inputs, are shown as H0, H1, H3, and H2. These signals are then sent to the coefficient multipliers, d1, d3, d7, d5, respectively, in the first post common block POSTC1 233. The signal EN-- BH is used to enable latches that hold the g0 to g3 values, as well as to enable the latches that hold the h0 to h3 values after they have been multiplied in the coefficient multipliers.
ADD, SUB: As FIG. 10 illustrates, the embodiment includes a bank of resolving adders/subtractors that sum and difference(k) and h(k) values in order to form the low-order outputs, respectively. The signals ADD, SUB are used to set the resolving adders/subtractors in the addition and subtraction modes, respectively. EN-- 0: This signal is used to enable output latches that latch the results from the resolving adders/subtractors.
MUX-- OUT70, MUX-- OUT61, MUX-- OUT43, MUX-- OUT52: In accordance with the present invention, the output data from the system is preferably transmitted over a single 22-bit output bus, so that only one output value (X0 to X7) is transferred at a time. These signals are activated sequentially to select which of the four latched output values is to be latched into a final output latch. Accordingly, these signals thus act as the control signals for a 4-to-1 multiplexer.
T-- OUT[21:0]: This label indicates the 22-bit output signal from the post-common block POSTC 233.
The output signals from the pre-common block PREC 231 are latched to form the input signals to the common block CBLK 232. As shown in FIG. 11, the output signals from the pre-common block PREC 231 are presented as the four 22-bit data words Cl10[21:0], Cl54[21:0], Cl32[21:0], Cl76[21:0], which become the input signals IN[0], IN[1], IN[3], IN[2], respectively, to the common block CBLK 232.
As FIG. 11 shows, the four 22-bit results from the common block CBLK 232 are transferred in parallel as output signals OUT0[21:0], OUT1[21:0], OUT3[21:0], OUT2[21:0], which are then latched as the input signals of the post common block POSTC 233 as CO70[20:1], CO61[21:0], CO43[21:0], CO52[21:0].
One should take particular note that no control signals are required for the common block CBLK Because of the unique structure of the IDCT system in this example, the common block of the system's operations can be performed as pure logic operations, with no need for clock, timing or control signals. This further reduces the complexity of the device. One should also note that in certain applications (particularly those in which there is plenty of time to perform all needed arithmetic operations) the pre-common and post-common blocks PREC 231, POSTC 233 may also be arranged to operate without clock timing or control signals.
FIG. 12 is a block diagram of the pre-common block PREC 231 of the present invention. In this and following Figures, the notation `S1[a], S2[b], . . . , SM[Z]`, where S is an arbitrary signal label and a, b, . . . , z are integers within the range of the signal's bus width, indicates that the selected bits a, b, . . . , z from the signals S1, S2, . . . , SM are transferred in parallel over the same bus, with the most significant bits (MSBs) being the selected bits `a` of the signal S1, and the least significant bits (LSBs) being the selected `z` of signal SM. The selected bits do not have to be individual bits, but rather, entire or partial multi-bit words may also be transmitted along with other single bits or complete or partial multi-bit words. In the Figures, the symbol S will be replaced by the corresponding signal label.
For example, in FIG. 12, a square root of two multiplier is shown as R2MUL. The `save`; or `unresolved sum` output from this non-resolving multiplier is indicated as the 21-bit word M5S[20:0], similarly, the `carry` output from the multiplier R2MUL is shown as the 22-bit word M5C[20:0], which is transferred over the bus to the `b` input of a carry-save resolving adder M5A. (Recall that a `0` is inserted as an MSB to the least significant 21 bits of the save output, however, this is accomplished before being applied to the `a` input of the resolving adder M5A. This is indicated in FIG. 12 by the notation GND.M5S[20:0]). In other words the conductor corresponding to the MSB input to the adder M5A is forced to be a `0` by tying it to ground GND.
In order to understand why a `0` is inserted as the 22'nd bit of the `sum`, observe that if the partial sum of a multiplication is n places wide, the carry word is shifted one place to the left relative to the partial sum. The carry word, therefore, extends to n+1 places with a valid data bit in the n+1'th position with an `0` in the least significant position (since there is nothing before this position to produce a carry bit into the units position). If these two words are used as inputs to a resolving binary adder, care must be taken to ensure that the bits (digits) of the carry word are properly aligned with the corresponding bits of the partial sum. This also ensures that the decimal point (even if only implied, as in integer arithmetic) is kept `aligned` in both words. Assuming the inputs to the adder are n+1 bits wide, a `0` can then be inserted into the highest-order bit of all n-bit positive partial sum words to provide an n+1 bit input that is aligned with the carry word at the other input.
As is described above previously, the four inputs that are processed at a given time in the pre-common block PREC 231 are transferred over the input bus T-- IN(21:0). This input bus is connected to the inputs of four input latches IN10L, IN54L, IN32L, AND IN76L. Each respective latch is enabled only when the input clock signal IN-- CLK and the corresponding latch selection signal LATCH10, LATCH54, LATCH32, LATCH76 are high. The four inputs can, therefore, be latched into their respective input latches in four periods of the IN-- CLK signal by sequential activation of the latch enabling signals LATCH10, LATCH54, LATCH32, and LATCH76. During this time, the LATCH signal should be low (or on a different phase) to enable the input latches IN10L, IN54L, IN32L, and IN76L to stabilize and latch the four input values.
An example of the timing of the latches, in accordance with the present invention, is illustrated in FIG. 13. Once the four input signals are latched in the preferred order, they are passed to a second bank of latches L10L, L54L, L32L, L76L. These second bank of latches are enabled when the signals OUT-- CLK and LATCH are high. This signal timing is also illustrated in FIG. 13.
Note that the system of the present invention does not have to delay receipt of all eight input words. Once all the even or odd input words are received and latched in IN10L, IN54L, IN32L and L76L, this frees the In latches, which can then begin to receive the other four input signals without delay at the next rising edge of IN-- CLK.
The 2-digit suffix notation [10, 54, 32, 76] used for the various components illustrated in the Figures indicates that odd-numbered signals are processed first, followed by the even-numbered signals on a subsequent pass through the structure. As is mentioned above, this order is not required by the present invention, and it will be appreciated by one of ordinary skill in the art that additional orders may be used.
Once the four input signals are latched in proper order in the second set of latches L10L, L54L, L32L, L76L, the corresponding values are either passed as inputs to output latches C10L, C54L, C32L and C76L on activation of the selected bypass signal SEL-- BYP, or they are passed as paired and multiplied inputs to the same output latches upon activation of the `select p` signal SEL-- P. In other words, all signals are passed, both directly and indirectly, via arithmetic devices, to the output latches C10L, C54L, C32L, C76L of the pre-common block PREC 231. The proper values, however, are loaded into these latches by activation of the `select bypass` signal SEL BYP (for even-numbered inputs Y0, Y2, Y4, and Y6) or the "select p" signal SEL-P (for the odd-numbered inputs Y1, Y3, Y5 and Y7). As will be appreciated by one of ordinary skill in the art, the desired timing and order of these and other control signals is easily accomplished in a known manner by proper configuration and/or [micro-] programming of the controller CNTL 241.
The uppermost input value at the output of latch L10L is passed first to the square root of two-multiplier R2MUL and then to the resolving adder M5A as indicated. The output from the resolving adder M5A is shown as an equivalent of the resolved multiplication of the output from the latch L10L by the square root of two. The outputs from the other three latches L54L, L32L, L76L are also transferred to corresponding output latches C54L, C32L and C76L, respectively, both directly via 22-bit latch buses LCH54[21:0], LCH32[21:0] LCH76[21:0] and indirectly to the output latches via resolving adders P2A, P1A and P3A, respectively.
In the present invention, each resolving adder P2A, P1A, P1A has two inputs "a" and "b". For adder, P2A, the one input is received from the latch L32L, and the other input is received from the latch L54L. For input values Y5 (latched in L54L) and Y3 (latched in L32L), the output from the adder P2A will, therefore, be equal to Y5+Y3, which, as is shown above, is equal to p(2). Hence, the adders "pair" the odd-numbered inputs to form the paired input values p(1), p(2) and p(3). Of course, the even-numbered input signals latched in L54L, L32L, and L76L will also pass through the resolving adders P2A, P1A and P3A, respectively, however, the resulting p "values" will not be passed to the output latches C54L, C32L and C76L because the "select p" signal SEL P will not be activated for even-numbered inputs.
The values that are latched in the output latches C10L, C54L, C32L and C76L upon activation of the input clock signal IN-- CLK will therefore be equal to either the even-numbered inputs Y0, Y2, Y4, Y6 or the paired input values P0, P1, P2, P3 for the odd-numbered inputs. One should recall that the input Y(1) is "paired" with the value U(-1), which is assumed to be zero. As illustrated in FIG. 12, this assumption is implemented by not adding anything to the value Y1. Instead, Y1 is only multiplied by the square root of two as is shown in FIGS. 9 and 10.
FIG. 14 illustrates the preferred architecture of the common block CBLK 232, in accordance with the present invention. Because of the various multiplications and additions in the different system blocks, it is necessary or advantageous to scale down the input values to the common block before performing the various calculations. This ensures a uniform position for the decimal point (which is implied for integer arithmetic) for corresponding inputs to the various arithmetic devices in the system.
Accordingly, the input values IN0[21:0] AND IN1[21:0] are accordingly scaled down by a factor of four, which corresponds in digital arithmetic to a right shift of two bits. In order to preserve the sign of the number (keep positive values positive and negative values negative) in binary representation, the most significant bit (MSB) must then be replicated in the two most significant bits of the resulting right-shifted word; this process is known as "sign extension". Hence, the input value IN0 is downshifted by two bits with sign extension to form the shifted input value indicated as IN[21], IN0[21], IN0[21:2]. The input value IN1[21:0] is similarly sign-extended two places. The input IN2 is also shifted and extended to form IN2[21], IN2[21:1]. These one-position shifts correspond to truncated division by a factor of two.
As shown in FIG. 10, the input IN2, IN3 are those which must be multiplied by the scaled coefficients c1s and c3s. Each input IN3 and IN2 must be multiplied by each of the scaled coefficients. As FIG. 14 illustrates, this is implemented by the four constant-coefficient carry-save multipliers MULC1S, MULNC1S, MULC3S3, and MULC2S2. One should note that the bottom multiplier for IN2 is an inverting multiplier MULC1S, that is, its output corresponds to the negative of the value of the input multiplied by the constant C1S. Therefore, the value latched in C76 is subtracted from the value latched in C32 (after multiplication by C3S). By providing the inverting multiplier MULNC1S, subtraction is implemented by adding the negative of the corresponding value, which is equivalent to forming a difference. This allows the use of identical circuitry for the subsequent adders, while allowing a non-inverting multiplier may be used with a following subtractor.
In the illustrated embodiment of the present invention, four cosine coefficient multipliers MULC1S, MULNC1S, MULC2S3, and MULC3S2 are included. If arrangements are made for signals to pass separately through the multipliers, however, the necessary multiplications can be implemented using only two multipliers one for the c1s coefficient and one for the c3s coefficient.
In accordance with the present invention, the multipliers for MULC1S, MULNC1S, MUL3S3 and MULC3S2 are preferably of the carry-save type, which means that they produce two output words, one corresponding to the result of the various rows of additions performed within a hardware multiplier, and another corresponding to the carry bits generated. The outputs from the multipliers are then connected as inputs to either of two 4-input resolving adders BT2, BT3.
For ease of illustration only, five of the output buses from the multipliers are not drawn connected to the corresponding input buses of the adders, as will be appreciated by one of ordinary skill in the art, these connections are to be understood, and are illustrated by each respective output and input having the same label. Hence, the save output M1S[20:0] of the multiplier MULC1S is connected to the lower 21 bits of the "save-a" of the adder BT3.
As shown in FIG. 14, five of the inputs to the adders BT2 and BT3 are shown as being "split". For example, the "ca" input of the adder BT2 is shown as having IN3[21] over M3C[20:0] being input as the least significant 21 bits. Similarly, the "sa" (the "save-a" input) of the same adder is shown as being GND, GND over M3S[19:0]. This means that two zeros are appended as the two most significant bits of this input word. Such appended bits ensure that the proper 22-bit wide input words are formed with the proper sign.
The carry-save adders BT2 and BT3 add the carry and save words of two different 22-bit inputs to form a 22-bit output save word T3S[21:0] and a 21-bit output carry word T3C[21:1]. Accordingly, the input to each adder is thus 88 bits wide and the output from each adder is 43 bits wide. As FIG. 10 illustrates, the output from the latch C10 is combined with the output from the latch C54 in the upper-most data path before addition with the output from the carry-save adder BT3. The "combination" is not, however, necessary until reaching the following adder in the upper data path. Consequently, as FIG. 14 shows, the shifted and sign-extended input value IN0 is connected to the upper carry input.
The upper carry input of adder CS0 is connected to the shifted and sign-extended input value IN0, and the shifted and sign-extended input IN1 is connected as the upper save input of the same adder. In other words, IN0 and IN1 are added later in the adder CS0.
The designation "dummy" adder/subtractor 240 used in FIG. 10, therefore, indicates which operation must be performed, although it does not necessarily have to be performed at the point indicated in FIG. 10. Similarly, the lower dummy subtractor 240 shown in FIG. 10 requires that the output from latch C54 be subtracted from the output from latch C10. This is the same as adding the output from C10 to the complement of the output of C54.
Referring once again to FIG. 14, the complement of the input IN1 (corresponding to the output of latch C54 in FIG. 10) is performed by a 22-bit input inverter IN1[21:0] (which generates the logical inverse of each bit of its input, bit-for-bit). The complement of IN1 value--NlN1[21:0]--is passed to the upper "save" input of the adder CS1, with the corresponding upper "carry" input being the shifted and sign-extended IN0. The upper portion of the adder CS1, therefore, performs the subtraction corresponding to IN0 minus IN1.
In the lower two data paths shown in FIG. 11, resolving subtractors are used instead of the resolving adders shown in the upper two data paths at the output of the common block CBLK 232. Each resolving adder or subtractor is equivalent to a carry-save adder or subtractor followed by a resolving adder. This is shown in FIG. 14. Subtractors CS2 and CS3 have as their inputs the processed values of IN0 to IN3 according to the connection structure shown in FIG. 10.
The 22-bit carry and save outputs from each of the adders/subtractors C20-CS3 are resolved in the resolving adders RES0-RES3. As will be appreciated by one of ordinary skill in the art, resolution of carry and save outputs is well understood in the art of digital design and is, therefore, not described in greater detail here. As FIG. 14 illustrates, the save outputs the carry-save adders/subtractors CS0-CS3 are passed directly as 22-bit inputs to the "a"-input of the corresponding resolving adders RES0-RES3.
As is also well known in the art, the 2's-complement of a binary number is formed by inverting each of its bits (changing all "1's" to "0's" and vice versa) and then adding "1". Note that the "1" can be added immediately after the bit inversion, or later. The LSB of a carry word will always be a "0" which is implemented in the illustrated embodiment of the present invention by tying the LSB of the carry words O0C and O1C to ground GND as they are input to the resolving adders RES0 and RES1, respectively. The addition of "1" to the carry outputs of the subtractors CS2 and CS3 to form 2'S-complemented values, however, is implemented by tying the LSB of these data words O2C and O3C to supply voltage VDD, thus "replacing" the "0" LSB of the carry word by a "1", which is equivalent to addition by "1".
For the reasons provided above, a "0" is appended as the LSB to the 21-bit carry words from the carry-save adders Cs0 and CS1 (by tying the LSB to ground GND) and the LSB of the carry words from the carry-save subtractors CS2 and CS3 is set equal to "one" by tying the corresponding data line to the supply voltage VDD. The resolving adders RES0-RES3, therefore, resolve the outputs from the adder/subtractors CS0-CS3 to form the 22-bit output signals OUT0[21:0]-OUT3[21:0].
Two advantages of the IDCT circuitry according to the embodiment of the present invention can be seen in FIG. 14. First, no control or timing signals are required for the common block CBLK 232. Rather, the input signals to the common block are already processed in such a way they can be applied immediately to the pure-logic arithmetic devise in the common block 232. Second, by proper scaling of the data words, integer arithmetic can be used throughout (or, at least, decimal point for all values will be fixed). This avoids the complexity and slowness of floating-point devices, with no unacceptable sacrifice of precision.
Yet another advantage of the embodiment of the present invention is that, by ordering the inputs as shown, and by using the balanced decimated method in accordance with the present invention, similar design structures can be used at several points in the silicon implementation. For example, as shown in FIG. 14, the constant coefficient multipliers MULC1S, MULC3S3, MULC3S2 and MULNC1S all have similar structures and receive data at the same point in the data path, so that all four multipliers can be working at the same time. This eliminates "bottlenecks" and the semiconductor implementation is, therefore, able to take full advantage of the duplicative, parallel structure. The carry-save adders BT2 and BT3 similarly will be able to work simultaneously, as will the following carry-save adders and subtractors. This symmetry of design and efficient simultaneous utilization of several devices is common throughout the structure according to the embodiment of the present invention.
FIG. 15 shows the preferred arrangement of the post-common block POSTC 233 in accordance with the present invention. As FIG. 10 shows, the primary functions of the post-common POSTC 233 are to form the h0 to h3 values by multiplying the outputs of the common block by the coefficients d1, d3, d5 and d7; to add the g(k) and h(k) values to form the low order outputs; and to subtract the h(k) values from the corresponding g(k) values to form the high-order outputs. Referring now to both FIG. 10 and FIG. 15, the post-common block POSTC 233 latches the corresponding outputs from the common block CBLK 232 into latches BH0L, BH1L, BH3L and BH2L when the Bh latches are enabled, the control circuitry sets the EN-- BH signal high, and the output clock signal OUTC-- CLK signal goes high. The g(k), g0 to g3 values are latched into corresponding latches G0L, G1L, G3L and G2L when the control circuitry enables these latches via the signal EN-- GH and input clock signal IN-- CLK goes high.
The processed odd-numbered inputs, that is, the values h0 to h3, are latched into latches H0L, H1L, H3L and H2L when the EN-- GH and IN-- CLK signals are high, via the constant coefficient multipliers D1MUL, D3MUL, D5MUL and D7MUL. These multipliers multiply, respectively by d1, d3, d5 and d7. In the preferred embodiment, these constant-coefficient multipliers are preferably carry-save multipliers in order to simplify the design and to increase calculation speed. As FIG. 15 illustrates, the "carry" ("c") outputs from the constant coefficient multipliers are connected, with certain changes described below, to the a inputs of resolving adders H0A, H1A, H3A and H2A. The "save" ("s") outputs from the coefficient multipliers are similarly, with certain forced changes described below, connected to other input of the corresponding resolving adder.
As FIG. 15 further illustrates, the LSB of the H0 signal is preferably forced to be a "1" by tying the corresponding "save" output for H0 is set to 0 (tied to ground GND), and the second bit (corresponding to H0S[1]) is set to "1". The data words from the carry and save outputs of the constant-coefficient multiplier D3MUL are similarly manipulated an input to the resolving adder H1A. The advantage of these manipulations and their input to the resolving adder H1A.
In accordance with the present invention, all 22-bits of the carry output from the coefficient multipliers D7MUL and D5MUL are connected directly to the "a" input of corresponding resolving adders H3A and H2A. The MSB of each multipliers "save" output, however, is forced to "0" by tying the corresponding data line to ground GND.
The IDCT system described was tested against the CCITT specification described above. Because of the scaling and other well-known properties of digital adders and multipliers, some precision is typically lost in the 10,000 sample, but run that forcing the various bits described above to either "0" or "1" reduced the expected error of the digital transformation. As a result of the bit manipulation of the data words, the embodiment of the present invention achieved acceptable accuracy under the CCITT standard using only 22-bit wide data words, whereas 24 bits would normally be required to produce equivalent accuracy.
Because of limited precision, and truncation and rounding errors, there is typically some inaccuracy in every data word in an IDCT system. However, forcing selected bits of a data word it was discovered that the error thereby systematically introduced into a particular data word at a particular point in the hardware yielded statistically better overall results. Bit-forcing may also be applied "within" a multiplication, for example, by selectively forcing one or more carry bits to predetermined values.
In the present invention, the bit-forcing scheme need not be static, with certain bits always forced to take specified values, but rather a dynamic scheme may also be used. For example, selected bits of a data word may be forced to "1" or "0" depending on whether the word (or even some other data) is even or odd, positive or negative, or above or below a predetermined threshold, and the like.
Normally, only small systematic changes will be needed to improve overall statistical performance. Consequently, according to this embodiment of the present invention, the LSB's of selected data words (preferably one bit and one data word at a time, although this is not necessary) are forced to be a "1" or a "0". The CCITT test is run, and the CCITT statistics for the run are compiled. The bit is then forced to the other of "1" or "0", and the test is rerun. Then the LSB (or LSBs) of other data words are forced to "1" or "0", and similar statistics are compiled. By examining the statistics for various combinations of forced bits in various forced words, a best statistical performance can be determined.
If this statistically based improvement is not required, however, the outputs from the constant-coefficient multipliers D1MUL, D3MUL, D5MUL and D7MUL may be resolved in the conventional manner in the resolving adders H0A-H3A. The lower 21-bits of the input of the corresponding latches H0L-H3L, with the LSB of these inputs tied to ground.
The outputs from the H-latches (H0L-H3L) and the G-latches (G0L-G3L) pairwise form the respective a and b inputs to resolving adder-subtractors S70A, S61A, S43A and S52A. As was indicated above, these devise add their inputs when the ADD signal is high, and subtract the "b" input from the "a" input when the subtraction enable signal SUB is high. The second bits of the upper two latch pairs H0L, G0L, H1L and G1L are manipulated by multiplexing arrangements in a manner described below.
The outputs from the resolving adder-subtractors S70A, S61A, S43A and S52A are latched into result latched R70L, R61L, R43L, R52L.
As depicted in FIG. 15b, the input words to the adder/subtractor S70A and dS61A, in accordance with the present invention, have the second bits of each input word manipulated. For example, the second bit of the input word to the "a"-input of the adder subtractor S70A is G0[1M], G0[1M], G0[0]. In other words, the second bit is set to have the value G01M. The second bits of the other inputs to the adder/subtractors S70A and S61A are similarly manipulated. This bit manipulation is accomplished by four 2:1-bit multiplexers H01MUX, G01MUX, H11MUX and G11MUX (shown to the right in FIG. 15b). In the present invention, these multiplexers are controlled by the ADD and SUB signals such that the second bit (H01M, G01M, H11M, and G11M) is set to one if the respective adder subtractor S70A, S61A is set to (ADD is high), and the second bit is set to its actual latch output value if the SUB signal is set too high. Setting of individual bits in this manner is an easily implemented high-speed operation. The preferred embodiment, therefore, includes this bit-forcing arrangement since, as is described above, statistical analysis of a large number of tests pixel words has indicated that more accurate results are thereby obtained. It is not necessary, however, to manipulate the second bits in this manner, although it gives the advantage of smaller word width.
The four high or low-order results are latched in the output latches R70L, R61L, R43L and R52L. The results are sequentially latched into the final output latched OUTF under the control of the multiplexing signals MUX-- OUT70, MUX-- OUT61, MUX-- OUT43, MUX-- OUT52. Hence, the order in which resulting signals are output can therefore be controlled simply by changing the sequence with which they are latched into the latch.
The relationship between the clock and control signals in the post-common block POSTC 233 is shown in FIGS. 13b and 13c.
As was discussed previously, two 1-dimensional IDCT operations may be performed in series, with an intervening transposition of data, in order to perform a 2-D IDCT. The output signals from the post-common block POSTC 233, are therefore, according to this embodiment of the present invention, first sorted in a known manner column-wise (or row-wise) in a conventional storage unit, such as a RAM memory circuit (not shown), and are then read from the storage unit row-wise (column-wise) so as to be passed as inputs to a subsequent pre-common block and for processing as described above in this block, and in a common block CBLK 232, and a post-common block POSTC 233.
Storing by row (column) and reading out by column (row) performs the required operation of transposing the data before the second 1-D IDCT. The output from the second POSTC 233 will be the desire, 2-D IDCT results and can be scaled in a conventional manner by shifting to offset the scaling shifts carried out in the various processing blocks. In particular, a right shift by one position will perform the division by 2 necessary to offset the two square root of two multiplications performed in the 1-D IDCT operations.
Depending on the applications, this second IDCT structure (which is preferably identical to that shown FIG. 11) is preferably a separate semiconductor implementation. This avoids the decrease in speed that would arise if the same circuits were used for both transforms, although separate 1-D transform implementations are not necessary if the pixel-clock rate is now sufficient such that a single implementation of the circuit will be able to handle two passes in real time.
As shown in FIGS. 16 through 38, a second preferred embodiment, in accordance with the present invention, uses a single one-dimensional transform. This embodiment does not require a lowering of the pixel-clock rate as discussed previously.
The existing "resolving-adders" in the first preferred embodiment have been changed to "fast-resolving-adders". As seen in FIG. 38, these have been titled, "Fast Resolving Adders". This change has the effect of allowing more time for each datapath arithmetic block to act on its data inputs. The existing "latches" in the first preferred embodiment have been changed to 2-phase "flip-flops" or "registers".
The latching memory elements located on the front and end of the existing 1D IDCT datapath pipelines have been combined into single blocks, as shown particularly in particular in FIG. 18. Additionally, the amount of memory elements present at the input and the output of the second preferred embodiment has been increased to allow variable amounts of T2 data to be buffered.
As shown in FIGS. 16 and 17, the two data streams, stream "T1" (raw unoperated upon data) and stream "T2" (data which has been through the 1D IDCT once and has been transposed in the TRAM), are introduced into the datapath pipeline in a time multiplexed fashion.
In the present invention, each stream takes its turn to introduce a group of data items into the datapath pipeline. The data streams are "interleaved" as they pass sequentially down the datapath pipeline and are "de-interleaved" at the datapath output, as shown in FIGS. 17, 18 and 33. A group can vary in number, but in this example, they are eight bits.
In accordance with the present invention, Ti must not be stalled. If T2 arrives at the point of interleaving with T1, but the input buffer should not introduce its data into the pipeline because this would clash with the T1 stream, then stream T2 provides an extra buffering so that T2 does not stall the data stream, but instead will buffer up data from its input stream until such a time as it may safely interleave with stream T1. This is shown in FIGS. 19 and 33 where the data from stream T1 is being loaded into the first transform in latches 0-7, using signals, "Latch 1(0) `through` Latch 1(7)". Additionally, data from T2 is being loaded in "Latch 2(0) `through` Latch 2(15)", as shown in FIG. 19, using signals shown in FIG. 33.
The interleaving is controlled by "T1 OK2 insert" and "T2 OK2 insert" signals. Under normal operation, the interleaving will occur when the signals go high. However, if the appropriate amount of data in the latch for T2 has not yet been reached when "T2 OK insert" goes high, then the latch will miss its opportunity and must continue buffering data until the next opportunity to insert data occurs.
In summary, if the above described buffering, in accordance with the present invention, is to occur, comparable "slippage" has to occur at the output of T2. T2 slips when it misses its data insertion point and has to continue buffering in the latches shown in FIG. 19. If T2 slipped and did not introduce data into the pipeline there will be a corresponding gap in the T2 stream output at the datapath output. This gap may be removed or "swallowed up" by use of the extra buffering at the T2 output. This process may be thought of as having a "fixed" T1-1D IDCT transform with a variable T2-1D IDCT, where the data streams are interleaved in a time multiplex fashion such that they may use the same piece of arithmetic datapath pipeline.
In the present invention, "Recovery" takes place when non-data enters T1. It is an opportunity for the T2 buffer to catch up to T1 and the datastream. Non-data is a data type that bypasses the IDCT and is shown as a data spike in "Latch 2 [φ]" of FIG. 34. This eventually makes its way to T2 input, which allows the T2 buffering to fill up at the output. Recovery is shown in FIG. 33 and FIG. 25 when the "T2 dout" signal and the "out" signal are gapped by a number of cycles. The gap is used as a reference to fix the data stream. It should be noted that the gap in cycles between these two signals is the same as the gap of buffering when the latch for T2 was waiting to insert its data.
Following the TRANSFORM in POSTC 233 part B, the interleaved stream is de-interleaved into "T2 out", as shown in FIGS. 18 and 23. The "T2 out" data stream has slip gaps in the data as described above. The T2 out [143:φ], shown in FIG. 17, enters a 16 to 1 multiplexor block, shown as block "IDDPMUX" in FIG. 17. This multiplexor block will select data from one of 16 positions in the output buffer block, as shown in FIG. 25. This position is selected by the control logic, shown in FIG. 29, which uses the gap by which T2 "buffered-up" at its input. This gap is used as a reference. The output stream, T2DOUT, from the multiplexer block is the "fixed" data stream.
In range tests carried out on an embodiment of the present invention for the IDCT arrangement described above, it was found that all intermediate and final values were kept well within a known range at each point while still meeting the CCITT standards. Because of this, it was possible to "adjust" selected values as described above by small amounts (for example, by forcing certain bits of selected data words to desired values) without any fear of overflow or underflow in the arithmetic calculations.
The method and system, in accordance with the present invention, can be varied in numerous ways. For example, the structures used to resolve additions or multiplications may be altered using any known technology. Thus, it is possible to use resolving adders of subtractors where the preferred embodiment uses carry-save devices with separate resolving adders. Also, the preferred embodiment of the present invention uses down-scaling at various points to ensure that all values remain within their acceptable ranges. Down-scaling is not necessary, however, because other precautions may be taken to avoid overflow or underflow.
In one embodiment of the present invention, certain bits of various data words were manipulated to reduce the required word width within the system. However, the various intermediate values may, of course, be passed without bit manipulation. Furthermore, although only data words were bit-manipulated in the illustrated example of the present invention, it is also possible to manipulate the bits of constant coefficients as well and evaluate the results under the CCITT standard. If a comparison of the results showed that it would be advantageous to force a particular bit to a given value, in some cases, on might then be able to increase the number of "zeros" in the binary representation of these coefficients in order to decrease further the silicon area required to implement the corresponding multiplier. Once again, bit manipulation is not necessary.
In summary of the above aspects of the present invention, the following is disclosed: an apparatus for transforming data having a first latch defining a first data stream source and a second latch defining a second data stream source. The first and second latches are in communication with a single arithmetic unit. The arithmetic unit communicates data to a transpose RAM, the transpose RAM transposes the data and communicates it to the second latch. The second latch is adjustable and can be varied in size to accommodate variable rates of data being received and transmitted. The second latch and first latch communicate 1st and 2nd data stream to the arithmetic unit sequentially, however, the sequential communication of the second latch does not interrupt the communication from the first latch. In this manner, common arithmetic unit is used for a first and second data stream. Furthermore, a process for transforming data using a common arithmetic unit having the following steps is described. First, loading the data into a first latch and, upon reaching a predefined number of cycles transmitting the data to an arithmetic unit and loading a first marker bit into a control shift register. Next, loading data into a second latch, the second latch is adjustable and can be varied in size to accommodate variable rate of data being received and transmitted at different rates. The next step is to transmit the data in the second latch to the arithmetic unit when the first control shift register reaches a predetermined state and the second latch is filled with a predetermined amount of data. Next, preventing transmission of data from the second latch, if the second latch is not filled with a predetermined amount of data and then recovering the second latch when the first latch is receiving non data.
Detailed Description of Invention for Time Synchronization
In MPEG-2, video and audio data is synchronized using information carried in the MPEG-2 systems stream. In this regard, there are essentially two types of information that deal with synchronization; clock references and time stamps. Clock references are used to inform the decoder what number is used to represent the time "now". This is used to initialize a counter that is incremented at regular intervals so that the decoder always knows what the current time is.
Time stamps are carried in some of the streams of data that are used to make up the programme (typically video and audio). In the case of video, a time stamp is associated with a picture and tells the decoder at what "time" (defined by the counter that was initialized by the clock reference) a picture should be displayed.
In MPEG, multiplexed into the system stream are a series of clock references. These clock references define the "system time". There are two types of clock reference; Program Clock References (PCRs) and System Clock References (SCRs). In the present invention, the distinction between PCRs and SCRs is not relevant since each of the clock references are used in the same manner by the decoder. PCRs and SCRs have timing information to a resolution of 90 kHz with a further field extending the resolution to 27 MHz (or 1/27×10e6 in seconds). Clock references are included in the data stream fairly often in order that "system time" may be reinitialized after a random access or channel change.
Accordingly, it is important to appreciate that timestamps refer to a hypothetical model of a decoder that can decode pictures instantly. As will be appreciated by one of ordinary skill in the art, any real decoder cannot do this and must take steps to modify the theoretical time in which pictures should be displayed. Furthermore, time stamps and the clock references are used to determine display time and errors in display time. This modification depends upon the details of the architecture of the particular decoder. Clearly any delay introduced by the video decoder must be matched by an equivalent delay in the audio decoder.
When decoding MPEG, discontinuities in the concept of "system time" may occur. For instance in an edited bitstream, each edit point will have discontinuous time. A similar situation occurs at channel change. It will be appreciated that care must be taken when using time stamps, because using a time stamp that was encoded in one time regime with respect to a "system time" defined by a clock reference from another regime will clearly lead to incorrect results.
FIG. 39 shows the demultiplexing of the MPEG systems stream into elementary streams 250. Each elementary stream will typically carries either video or audio data although, in general, any form of data may be transported. Each elementary stream is divided into a series of access units. In the case of video, the access unit is a picture. In the case of audio, it is a fixed number of samples of audio data.
Also multiplexed into the systems stream are a series of clock references. These clock references define the "system time".
In accordance with the present invention, associated with each elementary stream is a series of time stamps 251. The time stamps specify the "system time" at which the next access unit for the respective elementary stream is to be presented. These time stamps are referred to as presentation time stamps, "PTS".
In the case of video data, a second type of time stamp is also defined is referred to as a decode time stamp, "DTS". Since the DTS is only present when a PTS is also present and there is a simple relationship between them, the detailed differences between these two types of timestamps can be ignored since PTS/DTS differences have no bearing on the present invention.
The decode time stamps (DTS) define the time at which an access unit (picture in the case of video) is to be decoded. The presentation time stamps (PTS) define the time at which an access unit is to be presented (displayed). However, the timing model used is a hypothetical model in which the decoder is infinitely fast. In this case, the DTS and PTS would be identical to one another.
However, in MPEG video decoding, some of the pictures are reordered. Therefore, after decoding, the pictures are held in storage for a time period, e.g., several frame times, before they are displayed. During this time period, other pictures that are decoded subsequent to the picture in question are displayed. Consequently, for these reordered pictures there is a difference between the DTS and PTS.
In accordance with the present invention, it will be appreciated that to properly synchronize time, it is necessary to be consistent in the use of time stamps. In one preferred embodiment, the time synchronizing circuitry is placed at a point in the decoding pipeline when the pictures occur in their decoded order. Accordingly, this embodiment uses the DTS.
Nevertheless, the circuitry could equally be moved to a point in the decoding pipeline that occurs after the pictures are reordered and, therefore, the pictures would reach the synchronizing circuitry in their display order. Hence, as will be appreciated by one of ordinary skill in the art, PTS would be used in this embodiment.
In the preferred embodiments of the present invention, the information derived from the timestamps is transported through the various circuits by means of tokens. Tokens consist of a series of one or more words of information. The first word of the token contains a code which identifies the type of token and, hence, the type of information carried by that token. Associated with each word of the token is an extension bit which is set to one to indicate that there are more words in the current token. Therefore, the last word of a token is indicated by the extension bit being zero. In the present invention, the code in the first word indicating the type of token may be of a variable number of bits so that some codes use a small number of bits (allowing the remainder of the bits in the first word to be used to represent other information) while other codes use a larger number of bits.
Tokens may be characterized as being either control or DATA tokens. For example, at the interface between the system decoder and the video decoder, there are two types of information: (1) the coded video data and (2) the synchronization time derived from the time stamp information. The coded video data is viewed as data and is carried in a DATA token (e.g., the token called DATA) while the synchronization time is viewed as control information and is carried in a control token (called SYNC-- TIME). Additional control tokens may also be used from time to time in the present invention. For example, a FLUSH token that behaves in a manner similar to a reset signal may be required to initialize the video decoding circuitry before attempting to restart decoding because of an error.
In accordance with the present invention, it is an object of one preferred embodiment to time synchronize two circuits and, more particularly, to time synchronize two circuits without directly communicating system time from the first to the second circuit. In accordance with the invention, time synchronization of two circuits is accomplished without passing system time directly to the second circuit by providing synchronized time counters in each circuit.
The present invention also enables the system to time synchronize two circuits without communicating system time from the first to the second circuit by providing an elementary stream time counter in each circuit.
Accordingly, another object of the present invention is to time synchronize two circuits and to determine the presentation time error, if any, of the object being presented by using time stamp information, system time, and elementary stream time from the first circuit to generate synchronization time passed to the second circuit and compared to a copy of elementary stream time in the second circuit which is synchronized with the elementary stream time in the first circuit. The system of the present invention can time synchronize a system decoder and a video decoder without directly communicating system time from the system decoder to the video decoder, without passing system time directly to the video decoder by providing synchronized time counters in each circuit and without communicating system time from the system decoder to the video decoder by providing a video counter in each circuit.
The invention also enables the system to time synchronize a system decoder and a video decoder and to determine the display time error, if any, of the picture being displayed by using video time stamp information, system time, and video decoding time from the system decoder to generate synchronization time which is then passed to the video decoder and compared to a copy of video decoding time in the video decoder which is synchronized with the video decoding time in the system decoder.
In accordance with the present invention, information derived from the timestamps can be transported through the system using a control token as previously described.
FIG. 40 shows a first preferred embodiment implementing elementary stream timestamp management, in accordance with the present invention. The clock references 253, which represent system time, are decoded by the system demultiplexer 254 and placed initially, and then as needed, into a time counter 255 within the system decoder 256, and are incremented at 90 kHz. A second copy of the clock reference 253 is simultaneously loaded into the time counter 258 that is inside the elementary stream decoder 257, incremented also at 90 kHz, and synchronized to the time counter 255 in the system decoder 256.
The time stamps 251, in accordance with the present invention, flow from the system demux 254 through the elementary stream buffer 260 so that they are delayed by the same amount as the incoming data. The time stamps 251 may also have a correction added to compensate for the non-zero decode time of the elementary stream decoder 257. The corrected time stamps 251 are then compared with the copy of the time stored in the time counter 258 inside the elementary stream decoder 257 to determine whether the decoded information is presented too early or too late.
The above embodiment is better than merely passing system time directly to the elementary stream decoder 257 from the time counter 255 in the system decoder 256 because the counter in the system decoder changes 90,000 times a second. Therefore, system time would, in all essence, need to be continually passed to the elementary stream decoder 257. Passing system time continually would require dedicated pins or the like. By using a time counter 255 located in the system decoder 256 and a time counter 258 located in the elementary stream decoder 257, system time can be passed in the form of clock references 253 a few times a second.
Another embodiment is shown in FIG. 41. The embodiment shown in FIG. 41 avoids the need for the clock references 253 to be passed to the elementary stream decoder 257. This is achieved by using a second counter "es-- time" 262, containing information on elementary stream time, which is maintained in both the system decoder 256 and the elementary stream decoder 257. The two es-- time counters 262 and 263 are reset at power on, and at other times such as channel change, and then they free run from there on. Since this embodiment depends on the two es-- time counters 262 and 263 staying in step, it will be appreciated that it is necessary to take measures to ensure the es-- time counters do not get out of step. One way to ensure the es-- time counters 262 and 263 stay in step is to use carry out of the es-- time counter in the system decoder to reset the es-- time counter in the elementary stream decoder 257 as shown in FIG. 41.
As further shown in FIG. 41, the clock references 253, which represent system time, are decoded by the system demultiplexer 254 and placed into a time counter 255 within the system decoder 256 and incremented at 90 kHz. The es-- time counter 262 in the system decoder 256 of the present invention and the es-- time counter 263 in the elementary stream decoder 257 of the present invention are synchronized with each other and incremented at 90 kHz. Elementary stream time stamps are also decoded by the system demultiplexer 254. Accordingly, a synchronization value X is computed using the elementary stream timestamp, the system time contained in the time counter and the elementary stream time contained in the es-- time counter 262 contained in the system decoder 256 according to the equations 3-1.
The following set of equations 3-1 (a-d) is illustrative of one method in accordance with the present invention, for time synchronization which avoids passing the clock references 253 to the elementary stream decoder 257. Equation 3-1 (a) is the equation required for time synchronization. Since it is undesirable to pass system time directly to the elementary stream decoder circuit 257, as shown in FIG. 41, a synchronization time representation X is generated, using Equation 3-1 (b-d), by the system decoder 256 and this value is passed to the elementary stream decoder. Synchronization time X is then compared to the elementary stream time contained within the es-- time counter 263 located within the elementary stream decoder 257. Hence, the compared result is used to determine whether the decoded information is presented too early or too late and then is further used in time synchronizing the system.
Equations 3-1:
a) Time Synchronization=(Elementary stream timestamp-system time)
b) Time Synchronization=(X-elementary stream time)
c) (X-elementary stream time)=(elementary stream timestamp-system time)
d) X=(elementary stream timestamp-system time+elementary stream time)
In the present invention, the synchronization time, X, may have a correction added to compensate for the non-zero decode time of the elementary stream decoder 257. The corrected synchronization time is then compared with the elementary stream time contained in the es-- time counter 263 located inside the elementary stream decoder 257 to determine whether the decoded information is presented too early or too late and is further used to time synchronize the system. Note, the time correction could be subtracted from elementary stream time contained in the es-- time counter 263 located inside the elementary stream decoder 257 instead of added to synchronization time X for the same result. The above embodiment is an example of a solution for generating synchronization time X and determining whether the information is presented early or late. It will be apparent to those skilled in the art that there are many other equivalent solutions for accomplishing the above.
For example, FIG. 42 shows an alternative method for determining the synchronization time, X, in accordance with the present invention. In this arrangement, the system decoder 256 does not maintain an elementary stream time. Instead it records, in the register initial-- time 265, the value of system time at the instant that the elementary stream time counter, es-- time 263, located in the elementary stream decoder 257 is reset to zero. The value in es-- time 263 can be computed by the system decoder 256 because it will be the difference between the current system time and the value recorded in initial-- time.
The following equations 3-2 (a-c) is illustrative of this alternative method for time synchronization. Equation 3-2 (a) is the equation representing the value of the elementary stream time stored in es-- time 263 located in the elementary stream decoder 257. This is substituted into equation 3-1 (d) to give equation 3-2 (b) which is simplified to derive equation 3-2 (c) providing the synchronization time, X, as a function of the system time and the value stored in the initial-- time register 265.
Equations 3-2:
a) elementary stream time=system time-initial-- time
b) X=(elementary stream timestamp-system time+[system time-initial-- time])
c) X=(elementary stream timestamp-initial-- time)
Two solutions for deriving the synchronization time, X, in accordance with the present invention have been illustrated. However, it will be apparent to those skilled in the art that there are many other equivalent solutions.
FIG. 43 shows another embodiment of the present invention implementing video timestamp management. The clock references 253, which represent system time, are decoded by the system demultiplexer 254 and placed initially, and then as needed, into a time counter 255 within the system decoder 256 and are incremented at 90 kHz. A second copy of the clock references 253 are simultaneously loaded into the time counter 258 that is inside the video decoder 270 and incremented at 90 kHz, and synchronized to the time counter 255 in the system decoder 256.
The video time stamps flow from the system demux 254 through the video decoding buffer 271 so that they are delayed by the same amount as the incoming video data. The video time stamps may have a correction added to compensate for the non-zero decode time of the video decoder 270. The corrected video time stamps are than compared with the copy of the time in the time counter 258 inside the video decoder 270 to determine whether the decoded picture is displayed too early or too late.
The embodiment shown in FIG. 43 is an improvement over the process of merely passing system time directly to the video decoder from the time counter in the system decoder because the counter in the system decoder changes 90,000 times a second. Therefore, system time would in all essence need to be continually passed to the video decoder. Passing system time continually would require dedicated pins or the like. By using a time counter located in the system decoder and a time counter located in the video decoder system time can be passed in the form of clock references a few times a second.
Referring now to FIG. 44, the clock references, which represent system time, are decoded by the system demultiplexer 254 and placed into a time counter 255 within the system decoder 256 and incremented at 90 kHz. The vid-- time counter 272 in the system decoder 256 and the vid-- time counter 273 in the video decoder 270 are synchronized with each other and incremented at 90 kHz. Video time stamps are also decoded by the system demultiplexer 254. Accordingly, a synchronization value X is computed using a video timestamp, the system time contained in the time counter 273 and the video decoding time contained in the vid-- time counter 272 contained in the system decoder 256 according to the equations 3-3.
The following set of equations 3-3 (a-d) is illustrative of one method in accordance with the present invention, for time synchronization which avoids passing the clock reference 253 to the video decoder 270. Equation 3-3 (a) is the equation required for time synchronization. Since it is undesirable to pass system time directly to the video decoder circuit 270 as shown in FIG. 44, a synchronization time representation X is generated, using Equation 3-3 (b-d), by the system decoder 256 and passed to the video decoder 270. Synchronization time, X, is then compared to the video decoding time contained within the vid-- time counter 273 located within the video decoder 270. The compared result is used to determine whether the decoded picture is displayed too early or too late and then further used in time synchronizing the system.
Equations 3-3:
a) Time Synchronization=(Video timestamp-system time)
b) Time Synchronization=(X-video decoding time)
c) (X-video decoding time)=(video timestamp-system time)
d) X=(video timestamp-system time+video decoding time)
In the present invention, the synchronization time, X, may have a correction added to compensate for the non-zero decode time of the video decoder. The corrected synchronization time is then compared with the video decoding time contained in the vid-- time counter 273 located inside the video decoder 270 to determine whether the decoded picture is displayed too early or too late and is also used to time synchronize the system. Note, the time correction can be subtracted from the video decoding time contained in the vid-- time counter 273 located inside the video decoder 270 instead of added to synchronization time X for the same result. The above embodiment of the present invention is another example of a solution for generating synchronization time X and determining whether the picture is displayed early or late. However, it will be apparent to those skilled in the art that there are many other equivalent solutions for accomplishing the above.
Another nice feature, in accordance with the present invention, is that there is no need to deal with the full 33 bit time stamp number or 42 bit clock reference number. The present invention restricts the counters to 16 bits to allow 16 bit handling on the video decoder 270. At first glance, it would appear that 16 bits cannot represent a sufficient number range at a resolution of 90 kHz (only 2/3 second to be used). However, there is no need for such high precision because the time control on the video decoder 270 is only accurate to a field time (since the video timing generator VTG free-runs or is gen-locked to something that has nothing to do with the MPEG stream being decoded) and, therefore, it is not related to timestamps or presentation time in any way.
As shown in FIG. 44 and FIG. 45, the synchronization time X and the vid-- time counter 273 within the video decoder 270 use only sixteen bits. This is made possible by two factors. First, the difference between system time and the timestamp (used to derive the synchronization time; see Equation 3-3) should always be small, thus allowing the more significant bits to be discarded. Second, it is only possible to control the presentation of video to the nearest frametime. Therefore, the less significant bits are not required and are discarded by shifting right by four bits.
Thus, the sixteen bits of time information used in the present invention are able to deal with timing errors of up to about 11.5 seconds with an accuracy of about 180 μs (about 1% of a field time). A PAL or SECAM European 625 line TV system is, thus, 112.5 ticks of the 5625 Hz clock; a NTSC 525 line TV system is 93.84 ticks. Hence, using 16 bits allows timing calculations with an accuracy of about 1% of a field time.
FIG. 46 shows the preferred process, in accordance with the present invention, of the moving the time stamp through the hardware. The preferred method for communicating information in this hardware is Tokens, but it will be appreciated that alternative methods may also be employed. The hardware is divided into two modules. The first module is added just after the Start Code Detector 201. This module is responsible for generating a token, SYNC-- TIME containing the synchronization time X, as discussed above, and this occurs just before an associated PICTURE-- START token. In the MPEG systems stream, the time stamp is carried in a packet header and refers to the first picture in the packet of data. Since the packets do not line up with the video data, there will, in general, be the end of the previous picture before the start of the picture to which the time stamp refers.
The synchronization time information may be supplied to the present invention either via a microprocessor interface or by using a Token. In either case, the synchronization time date (16 bits) is stored in the synchronization time register (divided into two parts to allow access to each byte individually), as further detailed in Table 12.
                                  TABLE 12                                
__________________________________________________________________________
Microprocessor registers for handling synchronization time                
Register Name                                                             
       Size/Dir                                                           
           Reset State                                                    
                 Description                                              
__________________________________________________________________________
ts.sub.-- low                                                             
       8/rw                                                               
                 The lower eight bits of the synchronization time value.  
                     The ts.sub.-- low register is slaved so that new     
                 values may                                               
     be written into this register without affecting the value            
     previously written (that will become part of a                       
     SYNC.sub.-- TIME token).                                             
     Writes to ts.sub.-- low register affect the master register          
     whilst reads read-back the slave register. Until a                   
     master -to-slave transfer has been effected using                    
     ts.sub.-- valid the value written into ts.sub.-- low cannot be read  
                     back.                                                
  ts.sub.-- high 8/rw                                                     
  The upper eight bits of the synchronization time value.                 
     Slaved in the same way as ts.sub.-- low.                             
  ts.sub.-- valid 1/rw 0 This bit controls the master-slave transfer of   
                 ts.sub.-- low                                            
     and ts.sub.-- high.                                                  
     When values have been written into ts.sub.-- low and                 
     ts.sub.-- high the microprocessor should write the value one         
                     into this bit. It should then poll the bit unit it   
                 reads back                                               
     the value one. At this point the values written into                 
     ts.sub.-- low and ts.sub.-- high will have been transferred into the 
     slave registers (and can be read back) and ts.sub.-- waiting         
                     will be set to one.                                  
     The microprocessor should then write the value zero in               
     preparation for the next access.                                     
  ts-waiting 1/ro 0 When set to zero the registers ts.sub.-- low and      
                 ts.sub.-- high do                                        
     not contain valid synchronization time information.                  
     When set to one the registers ts.sub.-- low and ts.sub.-- high       
                     contain valid synchronization time information. A    
                     SYNC.sub.-- TIME token will be generated before the  
                 next                                                     
     PICTURE.sub.-- START token and ts.sub.-- waiting will then           
     become zero.                                                         
     This bit should be polled to ensure that it is zero before           
     writing a one into ts.sub.-- valid to ensure that the previous       
                     synchronization time value has been used before it   
                 is                                                       
     overwritten by the master-to-slave transfer.                         
__________________________________________________________________________
In the present invention, a flag, ts-- waiting, is set to indicate the fact that valid synchronization time information is in the timestamp register. If the data was supplied using the SYNC-- TIME token, then that token is removed from the stream of tokens.
When a PICTURE-- START token is encountered, the flag that indicates the status of the synchronization time register is examined. If the flag is not set, then no action is taken and the PICTURE-- START token and all subsequent data is unaffected. If, however, the flag is set, indicating that valid synchronization time information is available in the register, then a SYNC-- TIME token is generated and placed in the data stream before the PICTURE-- START token. The flag is then cleared and the synchronization time register is made available for the next time-stamp that occurs.
The second module as shown in FIG. 46, consists of a prescaler clocked at 27 MHz and a vid-- time counter clocked by the prescaler 278 which are associated with the microprogrammable state machine, (MSM) 218.
There is a prescaler 278 that divides the clock by 4800, as shown in FIG. 44 and FIG. 46. In other words, 4800 is 300 (27 MHz/90 kHz) times 16. The 4804.8 option shown in FIG. 45 and FIG. 46 is discussed below.
In the NTSC color television, the frame rate is not 30 Hz but is, in fact, approximately 29.94 Hz (precisely 30000/1001 Hz). [Before the advent of color television 30 Hz precisely was used.] There are precisely 1716, 27 MHz clock periods per NTSC line time (line time is 1/525 of frame time).
The American television standards body has expressed an interest in returning to 30 Hz in the future (or more probably 60 Hz for HDTV). As a result MPEG supports a frame rate of 30 Hz precisely. However, since it is not possible to generate a stable 30 Hz television signal from a 27 MHz clock (there being 1714.29 . . . cycles per line) it is difficult to generate a 30 Hz raster in the MPEG framework.
One possible solution is to "bend" the clock rate at the decoder so that instead of producing a 27 MHz clock, a 27.027 MHz clock is generated. This clock is generated using the MPEG clock references with a divider of 300.3 (rather than 300) to yield the 90 kHz clock. This 27.027 MHz clock when clocking the identical video timing circuitry that provides a 29.94 Hz frame rate from 27 MHz will give a precise 30 Hz rate.
In the framework of the present invention, the 90 kHz is prescaled by a further factor of 16. Accordingly, division of the 27.027 MHz clock by 300.3×16=4804.8.
The Vid-- time counter 273 (discussed above) contains the video decoding time and is incremented each time that the prescaler reaches its terminal count. The vid-- time counter 273 is reset by the reset-time pin.
The prescaler and vid-- time counter of the present invention can be implemented with fully clocked feed-back flip-flops which are much more resistant to α-particle corruption than the resistive-feedback or weak-feedback latches used elsewhere. Using clocked feedback flip-flops for time counters will help ensure that the time counter in the video decoder stays in step with the time counter in the system decoder.
FIG. 47 illustrates the process the MSM 218 performs when it receives the SYNC-- TIME token. The MSM 218 is able to read the current time indicated by the video time counter and to then compare it with the value supplied by the video SYNC-- TIME token. It can, therefore, determine whether it is early or late, as compared to the time at which it should be decoding the pictures.
In the present invention, a 16 bit signed timestamp correction is added to the synchronization time X (discussed above) that was carried by the video SYNC-- TIME token. This correction is reset to zero by the MSM 218 at chip-reset, and if no action is taken, the synchronization time remains be unaltered. The controlling microprocessor can always write value into ts-- correction to modify the synchronization time and, therefore, compensate for differential delays through the video and audio decoders.
The current video decoding time contained in the vid-- time counter 273 is subtracted from the corrected synchronization time. The sign of value gives the direction of the error (and determines the error code, if any, generated by the MSM 218). The absolute value of the difference is then taken and the result is compared to a threshold value to determine whether the timing error is within acceptable limits. Since, at present, the video timing can only be controlled to an accuracy of plus or minus a frame time from the nominal time (because the VTG 333 free-runs) this threshold is set at one frame time.
If the error exceeds a frame-time, then some correction must be made. The MSM 218 of the present invention can correct the situation itself if the decoding is too early since the MSM can simply delay the decoding until the appropriate time. However, if the decoding is later than the intended time, then time correction is more difficult because it is not possible to discard pictures reliably at the output of the coded data buffer. Essentially, the decoding of the sequence is broken and the most reliable way to correct the situation is to restart the decoding process in a manner similar to random-access or channel change. In order to facilitate this process, the control register of the MSM 218 may be programmed to discard all data until a suitable start code or FLUSH token is encountered. In addition, the error "ERR-- TOO-- EARLY" is not generated during start-up, irrespective of the setting of disable-- too-- early, because at start up, the first picture is expected to be early.
Table 13 is illustrative of how the MSM 218 registers work and details some of the actions and error messages information placed in the registers can generate.
                                  TABLE 13                                
__________________________________________________________________________
Timestamp MSM registers                                                   
Register Name                                                             
         Size/Dir                                                         
             Reset State                                                  
                   Description                                            
__________________________________________________________________________
ts.sub.-- correction                                                      
         16/rw                                                            
             zero  Correction added to synchronization time               
     before it is used.                                                   
  frame.sub.-- time 16/ rw  226 or 188 Represents the tolerance on the      
                   timing of                                              
     decoding pictures. Reset state determined                            
     by the PAL/NTSC pin.                                                 
  vid.sub.-- time 16/ro zero Reset by either reset or reset.sub.-- time.  
                   The                                                    
     current value of video decoding time.                                
  manual.sub.-- startup 1/rw zero When set to one the start-up is to be   
                       performed manually using                           
     decode.sub.-- disable. In this case                                  
     SEQUENCE.sub.-- END and FLUSH tokens at                              
     the MSM cause decode.sub.-- disable to be set                        
     to one.                                                              
  decode.sub.-- disable 1/rw zero When set to zero the decoding proceeds  
                       normally.                                          
     At the start of each picture the MSM                                 
     checks the status of decode.sub.-- disable and                       
     will not proceed if it is set to one.                                
     Note that if manual start-up is to be                                
     performed (i.e. without the time-stamp                               
     management hardware) then this bit                                   
     should be set to one at the same time as                             
     manual.sub.-- startup is set to one.                                 
  disable.sub.-- too.sub.-- early 1/rw zero When set to one the error     
                       "ERR.sub.-- TOO.sub.-- EARLY" indicating that the  
                       decoding is too early is suppressed and the        
                       MSM simply waits to correct the situation.         
                    NTSC.sub.-- 30 1/rw zero When set to one the prescaler
                    divides by                                            
     4804.8 rather than 4800. Set automatically                           
     when decoding 30 Hz frame rates.                                     
  discard.sub.-- if.sub.-- late 1/rw zero This has no effect unless an    
                       "ERR.sub.-- TOO.sub.-- LATE" is generated (or      
                   would                                                  
     be generated if errors were not masked out).                         
     If it is set to one then data is discarded until                     
     the condition indicated by discard.sub.-- until.                     
  discard.sub.-- until 2/rw zero Indicate the condition which causes      
                   time-stamp                                             
     triggered discarding to be terminated.                               
     0 - FLUSH                                                            
     1 - SEQUENCE.sub.-- START                                            
     2 - GROUP.sub.-- START                                               
     3 - NEXT PICTURE                                                     
     Note 1 - that discarding one picture may                             
     immediatety be un-done if that picture is a field                    
     picture by the generation of a dummy field to                        
     preserve the alternating top/bottom field                            
     structure. As a result if discard.sub.-- until is set to             
     "Next Picture" but the dummy field would be                          
     generated one further picture is discarded.                          
__________________________________________________________________________
As a result of the synchronization time handling of the present invention, it is possible that one of two errors will be generated.
ERR-- TOO-- EARLY is generated if the decoding is taking place earlier than the time indicated by the time-- stamp. ERR-- TOO-- EARLY may be suppressed, but ERR-- TOO-- LATE will always be generated unless all errors are masked out.
In summary, the present invention includes: an apparatus for synchronizing time having, a timestamp for determining presentation time, a clock reference for initializing system time in a first circuit, a first time counter in communication with the clock reference for keeping system time in a first circuit and a second time counter initialized by the clock reference in a second circuit synchronized with the first time counter, for keeping a local copy of the system time and for determining the presentation timing error between the local copy of system time and system time by comparing the timestamp to the second time counter. It further includes an apparatus for synchronizing a system decoder and a video decoder using a timestamp for determining display time, a clock reference for initializing system time in the system decoder, a first time counter in communication with the clock reference for keeping system time in the system decoder and a second time counter initialized by the clock reference in the video decoder synchronized with the first time counter, for keeping a local copy of system time and for determining the display timing error between the local copy of system time and system time by comparing the timestamp to the second time counter. A still another embodiment includes an apparatus for synchronizing a first circuit and a second circuit using a clock reference for initializing system time in the first circuit, a first circuit having a time counter in communication with the clock reference for keeping system time, a first elementary stream time counter in the first circuit for providing elementary stream time. The first circuit is adapted to receive a time stamp, and the first circuit generates synchronization time by adding elementary stream time to the time stamp and subtracting system time. The second circuit is adapted to receive synchronization time from the first circuit and has a second elementary stream time counter in synchronization with the first elementary stream time counter for providing a local copy of the elementary stream time and for determining a timing error between the system time and the time stamp by comparing synchronization time to the local copy of elementary stream time. In this way, the clock reference signal does not have to be passed directly to the second circuit in order to determine the timing error. In another embodiment, an apparatus for synchronizing a first circuit and a second circuit has a clock reference for initializing system time in the first circuit. The first circuit has a time counter in communication with the clock reference for keeping system time, and a first video time counter for providing video decoding time. The first circuit is adapted to receive a video time stamp and generates synchronization time by adding video decoding time to the video time stamp and subtracting system time. The second circuit is adapted to receive synchronization time from the first circuit and has a second video time counter in synchronization with the first video time counter for providing a local copy of video decoding time and for determining a timing error between system time and the video time stamp by comparing synchronization time to the local copy of video decoding time. Accordingly, the clock reference signal does not have to be passed directly to the second circuit in order to determine the timing error. The present invention also includes a method for providing timing information by providing a video data stream having a time stamp carried in packet header wherein the time stamp refers to the first picture in the packet of data. In the next step a register is provided having a flag used to indicate valid time stamp information which is taken from the packet header and placed into the register. Next, the timestamp is removed from the video data stream and placed in the register. Next, the method encounters a picture start and subsequently examines the status of the register to determine if valid time stamp information is contained in the register by checking the flag status. A time stamp is generated in response to the picture start if the flag indicates valid time stamp information is contained in the register and then the timestamp is inserted back into the data stream. Another embodiment includes an apparatus described above wherein the elementary stream time counters are restricted to 16 bits. Likewise, there is an apparatus as described above, wherein the second elementary stream time counter located in the elementary stream decoder is restricted to 16 bits. Furthermore, there is an apparatus as described above wherein the synchronization time is restricted to 16 bits for controlling the elementary stream decode. The present invention also has a process for decoding video and for determining display time errors against a threshold value. It then parses video data into tokens for further processing, determining if a time stamp token is indicated, comparing the time stamp token to a video time, and generates a compared value to determine an indicative of timing error. Next, it determines whether the compared value, when compared against a threshold value, is within acceptable parameters when a timing error is indicated and indicates when the compared value is outside acceptable parameters. An alternative embodiment includes an apparatus for using a system decoder and a video decoder. The system decoder is adapted to accept MPEG system streams and demultiplexing video data and the video time stamp from the stream. The system decoder has a first time counter representative of system time. The video decoder accepts the video data and the video time stamp, and has a second time counter in synchronization with the first time counter. The video decoder also has a video decoder buffer for accepting the video data at a substantially constant rate and outputting the video data at a varying rate and for passing a video time stamp. The video decoder while decoding a picture from the video data also compares the video time stamp for the decoded picture with the second time counter to determine the appropriate display time. There is also a method for determining a time error between a first circuit and a second circuit by providing the first circuit with a system time (SY), a time stamp (TS), and an elementary stream time (ET), obtaining synchronization time (X) by using the elementary stream time (ET), the time stamp (TS) and the system time (SY), in accordance with the equation; X=ET+TS-SY, providing synchronization time (X) to the second circuit and generating a synchronized elementary stream time (ET2) and obtaining a time error by using synchronized time (X) and in accordance with the equation ET2-X; hence, the first circuit can be time synchronized with the second circuit without passing system time to the second circuit. Another method for determining a time error between a first circuit and a second circuit has the following steps: providing the first circuit with a time stamp (TS), and an initial time(IT), obtaining synchronization time (X) by using the time stamp (TS) and the initial time (IT), in accordance with the equation X=TS-1, providing synchronization time (X) to the second circuit and generating a synchronized elementary stream time (ET) and obtaining a time error by using synchronized time (X) and in accordance with the equation ET-X. In this way, the first circuit can be time synchronized with the second circuit without passing system time to the second circuit. Still another method for determining a time error between a first circuit and a second circuit includes the following steps: providing the first circuit with a system time (SY), a video time stamp (VTS), and a video decoding time (VT), obtaining synchronization time (X) by using the video decoding time (VT), the video time stamp (VTS) and the system time (SY), in accordance with the equation; X=VT+VTS-SY, providing synchronization time (X) to the second circuit and generating a video decoding time (VT2) in the second circuit which is synchronized to the video decoding time (VT) in the first circuit, and obtaining a time error by using synchronized time (X) and in accordance with the equation VT2-X. Accordingly, the first circuit can be time synchronized with the second circuit without passing system time to the second circuit.
Detailed Description of the Invention for Asynchronous Swing Buffering
For asynchronous swing buffering, in accordance with the present invention, two buffers are operated asynchronously; one is written while the other is read. Accordingly, this allows for a data stream having a first rate of through-put to be resynchronized to another rate, while still maintaining a desired rate. In the invention, the write control and read control both have state indicators for communicating which buffer they are using and whether the controls are waiting for access or are, in fact, accessing that buffer. Each side communicates to the other side a single bit to indicate which buffer it is using. This is the only signal that must be synchronized between the two sides of asynchronous circuitry.
When one control circuit (read or write) finishes accessing a buffer, then the invention will allow control to pass to the other circuit. If, after the control has swung, and two control circuits are trying to use the same buffer, then the later control circuit will begin waiting. The control circuit will wait until each side is using alternate buffers, i.e., the other side has swung. If, after it has swung, it finds that it is now using the alternate buffer to the other side, it will not wait, but immediately commence accessing. This system of arbitration between the buffers is started up by both buffers using the same buffer, buffer 0, in this case. The read side starts up by waiting, while the write side is accessing, since there is nothing valid to read out of either buffer.
In one embodiment, in accordance with the present invention, the swing buffers are two discrete RAMS having all signals, such as enabling strobes, addresses and data multiplexed from either the read or write side, dependent on which buffer is being accessed by each side. This structure has been shown to use a lot of area in the busing of a large number of signals between the two buffers.
Combining the two RAMs into a single structure saves much of the busing area while still maintaining performance to the same standard. This structure contains twice as many rows of cells as one of the discrete RAMs found in the first embodiment of the present invention. However, the second embodiment must have two pairs of bit lines since the read and write to the discrete buffers is happening simultaneously and asynchronously. Each row will be of its original width (i.e., have the same number of cells) since accesses are the same width as for the discrete RAMS. Each pair of rows are accessed as if at the same address, but from different buffers, so they connect to a different pair of bitlines. Using the same address, these pair of rows can be readily accessed by one row decoder connected to the read address and one row decoder connected to the write address. Again, the read and write control never access the same buffer at the same time so there is no conflict as to which pair is accessed by which row decoder.
In the same way in which each row decoder can access rows from each buffer, both the read and write circuitry within the structure of the present invention connect to each pair of bitlines, one pair from each buffer. The read and writes are then multiplexed into each of the buffers and, for the same reasons explained above, there will not be conflict.
As shown in FIG. 48, a swing unit 1 includes swing buffers 10 with RAM 12 and 14 in accordance with the present invention. The swing unit 1 also includes a write control circuit and a read control circuit, which control the data into and out of the RAM 12 and 14. The read control circuit and the write control circuit accomplish this by use of strobes, data and address control lines, 8. Lines 7 and 9 are control lines to indicate the RAM used by the write control circuit and the RAM used by the read control circuit. Line 7 functions to control the write control circuitry, i.e., when the read control circuitry is using, RAM 12 if low, RAM 14 if high. Similarly, Line 9 functions to inform the read control circuitry that the write control circuitry is using RAM 12 if low, RAM 14 if high.
In the present invention, swing buffer 10 has two RAM arrays, 12 and 14. Swing Buffer 10 is capable of asynchronous, alternative reading and writing to the RAM area which enables the apparatus to achieve the necessary band width for high speed accessing of the memory. The RAMs 12 and 14 require the following signals: write address 16, read address 18, data in 20, data out 22; and a read and write enable signal (not shown). See also FIG. 49.
The write address and read address signals are multiplexed by multiplexers 24. The RAM array 12 and 14 operate with the write circuitry, row decoder and read circuitry in a conventional sense.
In the first embodiment of the present invention, during initiation of the swing buffer 10, RAM 12 will be written to until the control circuitry switches a write enable single to RAM 14.
Once the RAM array 12 has been written, it falls under the control of the read circuitry 4, to be read. During this time, the RAM array 14 is also being written. It is important to note when the RAM array falls under the control of the read array control 2, or the write control circuit 4, the control is established until reading or writing is completed and then control is turned over. In the situation where the read control circuit 4 is accessing the RAM array, such as 12, and the write control circuitry 2 needs to access the same RAM array 12, then the write control circuit will begin waiting.
Therefore, in accordance with the present invention, two control events are created. When a write control circuit or a read control circuit swings to a different RAM, it will either begin immediately accessing the RAM since the RAM is free and not under control of the alternative circuit, or it will begin to wait. During start up, the read side defers to the write side, since there is nothing valid to be read out of either buffer.
The second embodiment of the present invention is shown in FIG. 50. An integrated swing buffer 30 includes a RAM array 32 having the logical size of RAM array 12 combined with RAM array 14. In other words, there is the same amount of RAM in both the first and second embodiments, however, it is combined in the second embodiment. Accordingly, the integrated swing buffer has the advantage of saving much of the busing area while still performing the same swing buffer function.
In the second embodiment of the present invention, the write circuit and read circuit 34 and 36 respectively, are similar to those used in the swing buffer 10. However, these circuits now include selectors which choose from the pairs of bit lines described hereinafter. Likewise, the read access row decoder 38 and the write access row decoder 40 are similar to those contained in swing buffer 10, however, they are able to access a pair of rows as described hereinafter in FIG. 51.
As shown in FIG. 51, the particular structure of the integrated swing buffer 30, in accordance with the present invention, is detailed. Individual cells 42 are contained in rows 44. The read row decoder 38 and write row decoder 40 access the rows 44 in pairs. A pair of rows have the same address provided by the address lines 16 and 18. The read buffer line 52 and write buffer line 54 provide the control information for selecting one of the paired rows 44. The buffer 0 bitlines 48 and buffer 1 bitlines 50 connect to alternative rows of cells and to the read and write circuitry 34 and 36. For clarity in depicting the addressing, the lighter shading illustrates the read row decoder 38 accessing a row in buffer 0. Similarly, the darker shading illustrates the write row decoder 40 accessing a row in buffer 1.
In summary, the present invention includes a swing buffer apparatus having at least two RAM arrays, a write control circuit in communication with the RAM arrays for controlling data input into the RAM array, and a read control circuit in communication with the RAM arrays for controlling data output from the RAM arrays. Furthermore, the write control circuit and read control circuit are in communication with one another to allow a synchronized control of the RAM arrays. There is also a swing buffer apparatus having a RAM array, a write control circuit in communication with the RAM array through a pair of bit lines, a read control circuit in communication with the RAM array through another pair of bit lines and a read row decoder and a write row decoder for addressing the RAM through a pair of rows so that individual cells are read. The present invention also provides a method of asynchronously addressing RAM, by decoding at least a pair of cells in the RAM, using a row decoder to decode at least a pair of rows and selecting one of the rows to be assessed, using at least two pairs of bitlines connected to read a circuit and a write circuit and selecting the pair of bitlines to be used.
Detailed Description of the Invention for Storing Video Information
Video decompression systems contains three basic parts used to decode and display picture information. The three main parts of a video decompression system are the spatial decoder, temporal decoder and the video formatter. The present invention involves the temporal decoder and video formatter and the way in which the temporal decoder and video formatter manage their respective picture buffers, hereinafter the frame store buffer. In MPEG systems, the temporal decoder contains two frame store buffers and the video formatter contains two frame store buffers.
MPEG uses three different picture types: Intra (I), Predicted (P) and Bidirectionally interpolated (B). B pictures are based on predictions from two other pictures; one picture is from the future and one from the past. The I pictures require no further decoding by the temporal decoder, but must be stored in one of the two frame store buffers for later use in decoding P and B pictures. Decoding a P picture requires forming predictions from a previously decoded P or I picture. The decoded P picture is stored in a frame store buffer for use in decoding further P and B pictures. B pictures can require predictions from both of the frame store buffers. However, B pictures are not stored in the frame store buffers.
It will be appreciated that I and P pictures are not output from the temporal decoder as they are decoded. Instead, I and P pictures are written into one of the frame store buffers, and they are read out only when a subsequent I or P picture arrives for decoding. In other words, the temporal decoder relies on subsequent P or I pictures to flush previous pictures out of the two picture buffers. Accordingly, the spatial decoder of the present invention can provide a fake I or P picture when it is necessary to flush the temporal decoder's two frame store buffers. In turn, this fake picture is flushed when a subsequent video sequence begins.
As shown in Table 14, the picture frames are displayed in numerical order.
              TABLE 14                                                    
______________________________________                                    
Frame Stores                                                              
______________________________________                                    
Display Order                                                             
         I1    Be    B3  P4   B5  B6   P7  B8   B9  I10                   
  Transmit Order I P4 Be B3 P7 B5 B6 I10 B8 B9                            
______________________________________
However, in order to reduce the number of frames that must be stored in memory by the temporal decoder, the frames are transmitted in a different order. It is useful to begin the analysis from an intra frame (I frame). The I frame is transmitted in the order it is to be displayed. The next predicted frame (P frame), P4 is then transmitted. Then, any bidirectionally interpolated frames (B frames) to be displayed between the I frame and P4 frame are transmitted, represented by Be and B3. This allows the transmitted B frames to reference a previous frame (forward prediction) or a future frame (backward prediction).
After transmitting all the B frames to be displayed between I and P4, the P7 frame is transmitted. Next, all the B frames to be displayed between the P4 and P7 frames are transmitted, i.e., corresponding to B5 and B6. Then, the next I frame, 110, is transmitted. Finally, all the B frames to be displayed between the P7 and 110 frames are transmitted, corresponding to B8 and B9. This ordering of transmitted frames requires only 2 frames to be kept in memory by the temporal decoder at any one time, and does not require the decoder to wait for the transmission of the next P frame or I frame to display an interjecting B frame. As described above and shown in Table 14, the temporal decoder of the present invention can be configured to provide MPEG picture reordering. With this picture reordering, the output of P and I pictures is delayed until the next P or I picture in the data stream starts to be decoded by the temporal decoder.
As the P and I pictures are reordered, certain tokens, i.e. Picture-- Start, Picture-- Type, and Temporal-- Reference, are stored temporarily on the chip as the picture is written into the picture buffers. When the picture is read out for display, these stored tokens are retrieved. At the output of the temporal decoder, the DATA tokens of the newly decoded P or I picture are replaced with DATA tokens for the older P or I picture, and they are then sent to the video formatter. Note that the output from the temporal decoder is in tokenized macroblock format and there is no block-to-raster conversion.
In brief, the video formatter of the present invention stores two framestores or pictures. In some video formatters three pictures or framestores are used to accommodate such features as repeating or skipping pictures. The video formatter's off-chip DRAM holds three framestores. The use of three framestores here allows frames to be either repeated or skipped in situations where the frame rates of the decoded video and the display are different.
All I, B and P frames are stored in the framestores of the video formatter. At any one time, there may be one frame store from which data is being displayed, one frame store into which data is being written, and in video formatters with three framestores, one other frame may be being stored in the third frame store.
The present embodiment performs all the prediction, reordering and block-to-raster tasks MPEG normally handles by using a temporal decoder with two framestores and a video formatter with two framestores, i.e., for a total of four framestores. This is accomplished in the present invention by using a frame store sharing scheme that only uses three framestores. The present embodiment cannot, however, handle the repeat and skip frame tasks of a video formatter with only the three framestores.
The present invention stores I pictures in a first frame store and P pictures in a second frame store. Because of the need to perform the block-to-raster conversion, B frames are stored in the manner detailed below in a third frame store. In order to minimize the amount of external DRAM required, a scheme is used where successive B frames share the same third frame store.
When a B frame is decoded, it may refer to the two previously decoded I or P frames occupying the first and second framestores. The decoded B frame is written into the third frame store. The present embodiment allows the raster to commence prior to a frame store being completely filled. The raster is allowed to start before the frame store is filled so that the next B frame can be written into the same frame store to occupy the space vacated by the raster at the top of the previous frame.
In order to keep a record of which parts of the frame store are occupied with picture data, and which are available for new data, each frame store is split into sectors. In the present invention, each frame store is first split into two field stores, each of which comprises N sectors, where N is the number of block rows in the field.
Frames coded as field pictures are straightforward. Each successive macroblock row occupies two sectors in a field store. Once the write back has progressed far enough down the frame, the raster starts reading out each sector from the top. Once the write back of the first frame has been completed, the start of the next frame is written into the space left by the raster. Checks on the status of each of the sectors ensures that the sector to be rastered is indeed full, and that for write back, the two sectors required are empty.
Frames coded as frame pictures are more difficult. Unlike field pictures, the macroblock rows of data are not written to the DRAM in the same order as they are to be rastered. The field stores are written to in parallel, whereas the fields are rastered in turn.
Consider a picture with 8 sectors per field store. That is, Field store 0 consists of 8 sectors numbered 0 to 7, each of which contains one row of blocks (i.e., each 8 pixels deep by the width of the picture). Field store 1 consists of 8 sectors, numbered 8 to 15, each of which contains one row of blocks (i.e., each 8 pixels deep by the width of the picture).
The first macroblock row is written back into sector 0 in field store 0 and to sector 8 in field store 1. The field stores continue to be filed in parallel. At some point, the raster beings displaying sectors from field store 0, that point being chosen so that the raster of field store does not catch up with the write back. However, the second frame cannot be written back in the same manner as the first. Because the sectors are written and read in a different order, waiting for the same two sectors to be free at the start of a frame would mean that write and read could not run continuously. This must be achieved in order to maintain the display and to maintain decoding at the necessary rate.
Accordingly, the second frame must be written into sectors of the frame store already freed by the raster. This is implemented by dividing the framestores in two. Hence, for the second frame, the meanings of the half field stores change. Sectors 4-7 become the upper part of the second field store and sectors 8-11 become the lower part of the first field store, i.e., they swap over. The first macroblock row is written to sectors 0 and 4, once they are freed, with subsequent rows written to 1 and 5, then 2 and 6, and then 3 and 7. The next row is written to sectors 8 and 12, and so on through to 11 and 15. This reallocation to the memory is sufficient to allow the write back and raster to continue at the appropriate rate.
Should a third successive B frame arrive, the write back order reverts to that of the first frame.
In the shared B frame store, with FRAME pictures:
The FIRST picture is written back to--Sectors 0 and 8 [1st macroblock row=2 block rows] Then 1 and 9, 2 and 10, 3 and 11, 7 and 15.
The FIRST picture is rastered from--Sector 0, Then 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15.
The SECOND frame is written to-- Sectors 0 and 4, Then 1 and 5, 2 and 6, 3 and 7, 8 and 12, 9 and 13, 10 and 14, 11 and 15.
The SECOND frame is rastered from--Sector 0, Then 1, 2, 3, 8, 9, 10, 11, 4, 5, 6, 7, 12, 13, 14, 15.
Note that, in accordance with the present invention, the second frame, the first macroblock row is not written into sectors 0 and 1, which are, after all, the first two sectors to be freed by the raster. Instead, it waits for sector 4 to clear. This is done for two reasons: First, waiting for sector 4 to clear does not affect the system's ability to maintain continuous decoding and display, even in the situation of worst-case coded data, and it is easier to implement. Secondly, with picture sizes which divide into a number of sectors that are not a power of two, the sequence for writing to and reading from sectors of memory does not repeat often (for example, NTSC format has 30 sectors per field and the sequence would repeat every 58 frames). This makes testability and recovery difficult.
As far as implementation is concerned in the present invention, rather than keep a record of the status of each individual sector, each half field store is effectively implemented as a fifo, with pointers to the next location to be written and to be read. Thus, each fifo being full or empty causes write back and raster, respectively to be disabled. This makes use of the knowledge that each half field store is itself written and read only one way, just like a fifo.
In summary, the present invention, provides method for storing video information by providing video information in the form of an I Frame, a P Frame, a B1 Frame and a B2 Frame, storing the I Frame in a first Frame store, storing the P frame in a second frame store; providing a third Frame store having a first and second field store, the first and second field store being split into at least two memory areas respectively, storing the B1 Frame in the third register, reading the B1 Frame from a selected portion of the memory area in the first or second field store; writing a portion of the B2 Frame into the selected portion of the memory area from which the B1 Frame was read; whereby a reduced amount of memory can be used to store video information.
The two programs found herein below contain code to be used in the preferred embodiment of the invention.
Detailed Description of the Invention for a Parallel Huffman Decoder
In accordance with the present invention, the Parallel Huffman Decoder block will decode Huffman coded Variable Length Codes (VLCs) and Fixed Length Codes (FLCs), and pass through tokens under the control of the parser microprogrammable state machine (MSM).
This embodiment of the present invention handles both MPEG-2 as well as MPEG-1 Huffman codes. An important aspect of this embodiment of the invention is that it can sustain a high through-put due to the fact that it is a parallel decoder rather than a serial one.
This embodiment of the present invention uses a code lookup technique to decode Huffman codes. This is done to achieve the performance requirements and also to handle the second MPEG-2 transform coefficient table which is irregular or non-canonical in nature.
Furthermore, this embodiment of the invention has some features that allow it to decode certain more complex components from the stream in a single cycle without the assistance of an external controller. Examples of such complex components are Escape-coded coefficients, Intra-DC values and Motion Vector deltas, all of which are present in the stream as combined VLC/FLC components.
Referring now to FIG. 52, there is shown how the Parallel Huffman Decoder 300 deals with variable length codes (VLCs). FLCs require a bypass mechanism which uses the selector 301 output to generate data and an input field to specify the length of the FLC. Thus, the ROM 302 is not required at all during FLC decoding.
However, to decode a VLC, input is first loaded into the two input data registers, `MSReg` and `LSReg` as shown in FIG. 52. As the names imply, the "earlier" or most significant data is stored in MSReg. The selector is used to align the beginning of the next VLC with the ROM input. Thus, to decode the very first VLC, the selector outputs the top 28 bits of its 59-bit input and the top 16 bits of these are passed to the Huffman Code ROM 302. For subsequent VLCs, the selector effectively shifts the input according to the total count of bits decoded thus far. The count is maintained by adding the size of each VLC, as it is decoded, to a running total. The various word widths are a result of the maximum coded size which can be decoded, which is the 28-bit MPEG-1 Escape Coded Coefficient, and the maximum VLC size which is 16 bits (DCT coefficient tables).
The "table select" input is used to select between the various different Huffman code tables required by MPEG.
The Huffman Code ROM
The core of the implementation of the present invention, used to decode all the VLCs is a special ROM 302 whose addresses are controlled with a selector/shifter 301 as shown in FIGS. 52 and 53. The ROM 302 has the job of performing a VLC table index calculation, followed by the index-to-data operation that yields decoded data.
The index calculation can be thought of as a content addressable memory (CAM) operation with "don't care" matching implemented to handle the Huffman codes which form the presented data. Since all the VLC code tables are fixed, a CAM-ROM will suffice and this is the job of the ROM AND-plane shown in FIGS. 54 through 57. Since the index generation is performed in a look-up manner (rather than algorithmically) there is no restriction to handling tables which are canonical.
The ROM Or-plane converts the "index" (an activated word-line) into the decoded data and the size (or length) of the code. The data forms the decoded output (subject to error checking) and the size information is fed back to allow a calculation to be performed which controls the selector and, thus, presents the decoder ROM 302 with the correct data to perform the decoding of the next VLC in the subsequent cycle.
The ROM 302 address of the present invention is in two fields. The larger field is the bit-pattern to be decoded and the smaller field selects which Huffman code table is to be examined. The bit-pattern which must be examined is quite long, 16 bits, corresponding to the longest VLC code and there is an additional 4 bits of table select. Thus, there is a total address space of 20 bits (approximately one million addresses) although there are only in 450 entries in the ROM 302. The reason for the difference is due to the existence of "don't care" bits.
In order to decode VLCs, the AND-plane must be able to decode "don't care" bits in the VLC bit-pattern. This is because all VLCs which are shorter than the maximum 18 bits will be followed by additional bits which form no part of the decoding of that VLC. Because of the wide address, the AND-plane is predecoded (2→4), and the ROM 302 must combine "don't care" handling with this predecode. Furthermore, in addition to the complete MPEG code tables, the ROM 302 also has entries to identify illegal VLC patterns, which exist for some code tables.
Maximizing Throughput
In order to sustain output of one decoded item every cycle, some care must be taken to control the decoder input and special handling must be used for some "complex" symbols (i.e., ones which are not single FLCs or VLCs).
In order to sustain peak throughput of Escape-coded coefficients it must be possible to input at least one complete code per cycle. Since the maximum length required is 28 bits in MPEG-1 this dictates the input word width of 32 bits (being the next sensible size greater than 28).
Normal transform coefficients are also "complex" symbols, in the sense that they consist of a VLC followed by a 1-bit FLC which gives the sign of the level value and are handled in a similar manner to the other complex symbols (e.g. motion Vectors, Intra DC and Escape coded coefficients). Peak throughput cannot be achieved if coefficients are decoded as a VLC followed by an FLC (in separate cycles) and the alternative of allowing the ROM 302 to decode the sign bit would double the size of the two largest tables in the ROM. Thus in the present invention, special handling is used for various symbols so that a single cycle can produce the "final" required result.
FLCs and Tokens
The basis of FLC handling is to control the selector with the required length of the FLC and to bypass the ROM 302 and simply output the correctly selected FLC. Thus, simple FLCs are handled fairly naturally by the decoder, without significant extra hardware. Furthermore, tokens are not manipulated, but simply passed directly to the output of the decoder.
Implementation
This section describes several important features of the implementation of the decoder, in accordance with the present invention. The implementation includes the arrangement of registers with the counter 303 and selector 301, as shown in FIG. 52, and the actual code ROM.
The schematic of FIG. 53 shows how the core components are interconnected to implement the main Huffman decoding core section of the present invention. The registers ms[31:0] and ls[31:0] are MSReg and LSReg, respectively, and the block phselect is the selector. The counter logic is contained in the block phcclog (together with various other logic) and the count latch is called cntl[4:0]. The other logic on this schematic deals with handling commands, data and command dynamics, tokens, and the manipulation of the more "complex" symbols (performed in block phcop).
The schematic shown in FIG. 54 illustrates a very small sample ROM design of the type used to implement the Huffman code ROM 302 in accordance with the present invention. The unusual features of this ROM 302 lie in the AND-plane where predecode and "don't care" handling are used to implement a method of decoding variable length Huffman codes.
Referring now to FIGS. 55, 56 and 57 and, more particularly to FIG. 55, there is shown a first embodiment of a ROM AND-plane capable of "don't care" handling. In this embodiment, each address line (a[3], a[2], a[1] and a[0]) is driven across the AND-plane in both its true and inverted directions. To decode a "one" or a "zero" on a given address line, a transistor is connected to either the true or inverted address line in the conventional manner. In order to decode a "don't care" (denoted by x) a transistor is not connected to either the true or the inverted line.
FIGS. 56 and 57 show alternative embodiments that utilize pre-decoding to reduce worst-case number of series transistors in the decoding logic. In these examples, two address bits are combined together in predecoding such that one of four lines is driven high for each of the four possible numbers that can be represented with the two address bits. It will be appreciated by one of ordinary skill in the art that the present invention would work equally well with higher levels of predecoding in which more than two bits are combined together. If the two address bits that are grouped together in the predecoding have defined values (either 1 or zero, but not the "don't care") then a transistor is connected to the appropriate predecoded address line in the conventional manner. Similarly, if both of the address bits have a "don't care", then no transistor is used as before. However, if one of the address bits needs to have a defined value (1 or zero) whilst the other address bit requires "don't care", then the decoding requires that the wordline driven across the Or-plane be selected when either of two of the predecoded address lines is active. In the embodiment shown in FIG. 56, this is achieved by placing two transistors, one on each of the relevant predecoded address lines, in parallel as shown in the case for the code; 001x. In the embodiment shown in FIG. 57 the required decoding is achieved without using a parallel connection of transistors. In this case, two separate decodes are performed both of which must be selected. They are combined together using a NOR gate in the wordline driver such that the wordline is only activated if both of the selects are active.
The foregoing description is believed to adequately describe the overall concepts, system implementation and operation of the various aspects of the invention in sufficient detail to enable one of ordinary skill in the art to make and practice the invention with all of its attendant features, objects and advantages. However, in order to facilitate a further, more detailed in depth understanding of the invention, and additional details in connection with even more specific, commercial implementation of various embodiments of the invention, the following further description and explanation is proffered. ##SPC1##
Note that additional Figures, which are self explanatory to those of ordinary skill in the art, are included with this application for providing further insight into the detailed structure and operation of the environment in which the present invention is intended to function.
The aforedescribed pipeline system of the present invention satisfies a long existing need for further improvements in various aspects of video decoding systems, including an MPEG video decompression method and apparatus utilizing a plurality of stages interconnected by a two-wire interface arranged as a pipeline processing machine. Control tokens and DATA Tokens pass over the single two-wire interface for carrying both control and data in token format. A token decode circuit is positioned in certain of the stages for recognizing certain of the tokens as control tokens pertinent to that stage and for passing unrecognized control tokens along the pipeline. Reconfiguration processing circuits are positioned in selected stages and are responsive to a recognized control token for reconfiguring such stage to handle an identified DATA Token. A wide variety of unique supporting subsystem circuitry and processing techniques are disclosed for implementing the system, including memory addressing, transforming data using a common processing block, time synchronization, asynchronous swing buffering, storing of video information, a parallel Huffman decoder, and the like.
It will be apparent from the foregoing that, while particular forms of the invention have been illustrated and described, various modifications can be made without departing from the spirit and scope of the invention. Accordingly, it is not intended that the invention be limited, except as by the appended claims.

Claims (6)

We claim:
1. A method of asynchronously accessing cells in a memory, comprising the steps of:
providing a random access memory having storage locations arranged in a plurality of rows and a plurality of columns;
providing wordlines along said rows, and connecting said wordlines to said storage locations, each said wordline being connected to a read row decoder and to a write row decoder;
providing bitlines along said columns and connecting said bitlines to said storage locations;
assigning a row address to a first said row;
assigning said row address to a second said row;
providing a read address to said read row decoder, said read address encoding said row address;
providing a write address to said write row decoder, said write address encoding said row address;
decoding said read address;
selecting one of said first row and said second row for reading to define a first selected row;
decoding said write address;
asynchronous with said step of selecting one of said first row and said second row for reading, selecting one of said first row and said second row for writing to define a second selected row;
reading data from a first storage location of said first selected row;
asynchronous with said step of reading data, writing data into a second storage location of said second selected row; and
signaling to identify said first selected row and signaling to identify said second selected row to coordinate said steps of reading data and writing data so that when one of said steps of reading and writing data is being performed, another of said steps of reading and writing data is deferred until completion of said one step.
2. The method as recited in claim 1 wherein said first selected row and said second selected row are different.
3. The method as recited in claim 1, wherein the storage locations are commonly accessed by a write decoder and a read decoder and are accessed by at least one of a first group of bit lines and a second group of bit lines, said first group of bit lines and said second group of bit lines being respectively selected for access by a write buffer line and a read buffer line.
4. The method as recited in claim 1 wherein the cells are arranged into first and second groups which in an interval of operation are selected for exclusive access by said read row decoder and said write row decoder respectively, the first group of cells being accessed exclusively by a first group of bit lines.
5. The method as recited in claim 1 wherein said step of signaling to identify said first selected row is performed using a first control line, and said step of signaling to identify said second selected row is performed using a second control line, and said steps of reading data and writing data are performed responsive to a first signal on said first control line and second signal on said second control line.
6. The method as recited in claim 5 wherein said steps of reading data and writing data are coordinated by read control circuitry and write control circuitry that are interconnected by said first control line and said second control line, said read control circuitry generating said first signal, and said write control circuitry generating said second signal.
US09/272,521 1993-06-24 1999-03-18 Method of asynchronous memory access Expired - Lifetime US6141721A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US09/272,521 US6141721A (en) 1993-06-24 1999-03-18 Method of asynchronous memory access

Applications Claiming Priority (12)

Application Number Priority Date Filing Date Title
US8229193A 1993-06-24 1993-06-24
GB9415413A GB9415413D0 (en) 1994-07-29 1994-07-29 Method and apparatus for video decompression
GB9415413 1994-07-29
US38295895A 1995-02-02 1995-02-02
US40039795A 1995-03-07 1995-03-07
US08/400,201 US5603012A (en) 1992-06-30 1995-03-07 Start code detector
US47572995A 1995-06-07 1995-06-07
GB9511569A GB2293076B (en) 1994-07-29 1995-06-07 Time synchronisation in a multiplexed data stream
US08/473,813 US5821885A (en) 1994-07-29 1995-06-07 Video decompression
GB9511569 1995-06-07
US08/991,234 US6799246B1 (en) 1993-06-24 1997-12-16 Memory interface for reading/writing data from/to a memory
US09/272,521 US6141721A (en) 1993-06-24 1999-03-18 Method of asynchronous memory access

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
US08/400,201 Continuation-In-Part US5603012A (en) 1992-06-30 1995-03-07 Start code detector
US08/991,234 Division US6799246B1 (en) 1993-06-24 1997-12-16 Memory interface for reading/writing data from/to a memory

Publications (1)

Publication Number Publication Date
US6141721A true US6141721A (en) 2000-10-31

Family

ID=26305370

Family Applications (16)

Application Number Title Priority Date Filing Date
US08/482,381 Expired - Lifetime US5828907A (en) 1992-06-30 1995-06-07 Token-based adaptive video processing arrangement
US08/476,814 Expired - Lifetime US5798719A (en) 1994-07-29 1995-06-07 Parallel Huffman decoder
US08/488,348 Expired - Lifetime US5984512A (en) 1994-07-29 1995-06-07 Method for storing video information
US08/486,908 Expired - Lifetime US5829007A (en) 1993-06-24 1995-06-07 Technique for implementing a swing buffer in a memory array
US08/484,578 Expired - Lifetime US5878273A (en) 1993-06-24 1995-06-07 System for microprogrammable state machine in video parser disabling portion of processing stages responsive to sequence-- end token generating by token generator responsive to received data
US08/473,813 Expired - Lifetime US5821885A (en) 1992-06-30 1995-06-07 Video decompression
US08/481,785 Expired - Lifetime US5703793A (en) 1994-07-29 1995-06-07 Video decompression
US08/479,910 Expired - Lifetime US5768629A (en) 1993-06-24 1995-06-07 Token-based adaptive video processing arrangement
US08/484,170 Expired - Lifetime US5963154A (en) 1994-07-29 1995-06-07 Technique for decoding variable and fixed length codes
US08/487,134 Expired - Lifetime US5835792A (en) 1993-06-24 1995-06-07 Token-based adaptive video processing arrangement
US08/481,561 Expired - Lifetime US5801973A (en) 1994-07-29 1995-06-07 Video decompression
US08/487,356 Expired - Lifetime US6217234B1 (en) 1994-07-29 1995-06-07 Apparatus and method for processing data with an arithmetic unit
US08/481,772 Expired - Lifetime US5740460A (en) 1994-07-29 1995-06-07 Arrangement for processing packetized data
US08/947,677 Expired - Lifetime US5995727A (en) 1994-07-29 1997-10-07 Video decompression
US08/991,234 Expired - Fee Related US6799246B1 (en) 1993-06-24 1997-12-16 Memory interface for reading/writing data from/to a memory
US09/272,521 Expired - Lifetime US6141721A (en) 1993-06-24 1999-03-18 Method of asynchronous memory access

Family Applications Before (15)

Application Number Title Priority Date Filing Date
US08/482,381 Expired - Lifetime US5828907A (en) 1992-06-30 1995-06-07 Token-based adaptive video processing arrangement
US08/476,814 Expired - Lifetime US5798719A (en) 1994-07-29 1995-06-07 Parallel Huffman decoder
US08/488,348 Expired - Lifetime US5984512A (en) 1994-07-29 1995-06-07 Method for storing video information
US08/486,908 Expired - Lifetime US5829007A (en) 1993-06-24 1995-06-07 Technique for implementing a swing buffer in a memory array
US08/484,578 Expired - Lifetime US5878273A (en) 1993-06-24 1995-06-07 System for microprogrammable state machine in video parser disabling portion of processing stages responsive to sequence-- end token generating by token generator responsive to received data
US08/473,813 Expired - Lifetime US5821885A (en) 1992-06-30 1995-06-07 Video decompression
US08/481,785 Expired - Lifetime US5703793A (en) 1994-07-29 1995-06-07 Video decompression
US08/479,910 Expired - Lifetime US5768629A (en) 1993-06-24 1995-06-07 Token-based adaptive video processing arrangement
US08/484,170 Expired - Lifetime US5963154A (en) 1994-07-29 1995-06-07 Technique for decoding variable and fixed length codes
US08/487,134 Expired - Lifetime US5835792A (en) 1993-06-24 1995-06-07 Token-based adaptive video processing arrangement
US08/481,561 Expired - Lifetime US5801973A (en) 1994-07-29 1995-06-07 Video decompression
US08/487,356 Expired - Lifetime US6217234B1 (en) 1994-07-29 1995-06-07 Apparatus and method for processing data with an arithmetic unit
US08/481,772 Expired - Lifetime US5740460A (en) 1994-07-29 1995-06-07 Arrangement for processing packetized data
US08/947,677 Expired - Lifetime US5995727A (en) 1994-07-29 1997-10-07 Video decompression
US08/991,234 Expired - Fee Related US6799246B1 (en) 1993-06-24 1997-12-16 Memory interface for reading/writing data from/to a memory

Country Status (8)

Country Link
US (16) US5828907A (en)
EP (8) EP0695095A3 (en)
JP (3) JPH08172624A (en)
CN (1) CN1144434A (en)
AU (3) AU701335C (en)
CA (1) CA2154962A1 (en)
MX (1) MXPA99001886A (en)
SG (1) SG108204A1 (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030044075A1 (en) * 2001-08-30 2003-03-06 Jae-Woo Roh Block coding/decoding method and apparatus for increasing code rate
US20030218921A1 (en) * 2002-02-28 2003-11-27 Peter Schrogmeier Latency time switch for an S-DRAM
US20040085817A1 (en) * 2002-10-31 2004-05-06 Youngwoo Kim First-in first-out memory circuit and method for executing same
US20050071582A1 (en) * 2003-09-26 2005-03-31 Samsung Electronics Co., Ltd. Circuits and methods for providing variable data I/O width for semiconductor memory devices
US20070190686A1 (en) * 2006-02-13 2007-08-16 Advanced Semiconductor Engineering, Inc. Method of fabricating substrate with embedded component therein
US7385949B1 (en) 2001-06-05 2008-06-10 Broadcom Corporation System and method for de-interleaving data in a wireless receiver
US7463544B1 (en) * 2001-10-15 2008-12-09 Altera Corporation Device programmable to operate as a multiplexer, demultiplexer, or memory device
US20090214175A1 (en) * 2003-07-03 2009-08-27 Mccrossan Joseph Recording medium, reproduction apparatus, recording method, integrated circuit, program, and reproduction method

Families Citing this family (357)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5828907A (en) * 1992-06-30 1998-10-27 Discovision Associates Token-based adaptive video processing arrangement
US9286294B2 (en) 1992-12-09 2016-03-15 Comcast Ip Holdings I, Llc Video and digital multimedia aggregator content suggestion engine
US7168084B1 (en) 1992-12-09 2007-01-23 Sedna Patent Services, Llc Method and apparatus for targeting virtual objects
CA2145365C (en) 1994-03-24 1999-04-27 Anthony M. Jones Method for accessing banks of dram
DE19524688C1 (en) * 1995-07-06 1997-01-23 Siemens Ag Method for decoding and encoding a compressed video data stream with reduced memory requirements
JPH09168150A (en) * 1995-10-09 1997-06-24 Fujitsu Ltd Fixed length cell processing type image communication method, fixed length cell processing type image communication transmitter and fixed length cell processing type image communication receiver
US7266725B2 (en) 2001-09-03 2007-09-04 Pact Xpp Technologies Ag Method for debugging reconfigurable architectures
JP3063859B2 (en) * 1996-01-08 2000-07-12 インターナシヨナル・ビジネス・マシーンズ・コーポレーシヨン Method and file server for delivering multimedia files
US6678311B2 (en) 1996-05-28 2004-01-13 Qualcomm Incorporated High data CDMA wireless communication system using variable sized channel codes
JP2888288B2 (en) * 1996-10-03 1999-05-10 日本電気株式会社 Image coding device
EP0837474B1 (en) * 1996-10-17 2005-02-02 STMicroelectronics S.r.l. Method for optimising a memory cell matrix for a semiconductor integrated microcontroller
US5898897A (en) * 1996-10-18 1999-04-27 Samsung Electronics Company, Ltd. Bit stream signal feature detection in a signal processing system
US5889515A (en) * 1996-12-09 1999-03-30 Stmicroelectronics, Inc. Rendering an audio-visual stream synchronized by a software clock in a personal computer
DE19654595A1 (en) 1996-12-20 1998-07-02 Pact Inf Tech Gmbh I0 and memory bus system for DFPs as well as building blocks with two- or multi-dimensional programmable cell structures
US6542998B1 (en) 1997-02-08 2003-04-01 Pact Gmbh Method of self-synchronization of configurable elements of a programmable module
JPH10257488A (en) 1997-03-12 1998-09-25 Oki Data:Kk Image coder and image decoder
JPH10290464A (en) * 1997-04-14 1998-10-27 Kokusai Denshin Denwa Co Ltd <Kdd> Coding mode discrimination device
US5883907A (en) * 1997-05-01 1999-03-16 Motorola, Inc. Asymmetrical digital subscriber line (ADSL) block encoder circuit and method of operation
US6101195A (en) * 1997-05-28 2000-08-08 Sarnoff Corporation Timing correction method and apparatus
US6421812B1 (en) * 1997-06-10 2002-07-16 Altera Corporation Programming mode selection with JTAG circuits
US8686549B2 (en) * 2001-09-03 2014-04-01 Martin Vorbach Reconfigurable elements
DE19861088A1 (en) 1997-12-22 2000-02-10 Pact Inf Tech Gmbh Repairing integrated circuits by replacing subassemblies with substitutes
US6363511B1 (en) 1998-03-31 2002-03-26 Stmicroelectronics N.V. Device and method for decoding data streams from storage media
US6260169B1 (en) * 1998-03-31 2001-07-10 Stmicroelectronics N.V. Device and method for real time correction of row data from DVD media
US6904174B1 (en) * 1998-12-11 2005-06-07 Intel Corporation Simplified predictive video encoder
US7263127B1 (en) 1998-04-02 2007-08-28 Intel Corporation Method and apparatus for simplifying frame-based motion estimation
US7046734B2 (en) * 1998-04-02 2006-05-16 Intel Corporation Method and apparatus for performing real-time data encoding
US6279156B1 (en) * 1999-01-26 2001-08-21 Dell Usa, L.P. Method of installing software on and/or testing a computer system
US6370273B1 (en) * 1998-04-10 2002-04-09 Flashpoint Technology, Inc. Method and system for tiled image data decompression
JP3965539B2 (en) * 1998-04-23 2007-08-29 ソニー株式会社 Digital image decoding apparatus and method, and recording medium
KR100268480B1 (en) * 1998-06-05 2000-10-16 윤종용 Display system having a formatconverter module
US6408002B1 (en) * 1998-06-24 2002-06-18 Fujitsu Siemens Computers Llc Torus routing element error handling and self-clearing with missing or extraneous control code feature
US6754905B2 (en) 1998-07-23 2004-06-22 Diva Systems Corporation Data structure and methods for providing an interactive program guide
BR9912385A (en) 1998-07-23 2002-01-15 Diva Systems Corp User interface generated on a head end, interactive program guide, and processes for generating and distributing a user interface, and interacting with an interactive program guide
US9924234B2 (en) 1998-07-23 2018-03-20 Comcast Ip Holdings I, Llc Data structure and methods for providing an interactive program
HUP0105014A3 (en) * 1998-08-27 2002-11-28 Ibm System for embedding additional information in video data, and embedding method
US6952823B2 (en) * 1998-09-01 2005-10-04 Pkware, Inc. Software patch generator using compression techniques
US6141743A (en) * 1998-09-17 2000-10-31 Advanced Micro Devices, Inc. Token-based storage for general purpose processing
US6192148B1 (en) * 1998-11-05 2001-02-20 Winbond Electronics Corp. Method for determining to skip macroblocks in encoding video
US6671708B1 (en) * 1998-11-26 2003-12-30 Matsushita Electric Industrial Co., Ltd. Processor and image processing device
EP1053619B1 (en) * 1998-12-02 2007-08-22 Nxp B.V. System and method for generating a real-time signal
US6624761B2 (en) 1998-12-11 2003-09-23 Realtime Data, Llc Content independent data compression method and system
US7051309B1 (en) * 1999-02-16 2006-05-23 Crosetto Dario B Implementation of fast data processing with mixed-signal and purely digital 3D-flow processing boars
US6604158B1 (en) 1999-03-11 2003-08-05 Realtime Data, Llc System and methods for accelerated data storage and retrieval
US6601104B1 (en) 1999-03-11 2003-07-29 Realtime Data Llc System and methods for accelerated data storage and retrieval
US6754271B1 (en) 1999-04-15 2004-06-22 Diva Systems Corporation Temporal slice persistence method and apparatus for delivery of interactive program guide
US7096487B1 (en) 1999-10-27 2006-08-22 Sedna Patent Services, Llc Apparatus and method for combining realtime and non-realtime encoded content
US6904610B1 (en) 1999-04-15 2005-06-07 Sedna Patent Services, Llc Server-centric customized interactive program guide in an interactive television environment
US8230411B1 (en) 1999-06-10 2012-07-24 Martin Vorbach Method for interleaving a program over a plurality of cells
FI110743B (en) * 1999-06-28 2003-03-14 Valtion Teknillinen Method and system for performing motion estimation
JP4618960B2 (en) * 1999-09-21 2011-01-26 エヌエックスピー ビー ヴィ Clock recovery
WO2001031914A1 (en) 1999-10-27 2001-05-03 Diva Systems Corporation Picture-in-picture and multiple video streams using slice-based encoding
US6411301B1 (en) 1999-10-28 2002-06-25 Nintendo Co., Ltd. Graphics system interface
US6618048B1 (en) 1999-10-28 2003-09-09 Nintendo Co., Ltd. 3D graphics rendering system for performing Z value clamping in near-Z range to maximize scene resolution of visually important Z components
US6452600B1 (en) 1999-10-28 2002-09-17 Nintendo Co., Ltd. Graphics system interface
US6717577B1 (en) 1999-10-28 2004-04-06 Nintendo Co., Ltd. Vertex cache for 3D computer graphics
US6318156B1 (en) * 1999-10-28 2001-11-20 Micro Motion, Inc. Multiphase flow measurement system
FR2801463B1 (en) * 1999-11-23 2002-04-12 St Microelectronics Sa METHOD AND SYSTEM FOR PROCESSING DIGITAL IMAGES
US6498571B2 (en) 1999-12-09 2002-12-24 Luxxon Corporation Multiple stream variable length encoder and decoder
CA2395498C (en) * 1999-12-24 2013-08-27 Telstra New Wave Pty Ltd A virtual token
US20010047473A1 (en) 2000-02-03 2001-11-29 Realtime Data, Llc Systems and methods for computer initialization
EP1177692A1 (en) * 2000-02-23 2002-02-06 Koninklijke Philips Electronics N.V. Method, transmitter and transmission system
US6879988B2 (en) 2000-03-09 2005-04-12 Pkware System and method for manipulating and managing computer archive files
US7844579B2 (en) * 2000-03-09 2010-11-30 Pkware, Inc. System and method for manipulating and managing computer archive files
US20060143249A1 (en) * 2000-03-09 2006-06-29 Pkware, Inc. System and method for manipulating and managing computer archive files
US20050015608A1 (en) * 2003-07-16 2005-01-20 Pkware, Inc. Method for strongly encrypting .ZIP files
US20060143253A1 (en) * 2000-03-09 2006-06-29 Pkware, Inc. System and method for manipulating and managing computer archive files
US20060143199A1 (en) * 2000-03-09 2006-06-29 Pkware, Inc. System and method for manipulating and managing computer archive files
US20060143180A1 (en) * 2000-03-09 2006-06-29 Pkware, Inc. System and method for manipulating and managing computer archive files
US8959582B2 (en) 2000-03-09 2015-02-17 Pkware, Inc. System and method for manipulating and managing computer archive files
US20060143237A1 (en) * 2000-03-09 2006-06-29 Pkware, Inc. System and method for manipulating and managing computer archive files
US20060155788A1 (en) * 2000-03-09 2006-07-13 Pkware, Inc. System and method for manipulating and managing computer archive files
US8230482B2 (en) 2000-03-09 2012-07-24 Pkware, Inc. System and method for manipulating and managing computer archive files
US20060173847A1 (en) * 2000-03-09 2006-08-03 Pkware, Inc. System and method for manipulating and managing computer archive files
FR2806574B1 (en) * 2000-03-15 2002-05-03 Thomson Multimedia Sa DEVICE FOR SYNCHRONIZING INTERACTIVE APPLICATIONS IN A TELEVISION RECEIVER
US6687384B1 (en) * 2000-03-27 2004-02-03 Sarnoff Corporation Method and apparatus for embedding data in encoded digital bitstreams
US6857061B1 (en) 2000-04-07 2005-02-15 Nintendo Co., Ltd. Method and apparatus for obtaining a scalar value directly from a vector register
US6859862B1 (en) 2000-04-07 2005-02-22 Nintendo Co., Ltd. Method and apparatus for software management of on-chip cache
WO2001086820A1 (en) * 2000-05-09 2001-11-15 Sony Corporation Data processing device and data processing method, and recorded medium
US7395209B1 (en) * 2000-05-12 2008-07-01 Cirrus Logic, Inc. Fixed point audio decoding system and method
US7119813B1 (en) 2000-06-02 2006-10-10 Nintendo Co., Ltd. Variable bit field encoding
US7083071B1 (en) 2000-06-08 2006-08-01 Beverage Works, Inc. Drink supply canister for beverage dispensing apparatus
US7754025B1 (en) 2000-06-08 2010-07-13 Beverage Works, Inc. Dishwasher having a door supply housing which holds dish washing supply for multiple wash cycles
US6799085B1 (en) * 2000-06-08 2004-09-28 Beverage Works, Inc. Appliance supply distribution, dispensing and use system method
US6751525B1 (en) 2000-06-08 2004-06-15 Beverage Works, Inc. Beverage distribution and dispensing system and method
EP2226732A3 (en) 2000-06-13 2016-04-06 PACT XPP Technologies AG Cache hierarchy for a multicore processor
US7538772B1 (en) 2000-08-23 2009-05-26 Nintendo Co., Ltd. Graphics processing system with enhanced memory controller
US6580430B1 (en) 2000-08-23 2003-06-17 Nintendo Co., Ltd. Method and apparatus for providing improved fog effects in a graphics system
US7134960B1 (en) * 2000-08-23 2006-11-14 Nintendo Co., Ltd. External interfaces for a 3D graphics system
US6609977B1 (en) 2000-08-23 2003-08-26 Nintendo Co., Ltd. External interfaces for a 3D graphics system
US7196710B1 (en) 2000-08-23 2007-03-27 Nintendo Co., Ltd. Method and apparatus for buffering graphics data in a graphics system
US6700586B1 (en) 2000-08-23 2004-03-02 Nintendo Co., Ltd. Low cost graphics with stitching processing hardware support for skeletal animation
US6999100B1 (en) 2000-08-23 2006-02-14 Nintendo Co., Ltd. Method and apparatus for anti-aliasing in a graphics system
US6707458B1 (en) 2000-08-23 2004-03-16 Nintendo Co., Ltd. Method and apparatus for texture tiling in a graphics system
US7576748B2 (en) 2000-11-28 2009-08-18 Nintendo Co. Ltd. Graphics system with embedded frame butter having reconfigurable pixel formats
US6867781B1 (en) * 2000-08-23 2005-03-15 Nintendo Co., Ltd. Graphics pipeline token synchronization
US6825851B1 (en) 2000-08-23 2004-11-30 Nintendo Co., Ltd. Method and apparatus for environment-mapped bump-mapping in a graphics system
US6636214B1 (en) 2000-08-23 2003-10-21 Nintendo Co., Ltd. Method and apparatus for dynamically reconfiguring the order of hidden surface processing based on rendering mode
US6639595B1 (en) 2000-08-23 2003-10-28 Nintendo Co., Ltd. Achromatic lighting in a graphics system and method
US6811489B1 (en) 2000-08-23 2004-11-02 Nintendo Co., Ltd. Controller interface for a graphics system
US6664962B1 (en) 2000-08-23 2003-12-16 Nintendo Co., Ltd. Shadow mapping in a low cost graphics system
US6606689B1 (en) 2000-08-23 2003-08-12 Nintendo Co., Ltd. Method and apparatus for pre-caching data in audio memory
US6980218B1 (en) 2000-08-23 2005-12-27 Nintendo Co., Ltd. Method and apparatus for efficient generation of texture coordinate displacements for implementing emboss-style bump mapping in a graphics rendering system
US6937245B1 (en) 2000-08-23 2005-08-30 Nintendo Co., Ltd. Graphics system with embedded frame buffer having reconfigurable pixel formats
US6664958B1 (en) 2000-08-23 2003-12-16 Nintendo Co., Ltd. Z-texturing
US8692695B2 (en) 2000-10-03 2014-04-08 Realtime Data, Llc Methods for encoding and decoding data
US7417568B2 (en) 2000-10-03 2008-08-26 Realtime Data Llc System and method for data feed acceleration and encryption
US9143546B2 (en) 2000-10-03 2015-09-22 Realtime Data Llc System and method for data feed acceleration and encryption
JP3489676B2 (en) * 2000-10-16 2004-01-26 日本電気株式会社 Image display device and driving method thereof
US6697074B2 (en) 2000-11-28 2004-02-24 Nintendo Co., Ltd. Graphics system interface
US20020073136A1 (en) * 2000-12-07 2002-06-13 Tomoaki Itoh Data reproduction method, data receiving terminal and data receiving method
US6879725B2 (en) * 2001-01-26 2005-04-12 International Business Machine Corporation Method, system, and program for decoding a section from compressed data
US7386046B2 (en) 2001-02-13 2008-06-10 Realtime Data Llc Bandwidth sensitive data compression and decompression
US7844796B2 (en) 2001-03-05 2010-11-30 Martin Vorbach Data processing device and method
US9037807B2 (en) 2001-03-05 2015-05-19 Pact Xpp Technologies Ag Processor arrangement on a chip including data processing, memory, and interface elements
US7444531B2 (en) 2001-03-05 2008-10-28 Pact Xpp Technologies Ag Methods and devices for treating and processing data
US6901422B1 (en) * 2001-03-21 2005-05-31 Apple Computer, Inc. Matrix multiplication in a vector processing system
US6922441B2 (en) * 2001-05-11 2005-07-26 International Business Machines Corporation Method for performing integer divides without propagation of truncation error
KR100410554B1 (en) * 2001-07-13 2003-12-18 삼성전자주식회사 method for outputting package map information in semiconductor memory device and circuit therefor
KR20030010233A (en) * 2001-07-26 2003-02-05 주식회사 글로네트 A television have multiple function
JP4042364B2 (en) * 2001-07-27 2008-02-06 日本電気株式会社 Address generation circuit, selection decision circuit
US20030021486A1 (en) * 2001-07-27 2003-01-30 Tinku Acharya Method and apparatus for image scaling
US7793326B2 (en) 2001-08-03 2010-09-07 Comcast Ip Holdings I, Llc Video and digital multimedia aggregator
US7908628B2 (en) 2001-08-03 2011-03-15 Comcast Ip Holdings I, Llc Video and digital multimedia aggregator content coding and formatting
US7996827B2 (en) 2001-08-16 2011-08-09 Martin Vorbach Method for the translation of programs for reconfigurable architectures
US7003588B1 (en) 2001-08-22 2006-02-21 Nintendo Co., Ltd. Peripheral devices for a video game system
US7434191B2 (en) 2001-09-03 2008-10-07 Pact Xpp Technologies Ag Router
EP1388792A1 (en) * 2001-09-05 2004-02-11 Matsushita Electric Industrial Co., Ltd. Synchronization message processing method
US8686475B2 (en) * 2001-09-19 2014-04-01 Pact Xpp Technologies Ag Reconfigurable elements
JP2003099250A (en) * 2001-09-20 2003-04-04 Oki Electric Ind Co Ltd Register readout circuit and microprocessor
ES2587022T3 (en) * 2001-11-22 2016-10-20 Godo Kaisha Ip Bridge 1 Variable length coding procedure and variable length decoding procedure
US20030101312A1 (en) * 2001-11-26 2003-05-29 Doan Trung T. Machine state storage apparatus and method
US7089541B2 (en) * 2001-11-30 2006-08-08 Sun Microsystems, Inc. Modular parser architecture with mini parsers
US6879523B1 (en) * 2001-12-27 2005-04-12 Cypress Semiconductor Corporation Random access memory (RAM) method of operation and device for search engine systems
US7301961B1 (en) 2001-12-27 2007-11-27 Cypress Semiconductor Corportion Method and apparatus for configuring signal lines according to idle codes
AU2003208266A1 (en) 2002-01-19 2003-07-30 Pact Xpp Technologies Ag Reconfigurable processor
US7113886B2 (en) * 2002-01-23 2006-09-26 Credence Systems Corporation Circuit and method for distributing events in an event stream
US6768684B2 (en) * 2002-01-25 2004-07-27 Sun Microsystems, Inc. System and method for small read only data
US20110161977A1 (en) * 2002-03-21 2011-06-30 Martin Vorbach Method and device for data processing
US8914590B2 (en) 2002-08-07 2014-12-16 Pact Xpp Technologies Ag Data processing method and device
US8284844B2 (en) 2002-04-01 2012-10-09 Broadcom Corporation Video decoding system supporting multiple standards
US8401084B2 (en) * 2002-04-01 2013-03-19 Broadcom Corporation System and method for multi-row decoding of video with dependent rows
JP3783645B2 (en) 2002-04-05 2006-06-07 株式会社日立製作所 Contrast adjustment method, contrast adjustment circuit, and video display device using the same
KR100495657B1 (en) * 2002-05-03 2005-06-16 삼성전자주식회사 Integrated circuit device with multiple communication modes and operating method thereof
US7073099B1 (en) * 2002-05-30 2006-07-04 Marvell International Ltd. Method and apparatus for improving memory operation and yield
US7133972B2 (en) 2002-06-07 2006-11-07 Micron Technology, Inc. Memory hub with internal cache and/or memory access prediction
FI116813B (en) * 2002-06-20 2006-02-28 Nokia Corp A method and system for decoding variable length coding, and a code word locator
CA2459070C (en) * 2002-06-28 2013-10-22 Lg Electronics Inc. Recording medium having data structure for managing reproduction of multiple playback path video data recorded thereon and recording and reproducing methods and apparatuses
KR100554767B1 (en) * 2002-06-28 2006-02-22 엘지전자 주식회사 Recording medium having data structure for managing recording and reproduction of multiple path data recorded thereon and recording and reproducing methods and apparatus
US7117316B2 (en) 2002-08-05 2006-10-03 Micron Technology, Inc. Memory hub and access method having internal row caching
AU2003286131A1 (en) 2002-08-07 2004-03-19 Pact Xpp Technologies Ag Method and device for processing data
US6820181B2 (en) 2002-08-29 2004-11-16 Micron Technology, Inc. Method and system for controlling memory accesses to memory modules having a memory hub architecture
US7836252B2 (en) 2002-08-29 2010-11-16 Micron Technology, Inc. System and method for optimizing interconnections of memory devices in a multichip module
US7394284B2 (en) 2002-09-06 2008-07-01 Pact Xpp Technologies Ag Reconfigurable sequencer structure
US7533402B2 (en) 2002-09-30 2009-05-12 Broadcom Corporation Satellite set-top box decoder for simultaneously servicing multiple independent programs for display on independent display device
US7230987B2 (en) * 2002-09-30 2007-06-12 Broadcom Corporation Multiple time-base clock for processing multiple satellite signals
US7336268B1 (en) * 2002-10-30 2008-02-26 National Semiconductor Corporation Point-to-point display system having configurable connections
WO2004049330A1 (en) * 2002-11-22 2004-06-10 Lg Electronics Inc. Recording medium having data structure for managing reproduction of multiple reproduction path video data recorded thereon and recording and reproducing methods and apparatuses
AU2003283636A1 (en) * 2002-12-04 2004-06-23 Koninklijke Philips Electronics N.V. Method and apparatus for selecting particular decoder based on bitstream format detection
KR100469278B1 (en) * 2002-12-26 2005-02-02 엘지전자 주식회사 Decoder Application Specific Integrated Circuit in Digital TV
JP3938054B2 (en) * 2003-01-17 2007-06-27 セイコーエプソン株式会社 Computer-readable storage medium on which data having an image data structure is recorded, image recording method, apparatus, and program
JP4140709B2 (en) * 2003-02-05 2008-08-27 松下電器産業株式会社 Image signal reproducing apparatus and image signal reproducing method
DE10304911B4 (en) * 2003-02-06 2014-10-09 Heinz Lindenmeier Combination antenna arrangement for multiple radio services for vehicles
TWI228932B (en) * 2003-02-11 2005-03-01 Ind Tech Res Inst Encoding and decoding method of recording medium
EP1593046A2 (en) * 2003-02-13 2005-11-09 Nokia Corporation Rate adaptation method and device in multimedia streaming
US7283591B2 (en) * 2003-03-28 2007-10-16 Tarari, Inc. Parallelized dynamic Huffman decoder
EP1611528A2 (en) * 2003-04-04 2006-01-04 PACT XPP Technologies AG Method and device for data processing
US7702878B2 (en) * 2004-03-19 2010-04-20 Broadcom Corporation Method and system for scalable video data width
WO2004107063A1 (en) * 2003-05-28 2004-12-09 Fujitsu Limited Time manager and time management method
US7756288B2 (en) * 2003-05-29 2010-07-13 Jeffrey Lubin Method and apparatus for analog insertion of low frequency watermarks
US7245145B2 (en) 2003-06-11 2007-07-17 Micron Technology, Inc. Memory module and method having improved signal routing topology
US7120727B2 (en) 2003-06-19 2006-10-10 Micron Technology, Inc. Reconfigurable memory module and method
US7260685B2 (en) 2003-06-20 2007-08-21 Micron Technology, Inc. Memory hub and access method having internal prefetch buffers
US20040264564A1 (en) * 2003-06-26 2004-12-30 Senger Michael D. System and method for efficiently using video encoding resources
US7389364B2 (en) 2003-07-22 2008-06-17 Micron Technology, Inc. Apparatus and method for direct memory access in a hub-based memory system
US8234395B2 (en) 2003-07-28 2012-07-31 Sonos, Inc. System and method for synchronizing operations among a plurality of independently clocked digital data processing devices
US11650784B2 (en) 2003-07-28 2023-05-16 Sonos, Inc. Adjusting volume levels
US10613817B2 (en) 2003-07-28 2020-04-07 Sonos, Inc. Method and apparatus for displaying a list of tracks scheduled for playback by a synchrony group
US11106425B2 (en) 2003-07-28 2021-08-31 Sonos, Inc. Synchronizing operations among a plurality of independently clocked digital data processing devices
US11294618B2 (en) 2003-07-28 2022-04-05 Sonos, Inc. Media player system
US8086752B2 (en) 2006-11-22 2011-12-27 Sonos, Inc. Systems and methods for synchronizing operations among a plurality of independently clocked digital data processing devices that independently source digital data
US11106424B2 (en) 2003-07-28 2021-08-31 Sonos, Inc. Synchronizing operations among a plurality of independently clocked digital data processing devices
US8290603B1 (en) 2004-06-05 2012-10-16 Sonos, Inc. User interfaces for controlling and manipulating groupings in a multi-zone media system
US7210059B2 (en) 2003-08-19 2007-04-24 Micron Technology, Inc. System and method for on-board diagnostics of memory modules
US7133991B2 (en) * 2003-08-20 2006-11-07 Micron Technology, Inc. Method and system for capturing and bypassing memory transactions in a hub-based memory system
US7136958B2 (en) 2003-08-28 2006-11-14 Micron Technology, Inc. Multiple processor system and method including multiple memory hub modules
US7091967B2 (en) * 2003-09-01 2006-08-15 Realtek Semiconductor Corp. Apparatus and method for image frame synchronization
US7310752B2 (en) 2003-09-12 2007-12-18 Micron Technology, Inc. System and method for on-board timing margin testing of memory modules
US7194593B2 (en) 2003-09-18 2007-03-20 Micron Technology, Inc. Memory hub with integrated non-volatile memory
US7120743B2 (en) 2003-10-20 2006-10-10 Micron Technology, Inc. Arbitration system and method for memory responses in a hub-based memory system
KR101102639B1 (en) * 2003-11-12 2012-01-04 파나소닉 주식회사 Recording medium, playback apparatus and method, recording method, and computer-readable medium
US6903668B1 (en) * 2003-11-18 2005-06-07 M-Systems Flash Disk Pioneers Ltd. Decompression accelerator for flash memory
US7330992B2 (en) 2003-12-29 2008-02-12 Micron Technology, Inc. System and method for read synchronization of memory modules
US6956511B2 (en) * 2004-01-06 2005-10-18 Sharp Laboratories Of America, Inc. Multi-symbol/coefficient decode operation for Huffman codes
US6988237B1 (en) 2004-01-06 2006-01-17 Marvell Semiconductor Israel Ltd. Error-correction memory architecture for testing production errors
KR100640885B1 (en) * 2004-01-27 2006-11-02 엘지전자 주식회사 Video buffer controlling apparatus for dual video decoding
US7188219B2 (en) 2004-01-30 2007-03-06 Micron Technology, Inc. Buffer control system and method for a memory system having outstanding read and write request buffers
US8023564B2 (en) * 2004-02-04 2011-09-20 Broadcom Corporaiton System and method for providing data starting from start codes aligned with byte boundaries in multiple byte words
US7788451B2 (en) 2004-02-05 2010-08-31 Micron Technology, Inc. Apparatus and method for data bypass for a bi-directional data bus in a hub-based memory sub-system
US20050175027A1 (en) * 2004-02-09 2005-08-11 Phonex Broadband Corporation System and method for requesting and granting access to a network channel
US7366864B2 (en) 2004-03-08 2008-04-29 Micron Technology, Inc. Memory hub architecture having programmable lane widths
US20080246643A1 (en) * 2004-03-15 2008-10-09 Mark Francis Rumreich Technique For Efficient Video Re-Sampling
US7257683B2 (en) 2004-03-24 2007-08-14 Micron Technology, Inc. Memory arbitration system and method having an arbitration packet protocol
US7120723B2 (en) 2004-03-25 2006-10-10 Micron Technology, Inc. System and method for memory hub-based expansion bus
US9977561B2 (en) 2004-04-01 2018-05-22 Sonos, Inc. Systems, methods, apparatus, and articles of manufacture to provide guest access
US6980042B2 (en) 2004-04-05 2005-12-27 Micron Technology, Inc. Delay line synchronizer apparatus and method
US7590797B2 (en) 2004-04-08 2009-09-15 Micron Technology, Inc. System and method for optimizing interconnections of components in a multichip memory module
US7162567B2 (en) * 2004-05-14 2007-01-09 Micron Technology, Inc. Memory hub and method for memory sequencing
US8024055B1 (en) 2004-05-15 2011-09-20 Sonos, Inc. Method and system for controlling amplifiers
US7363419B2 (en) 2004-05-28 2008-04-22 Micron Technology, Inc. Method and system for terminating write commands in a hub-based memory system
US7310748B2 (en) 2004-06-04 2007-12-18 Micron Technology, Inc. Memory hub tester interface and method for use thereof
US7519788B2 (en) 2004-06-04 2009-04-14 Micron Technology, Inc. System and method for an asynchronous data buffer having buffer write and read pointers
US8868698B2 (en) 2004-06-05 2014-10-21 Sonos, Inc. Establishing a secure wireless network with minimum human intervention
US8326951B1 (en) 2004-06-05 2012-12-04 Sonos, Inc. Establishing a secure wireless network with minimum human intervention
US7693797B2 (en) * 2004-06-21 2010-04-06 Nokia Corporation Transaction and payment system security remote authentication/validation of transactions from a transaction provider
US7426651B2 (en) * 2004-07-19 2008-09-16 Sony Corporation System and method for encoding independent clock using communication system reference clock
EP1622009A1 (en) * 2004-07-27 2006-02-01 Texas Instruments Incorporated JSM architecture and systems
US7392331B2 (en) 2004-08-31 2008-06-24 Micron Technology, Inc. System and method for transmitting data packets in a computer system having a memory hub architecture
US7132963B2 (en) * 2004-09-13 2006-11-07 Ati Technologies Inc. Methods and apparatus for processing variable length coded data
US20060119557A1 (en) * 2004-12-03 2006-06-08 Toppoly Optoelectronics Corporation System and method for driving an LCD
KR100666880B1 (en) * 2005-01-14 2007-01-10 삼성전자주식회사 System and method for decoding dual video signals
US20060176960A1 (en) * 2005-02-07 2006-08-10 Paul Lu Method and system for decoding variable length code (VLC) in a microprocessor
US20060176959A1 (en) * 2005-02-07 2006-08-10 Paul Lu Method and system for encoding variable length code (VLC) in a microprocessor
US7482954B1 (en) * 2005-02-25 2009-01-27 Xilinx, Inc. Bitstream compression for a programmable device
US7483173B2 (en) * 2005-03-10 2009-01-27 Kabushiki Kaisha Toshiba Data processor having a synchronizing function of a plurality of chips
CN100539437C (en) * 2005-07-29 2009-09-09 上海杰得微电子有限公司 A kind of implementation method of audio codec
GB2431800A (en) * 2005-10-31 2007-05-02 Sony Uk Ltd Interpolation involving motion vectors and mutiple tap spatial filters.
US20070116117A1 (en) * 2005-11-18 2007-05-24 Apple Computer, Inc. Controlling buffer states in video compression coding to enable editing and distributed encoding
US8780997B2 (en) * 2005-11-18 2014-07-15 Apple Inc. Regulation of decode-side processing based on perceptual masking
US8295343B2 (en) 2005-11-18 2012-10-23 Apple Inc. Video bit rate control method
US8031777B2 (en) * 2005-11-18 2011-10-04 Apple Inc. Multipass video encoding and rate control using subsampling of frames
US8233535B2 (en) 2005-11-18 2012-07-31 Apple Inc. Region-based processing of predicted pixels
US20070162531A1 (en) * 2006-01-12 2007-07-12 Bhaskar Kota Flow transform for integrated circuit design and simulation having combined data flow, control flow, and memory flow views
EP1974265A1 (en) * 2006-01-18 2008-10-01 PACT XPP Technologies AG Hardware definition method
TWI299133B (en) 2006-01-23 2008-07-21 Realtek Semiconductor Corp Webcasting system and audio regulating methods therefor
US8525842B1 (en) * 2006-06-16 2013-09-03 Nvidia Corporation System and method for utilizing semaphores in a graphics pipeline
TWI453751B (en) * 2006-08-04 2014-09-21 Marvell World Trade Ltd Fully-buffered dual in-line memory module with fault correction
KR100792431B1 (en) * 2006-08-31 2008-01-10 주식회사 하이닉스반도체 Semiconductor memory device
US8788080B1 (en) 2006-09-12 2014-07-22 Sonos, Inc. Multi-channel pairing in a media system
US8483853B1 (en) 2006-09-12 2013-07-09 Sonos, Inc. Controlling and manipulating groupings in a multi-zone media system
US9202509B2 (en) 2006-09-12 2015-12-01 Sonos, Inc. Controlling and grouping in a multi-zone media system
US20080103875A1 (en) * 2006-10-31 2008-05-01 Michael Kokernak Methods and systems for an interactive data finder
US7817470B2 (en) * 2006-11-27 2010-10-19 Mosaid Technologies Incorporated Non-volatile memory serial core architecture
JP4453697B2 (en) * 2006-12-15 2010-04-21 ソニー株式会社 Arithmetic processing device, arithmetic processing control method, and computer program
US20080167992A1 (en) * 2007-01-05 2008-07-10 Backchannelmedia Inc. Methods and systems for an accountable media advertising application
US8347019B2 (en) * 2007-01-26 2013-01-01 International Business Machines Corporation Structure for hardware assisted bus state transition circuit using content addressable memories
US9597019B2 (en) * 2007-02-09 2017-03-21 Lifescan, Inc. Method of ensuring date and time on a test meter is accurate
KR100839504B1 (en) * 2007-02-23 2008-06-19 삼성전자주식회사 Method of decoding an image, image decoder performing the same, and mobile device having the same
US20080282072A1 (en) * 2007-05-08 2008-11-13 Leonard Todd E Executing Software Within Real-Time Hardware Constraints Using Functionally Programmable Branch Table
JP5040427B2 (en) * 2007-05-11 2012-10-03 ソニー株式会社 DATA PROCESSING METHOD, DATA PROCESSING DEVICE, SOLID-STATE IMAGING DEVICE, IMAGING DEVICE, ELECTRONIC DEVICE
WO2009034487A2 (en) * 2007-09-10 2009-03-19 Nxp B.V. Method and apparatus for motion estimation and motion compensation in video image data
US20090080538A1 (en) * 2007-09-20 2009-03-26 Aten International Co., Ltd. Method and Apparatus for Decoding a Video Signal
KR101372418B1 (en) * 2007-10-19 2014-03-12 (주)휴맥스 Bitstream decoding device and method
JP2011503733A (en) * 2007-11-17 2011-01-27 トーマス リヒター Reconfigurable floating point level and bit level data processing unit
US7760135B2 (en) * 2007-11-27 2010-07-20 Lockheed Martin Corporation Robust pulse deinterleaving
WO2009068014A2 (en) * 2007-11-28 2009-06-04 Pact Xpp Technologies Ag On data processing
EP2235627A1 (en) * 2007-12-07 2010-10-06 Krass, Maren Using function calls as compiler directives
US8051455B2 (en) 2007-12-12 2011-11-01 Backchannelmedia Inc. Systems and methods for providing a token registry and encoder
JP4536109B2 (en) * 2007-12-26 2010-09-01 富士通株式会社 Semiconductor device and signal processing method
CN101593095B (en) * 2008-05-28 2013-03-13 国际商业机器公司 Method and system for processing data based on pipelining-stage
KR101649493B1 (en) * 2008-09-18 2016-08-19 삼성전자주식회사 Method and apparatus for extending the length of cyclic redundancy check
US20100098074A1 (en) * 2008-10-22 2010-04-22 Backchannelmedia Inc. Systems and methods for providing a network link between broadcast content and content located on a computer network
US9094721B2 (en) 2008-10-22 2015-07-28 Rakuten, Inc. Systems and methods for providing a network link between broadcast content and content located on a computer network
US8160064B2 (en) 2008-10-22 2012-04-17 Backchannelmedia Inc. Systems and methods for providing a network link between broadcast content and content located on a computer network
KR101590633B1 (en) * 2008-11-11 2016-02-02 삼성전자주식회사 / /apparatus for processing video encoding and decoding using video separation based on slice level and method therefor
US7773005B2 (en) * 2008-12-05 2010-08-10 Advanced Micro Devices, Inc. Method and apparatus for decoding variable length data
WO2010086500A1 (en) * 2009-01-28 2010-08-05 Nokia Corporation Method and apparatus for video coding and decoding
US7746109B1 (en) * 2009-04-02 2010-06-29 Xilinx, Inc. Circuits for sharing self-timed logic
KR20100136890A (en) * 2009-06-19 2010-12-29 삼성전자주식회사 Apparatus and method for arithmetic encoding and arithmetic decoding based context
US8423088B2 (en) 2009-07-22 2013-04-16 Microsoft Corporation Aggregated, interactive communication timeline
MY160807A (en) 2009-10-20 2017-03-31 Fraunhofer-Gesellschaft Zur Förderung Der Angewandten Audio encoder,audio decoder,method for encoding an audio information,method for decoding an audio information and computer program using a detection of a group of previously-decoded spectral values
US8885663B2 (en) 2009-12-03 2014-11-11 Thomson Licensing Data block processor in a mobile DTV system with diversity
US8739262B2 (en) * 2009-12-18 2014-05-27 Sabre Glbl Inc. Tokenized data security
US20120243611A1 (en) * 2009-12-22 2012-09-27 Sony Corporation Image processing apparatus and method as well as program
MY159982A (en) 2010-01-12 2017-02-15 Fraunhofer Ges Forschung Audio encoder, audio decoder, method for encoding and decoding an audio information, and computer program obtaining a context sub-region value on the basis of a norm of previously decoded spectral values
US8935489B2 (en) * 2010-01-19 2015-01-13 Rambus Inc. Adaptively time-multiplexing memory references from multiple processor cores
KR101676477B1 (en) * 2010-07-21 2016-11-15 삼성전자주식회사 Method and apparatus lossless encoding and decoding based on context
US9319880B2 (en) 2010-09-15 2016-04-19 Intel Corporation Reformatting data to decrease bandwidth between a video encoder and a buffer
US8402164B1 (en) 2010-10-27 2013-03-19 Xilinx, Inc. Asynchronous communication network and methods of enabling the asynchronous communication of data in an integrated circuit
WO2012090584A1 (en) 2010-12-28 2012-07-05 インターナショナル・ビジネス・マシーンズ・コーポレーション Device and method for determining search starting point
US11429343B2 (en) 2011-01-25 2022-08-30 Sonos, Inc. Stereo playback configuration and control
US11265652B2 (en) 2011-01-25 2022-03-01 Sonos, Inc. Playback device pairing
US9325999B2 (en) * 2011-03-10 2016-04-26 Sharp Kabushiki Kaisha Video decoder for slices
WO2012124063A1 (en) * 2011-03-15 2012-09-20 富士通株式会社 Semiconductor recording device and method for controlling semiconductor recording device
US8938312B2 (en) 2011-04-18 2015-01-20 Sonos, Inc. Smart line-in processing
US9172982B1 (en) * 2011-06-06 2015-10-27 Vuemix, Inc. Audio selection from a multi-video environment
US9154813B2 (en) 2011-06-09 2015-10-06 Comcast Cable Communications, Llc Multiple video content in a composite video stream
US9020044B2 (en) * 2011-06-13 2015-04-28 Ati Technologies Ulc Method and apparatus for writing video data in raster order and reading video data in macroblock order
US8767824B2 (en) 2011-07-11 2014-07-01 Sharp Kabushiki Kaisha Video decoder parallelization for tiles
US9042556B2 (en) 2011-07-19 2015-05-26 Sonos, Inc Shaping sound responsive to speaker orientation
CN103797811B (en) 2011-09-09 2017-12-12 乐天株式会社 The system and method for the control contacted for consumer to interactive television
TWI455595B (en) * 2011-09-29 2014-10-01 Mstar Semiconductor Inc Boolean entropy decoder and boolean entropy decoding method in video display system
US8824569B2 (en) 2011-12-07 2014-09-02 International Business Machines Corporation High bandwidth decompression of variable length encoded data streams
CN102413382B (en) * 2011-12-27 2014-06-11 四川九洲电器集团有限责任公司 Method for promoting smoothness of real-time video
JP5422640B2 (en) * 2011-12-28 2014-02-19 京セラドキュメントソリューションズ株式会社 Image reading device
CN104094603B (en) * 2011-12-28 2018-06-08 英特尔公司 For carrying out the system and method that integrated metadata is inserted into video coding system
US9729115B2 (en) 2012-04-27 2017-08-08 Sonos, Inc. Intelligently increasing the sound level of player
WO2014070129A2 (en) * 2012-06-04 2014-05-08 Hewlett-Packard Development Company, L.P. Managing an analytic function to be performed on data stored in an input block
US9008330B2 (en) 2012-09-28 2015-04-14 Sonos, Inc. Crossover frequency adjustments for audio speakers
WO2014163241A1 (en) 2013-04-02 2014-10-09 주식회사 칩스앤미디어 Method and apparatus for processing video
US9626184B2 (en) * 2013-06-28 2017-04-18 Intel Corporation Processors, methods, systems, and instructions to transcode variable length code points of unicode characters
US9374106B2 (en) 2013-08-28 2016-06-21 International Business Machines Corporation Efficient context save/restore during hardware decompression of DEFLATE encoded data
US8933824B1 (en) 2013-08-28 2015-01-13 International Business Machines Corporation Hardware decompression of deflate encoded data with multiple blocks
WO2015035320A1 (en) * 2013-09-06 2015-03-12 Huawei Technologies Co., Ltd. System and method for an asynchronous processor with a hierarchical token system
US9244516B2 (en) 2013-09-30 2016-01-26 Sonos, Inc. Media playback system using standby mode in a mesh network
US9800640B2 (en) 2013-10-02 2017-10-24 International Business Machines Corporation Differential encoder with look-ahead synchronization
US9226087B2 (en) 2014-02-06 2015-12-29 Sonos, Inc. Audio output balancing during synchronized playback
US9226073B2 (en) 2014-02-06 2015-12-29 Sonos, Inc. Audio output balancing during synchronized playback
TWI675372B (en) * 2014-06-05 2019-10-21 美商積佳半導體股份有限公司 Systems and methods involving multi-bank, dual-pipe memory circuitry
US9647684B2 (en) * 2014-10-21 2017-05-09 Huawei Technologies Co., Ltd. Memory-based history search
CN104486585B (en) * 2014-12-18 2018-01-05 深圳先进技术研究院 A kind of city magnanimity monitor video management method and system based on GIS
US9252805B1 (en) * 2015-03-28 2016-02-02 International Business Machines Corporation Parallel huffman decoder
US9838571B2 (en) * 2015-04-10 2017-12-05 Gvbb Holdings S.A.R.L. Precision timing for broadcast network
US9378782B1 (en) * 2015-05-24 2016-06-28 Silicon Laboratories Inc. Apparatus with write-back buffer and associated methods
US10248376B2 (en) 2015-06-11 2019-04-02 Sonos, Inc. Multiple groupings in a playback system
US9484954B1 (en) 2015-09-10 2016-11-01 Intel Corporation Methods and apparatus to parallelize data decompression
JP2017182854A (en) * 2016-03-31 2017-10-05 マイクロン テクノロジー， インク． Semiconductor device
CN109643456A (en) * 2016-06-17 2019-04-16 因默希弗机器人私人有限公司 Method for compressing image and equipment
US10701404B2 (en) * 2016-08-30 2020-06-30 Dolby Laboratories Licensing Corporation Real-time reshaping of single-layer backwards-compatible codec
JP2018049381A (en) * 2016-09-20 2018-03-29 東芝メモリ株式会社 Memory control circuit, memory system, and processor system
US9825649B1 (en) * 2016-09-29 2017-11-21 Intel Corporation Efficient huffman decoder improvements
US10712997B2 (en) 2016-10-17 2020-07-14 Sonos, Inc. Room association based on name
US10567800B2 (en) 2016-11-29 2020-02-18 Qualcomm Incorporated Transform hardware architecture for video coding
US9819359B1 (en) 2016-12-11 2017-11-14 Microsoft Technology Licensing, Llc Multi-symbol, multi-format, parallel symbol decoder for hardware decompression engines
EP3579940A4 (en) 2017-02-08 2020-11-18 Immersive Robotics Pty Ltd Displaying content to users in a multiplayer venue
US10691361B2 (en) 2017-02-24 2020-06-23 Microsoft Technology Licensing, Llc Multi-format pipelined hardware decompressor
US10020819B1 (en) * 2017-09-28 2018-07-10 Amazon Technologies, Inc. Speculative data decompression
WO2019100108A1 (en) 2017-11-21 2019-05-31 Immersive Robotics Pty Ltd Image compression for digital reality
EP3714598A4 (en) 2017-11-21 2021-03-31 Immersive Robotics Pty Ltd Frequency component selection for image compression
CN107886081B (en) * 2017-11-23 2021-02-02 武汉理工大学 Intelligent underground dangerous behavior grading identification method for double-path U-Net deep neural network mine
WO2019126030A1 (en) * 2017-12-18 2019-06-27 Mythic, Inc. Systems and methods for mapping matrix calculations to a matrix multiply accelerator
US10044369B1 (en) * 2018-03-16 2018-08-07 Centri Technology, Inc. Interleaved codes for dynamic sizeable headers
US11152050B2 (en) 2018-06-19 2021-10-19 Micron Technology, Inc. Apparatuses and methods for multiple row hammer refresh address sequences
US20190392287A1 (en) 2018-06-22 2019-12-26 Samsung Electronics Co., Ltd. Neural processor
EP3818439A4 (en) * 2018-07-05 2022-04-27 Mythic, Inc. Systems and methods for implementing an intelligence processing computing architecture
CN109120273A (en) * 2018-08-29 2019-01-01 重庆物奇科技有限公司 Code device, code translator and system based on huffman coding
CN109104199A (en) * 2018-08-29 2018-12-28 重庆物奇科技有限公司 Coding method, interpretation method and application based on huffman coding
US10878534B2 (en) * 2018-11-07 2020-12-29 Kyocera Document Solutions Inc. Apparatuses, processes, and computer program products for image data downscaling
CN109542059B (en) * 2018-11-19 2022-04-01 国核自仪系统工程有限公司 Historical data compression device and method
CN113383317B (en) * 2019-01-31 2023-07-18 华为技术有限公司 Processing device, method and related equipment
CN110175185B (en) * 2019-04-17 2023-04-07 上海天数智芯半导体有限公司 Self-adaptive lossless compression method based on time sequence data distribution characteristics
US11211944B2 (en) 2019-04-17 2021-12-28 Samsung Electronics Co., Ltd. Mixed-precision compression with random access
US11671111B2 (en) 2019-04-17 2023-06-06 Samsung Electronics Co., Ltd. Hardware channel-parallel data compression/decompression
US11264096B2 (en) 2019-05-14 2022-03-01 Micron Technology, Inc. Apparatuses, systems, and methods for a content addressable memory cell with latch and comparator circuits
CN110134676B (en) * 2019-06-03 2021-01-29 西安电子科技大学 Method for monitoring data quality of sensor
US11158373B2 (en) 2019-06-11 2021-10-26 Micron Technology, Inc. Apparatuses, systems, and methods for determining extremum numerical values
US11139015B2 (en) 2019-07-01 2021-10-05 Micron Technology, Inc. Apparatuses and methods for monitoring word line accesses
TWI743774B (en) * 2019-07-18 2021-10-21 瑞昱半導體股份有限公司 Method for synchronizing audio and video and related apparatus
US10964378B2 (en) 2019-08-22 2021-03-30 Micron Technology, Inc. Apparatus and method including analog accumulator for determining row access rate and target row address used for refresh operation
US10976709B1 (en) 2020-03-30 2021-04-13 Stmicroelectronics (Research & Development) Limited Latched gray code for ToF applications
US11600314B2 (en) 2021-03-15 2023-03-07 Micron Technology, Inc. Apparatuses and methods for sketch circuits for refresh binning
CN112987616B (en) * 2021-03-15 2022-03-25 沈阳智谷科技有限公司 Ultrahigh-speed electronic package acquisition system and method based on magnetic variable signals
US11664063B2 (en) 2021-08-12 2023-05-30 Micron Technology, Inc. Apparatuses and methods for countering memory attacks
CN113839678B (en) * 2021-08-31 2023-11-03 山东云海国创云计算装备产业创新中心有限公司 Huffman decoding system, method, equipment and computer readable storage medium
CN114138057A (en) * 2021-11-19 2022-03-04 广西电网有限责任公司 Agent-based intelligent clock time synchronization device and use method
US11688451B2 (en) 2021-11-29 2023-06-27 Micron Technology, Inc. Apparatuses, systems, and methods for main sketch and slim sketch circuit for row address tracking
CN117095628B (en) * 2023-10-17 2023-12-26 北京数字光芯集成电路设计有限公司 Progressive and bitwise scanning method and system based on digital pulse width modulation display

Citations (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4135242A (en) * 1977-11-07 1979-01-16 Ncr Corporation Method and processor having bit-addressable scratch pad memory
GB2039106A (en) * 1979-01-02 1980-07-30 Honeywell Inf Systems Number format conversion in computer Fig 3A
US4236228A (en) * 1977-03-17 1980-11-25 Tokyo Shibaura Electric Co., Ltd. Memory device for processing picture images data
EP0075893A2 (en) * 1981-09-30 1983-04-06 Siemens Aktiengesellschaft Memory operand alignment circuit arrangement for decimal and logical instructions
US4677500A (en) * 1984-02-08 1987-06-30 U.S. Philips Corporation System for playing back a film recorded as a video signal on a disc-shaped record carrier
EP0280573A2 (en) * 1987-02-26 1988-08-31 Pioneer Electronic Corporation Device for reproducing still pictures with an audio portion
US4807028A (en) * 1986-11-10 1989-02-21 Kokusai Denshin Denwa Co., Ltd. Decoding device capable of producing a decoded video signal with a reduced delay
EP0321628A1 (en) * 1987-12-23 1989-06-28 International Business Machines Corporation Shared memory interface for a data processing system
US4875196A (en) * 1987-09-08 1989-10-17 Sharp Microelectronic Technology, Inc. Method of operating data buffer apparatus
DE3832563A1 (en) * 1988-09-24 1990-03-29 Bosch Gmbh Robert Memory arrangement for a narrow-band picture transmission device
EP0446956A2 (en) * 1990-03-16 1991-09-18 Fujitsu Limited An ATM transmission system with a variable transmission rate
EP0460751A2 (en) * 1990-06-05 1991-12-11 Koninklijke Philips Electronics N.V. Method of transmitting audio and/or video signals
EP0468480A2 (en) * 1990-07-25 1992-01-29 Oki Electric Industry Co., Ltd. Synchronous burst-access memory and word-line driving circuit therefor
US5089992A (en) * 1988-06-30 1992-02-18 Mitsubishi Denki Kabushiki Kaisha Semiconductor memory device and a data path using the same
EP0503956A2 (en) * 1991-03-15 1992-09-16 C-Cube Microsystems Decompression of video signal
EP0506294A2 (en) * 1991-03-28 1992-09-30 AT&T Corp. Hierarchical variable length decoder for digital video data
US5173695A (en) * 1990-06-29 1992-12-22 Bell Communications Research, Inc. High-speed flexible variable-length-code decoder
EP0542195A2 (en) * 1991-11-13 1993-05-19 Sony Corporation Interframe predicting method and picture signal coding/decoding apparatus
EP0562419A2 (en) * 1992-03-26 1993-09-29 General Instrument Corporation Of Delaware Variable length code word decoder for use in digital communication systems
US5253053A (en) * 1990-12-31 1993-10-12 Apple Computer, Inc. Variable length decoding using lookup tables
EP0572263A2 (en) * 1992-05-28 1993-12-01 C-Cube Microsystems, Inc. Variable length code decoder for video decompression operations
EP0577329A2 (en) * 1992-07-02 1994-01-05 AT&T Corp. Timing recovery for variable bit-rate video on asynchronous transfer mode (ATM) networks
US5280349A (en) * 1992-02-13 1994-01-18 Industrial Technology Research Institute HDTV decoder
US5293229A (en) * 1992-03-27 1994-03-08 Matsushita Electric Corporation Of America Apparatus and method for processing groups of fields in a video data compression system
EP0587443A2 (en) * 1992-09-10 1994-03-16 Texas Instruments Incorporated Image processor and method
US5319724A (en) * 1990-04-19 1994-06-07 Ricoh Corporation Apparatus and method for compressing still images
EP0600446A2 (en) * 1992-11-30 1994-06-08 Sony Corporation Decoder for a compressed digital video signal using a common memory for decoding and 2/3 pull-down conversion
US5325092A (en) * 1992-07-07 1994-06-28 Ricoh Company, Ltd. Huffman decoder architecture for high speed operation and reduced memory
EP0618728A2 (en) * 1993-02-26 1994-10-05 Sony Corporation Synchronization of audio/video information
EP0618772A1 (en) * 1992-10-23 1994-10-12 Schaartec Gmbh & Co Kg Deboning device and method.
US5430485A (en) * 1993-09-30 1995-07-04 Thomson Consumer Electronics, Inc. Audio/video synchronization in a digital transmission system
EP0674266A2 (en) * 1994-03-24 1995-09-27 Discovision Associates Method and apparatus for interfacing with ram
EP0695095A2 (en) * 1994-07-29 1996-01-31 Discovision Associates Video decompression
US5561465A (en) * 1993-03-31 1996-10-01 U.S. Philips Corporation Video decoder with five page memory for decoding of intraframes, predicted frames and bidirectional frames

Family Cites Families (305)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US33632A (en) * 1861-11-05 Improved water-closet
GB9405914D0 (en) * 1994-03-24 1994-05-11 Discovision Ass Video decompression
GB1243631A (en) 1967-07-18 1971-08-25 Nat Res Dev Improvements in or relating to elastic encoder storage systems
US3676802A (en) 1971-06-21 1972-07-11 Us Navy Submarine propeller cavitation noise simulator
US3873976A (en) * 1973-07-30 1975-03-25 Burroughs Corp Memory access system
US3875391A (en) * 1973-11-02 1975-04-01 Raytheon Co Pipeline signal processor
US3872430A (en) * 1973-11-23 1975-03-18 Paul Emile Boudreau Method and apparatus of error detection for variable length words using a polynomial code
US3893042A (en) * 1973-12-12 1975-07-01 Us Navy Lock indicator for phase-locked loops
US3962685A (en) * 1974-06-03 1976-06-08 General Electric Company Data processing system having pyramidal hierarchy control flow
GB1532275A (en) * 1976-01-28 1978-11-15 Nat Res Dev Apparatus for controlling raster-scan displays
US4125369A (en) * 1977-03-28 1978-11-14 The Dow Chemical Company Permanent topical textile antistats
US4142205A (en) * 1976-07-21 1979-02-27 Nippon Electric Co., Ltd. Interframe CODEC for composite color TV signals comprising means for inverting the polarity of carrier chrominance signals in every other frame or line
US4149242A (en) * 1977-05-06 1979-04-10 Bell Telephone Laboratories, Incorporated Data interface apparatus for multiple sequential processors
US4228497A (en) * 1977-11-17 1980-10-14 Burroughs Corporation Template micromemory structure for a pipelined microprogrammable data processing system
US4215369A (en) * 1977-12-20 1980-07-29 Nippon Electric Company, Ltd. Digital transmission system for television video signals
US4196448A (en) * 1978-05-15 1980-04-01 The United States Of America As Represented By The Secretary Of The Navy TV bandwidth reduction system using a hybrid discrete cosine DPCM
US4225920A (en) * 1978-09-11 1980-09-30 Burroughs Corporation Operator independent template control architecture
US4302775A (en) * 1978-12-15 1981-11-24 Compression Labs, Inc. Digital video compression system and methods utilizing scene adaptive coding with rate buffer feedback
JPS6046585B2 (en) 1979-03-06 1985-10-16 株式会社リコー Serial data transmission method
US4307447A (en) * 1979-06-19 1981-12-22 Gould Inc. Programmable controller
JPS6010458B2 (en) * 1979-08-23 1985-03-18 富士通株式会社 Phase locked loop circuit
GB2059724B (en) 1979-09-28 1984-04-04 Racal Datacom Ltd Data transmission systems
DE3015125A1 (en) * 1980-04-19 1981-10-22 Ibm Deutschland Gmbh, 7000 Stuttgart DEVICE FOR STORING AND DISPLAYING GRAPHIC INFORMATION
US4334246A (en) * 1980-05-16 1982-06-08 Xerox Corporation Data decompressor circuit
US4467409A (en) * 1980-08-05 1984-08-21 Burroughs Corporation Flexible computer architecture using arrays of standardized microprocessors customized for pipeline and parallel operations
JPS6228086Y2 (en) 1980-12-08 1987-07-18
JPS59126368A (en) * 1983-01-10 1984-07-20 Hitachi Ltd Coder and encoder
US4495629A (en) * 1983-01-25 1985-01-22 Storage Technology Partners CMOS scannable latch
JPS59146345A (en) * 1983-02-10 1984-08-22 Masahiro Sowa Control flow parallel computer system
US4799677A (en) * 1983-09-02 1989-01-24 Bally Manufacturing Corporation Video game having video disk read only memory
US4710866A (en) * 1983-09-12 1987-12-01 Motorola, Inc. Method and apparatus for validating prefetched instruction
US4540903A (en) * 1983-10-17 1985-09-10 Storage Technology Partners Scannable asynchronous/synchronous CMOS latch
US4598372A (en) * 1983-12-28 1986-07-01 Motorola, Inc. Apparatus and method of smoothing MAPS compressed image data
US4689823A (en) * 1984-01-04 1987-08-25 Itek Corporation Digital image frame processor
US4747070A (en) * 1984-01-09 1988-05-24 Wang Laboratories, Inc. Reconfigurable memory system
US4630198A (en) * 1984-02-21 1986-12-16 Yuan Houng I Intelligent stand-alone printfile buffer with paging control
FR2561011B1 (en) * 1984-03-09 1986-09-12 Cit Alcatel PROCESSOR FOR CALCULATING A DISCRETE INVERSE COSINUS TRANSFORM
US4580066A (en) * 1984-03-22 1986-04-01 Sperry Corporation Fast scan/set testable latch using two levels of series gating with two current sources
US4885786A (en) * 1984-10-24 1989-12-05 International Business Machines Corporation Method for enlarging an image stored in run representation form
CA1220830A (en) * 1984-12-28 1987-04-21 David S. Drynan Transmitting sequence numbers of information in a packet data transmission system
US4646151A (en) * 1985-02-01 1987-02-24 General Electric Company Television frame synchronizer with independently controllable input/output rates
US4808398A (en) 1985-02-14 1989-02-28 The Dow Chemical Company Narrow size distribution zinc oxide
JPS61194989A (en) 1985-02-22 1986-08-29 Mitsubishi Electric Corp Still picture transmitter
US4680581A (en) 1985-03-28 1987-07-14 Honeywell Inc. Local area network special function frames
US5233420A (en) 1985-04-10 1993-08-03 The United States Of America As Represented By The Secretary Of The Navy Solid state time base corrector (TBC)
JPH0632056B2 (en) * 1985-05-31 1994-04-27 松下電器産業株式会社 Data processing device
JPS61276032A (en) * 1985-05-31 1986-12-06 Matsushita Electric Ind Co Ltd Information processing device
US5263135A (en) 1985-07-18 1993-11-16 Canon Kabushiki Kaisha Image processing apparatus
DE3525898A1 (en) * 1985-07-19 1987-01-29 Reinhard Lidzba METHOD FOR COMPRESSING AND DECOMPRESSING SEVERAL STRUCTURE-RELATED DATA SEQUENCES AND DEVICES FOR IMPLEMENTING THE METHOD
JP2564805B2 (en) * 1985-08-08 1996-12-18 日本電気株式会社 Information processing device
US4720780A (en) * 1985-09-17 1988-01-19 The Johns Hopkins University Memory-linked wavefront array processor
US4692880A (en) * 1985-11-15 1987-09-08 General Electric Company Memory efficient cell texturing for advanced video object generator
JPS62139081A (en) 1985-12-13 1987-06-22 Canon Inc Formation of synthetic image
US4894823A (en) * 1986-02-28 1990-01-16 American Telephone And Telegraph Company Time stamping for packet system nodes
US4726019A (en) * 1986-02-28 1988-02-16 American Telephone And Telegraph Company, At&T Bell Laboratories Digital encoder and decoder synchronization in the presence of late arriving packets
US5021947A (en) * 1986-03-31 1991-06-04 Hughes Aircraft Company Data-flow multiprocessor architecture with three dimensional multistage interconnection network for efficient signal and data processing
US4789927A (en) * 1986-04-07 1988-12-06 Silicon Graphics, Inc. Interleaved pipeline parallel processing architecture
JP2500858B2 (en) * 1986-04-11 1996-05-29 インターナショナル・ビジネス・マシーンズ・コーポレーション Display system having extended raster operation circuit
US4843632A (en) * 1986-05-09 1989-06-27 Prodigy Systems Corporation Compressed image expansion system
FR2599872B1 (en) * 1986-06-06 1988-07-29 Thomson Csf DEVICES FOR CALCULATING MONO-DIMENSIONAL COSINE TRANSFORMS, AND CODING DEVICE AND IMAGE DECODING DEVICE COMPRISING SUCH COMPUTING DEVICES
US4829465A (en) * 1986-06-19 1989-05-09 American Telephone And Telegraph Company, At&T Bell Laboratories High speed cosine transform
US4814978A (en) * 1986-07-15 1989-03-21 Dataflow Computer Corporation Dataflow processing element, multiprocessor, and processes
GB8618060D0 (en) 1986-07-24 1986-12-17 Gec Avionics Data processing apparatus
EP0255767A3 (en) 1986-07-31 1990-04-04 AT&T Corp. Selective broadcasting arrangement for local area networks
US4887224A (en) * 1986-08-28 1989-12-12 Canon Kabushiki Kaisha Image data processing apparatus capable of high-speed data encoding and/or decoding
US4910417A (en) * 1986-09-19 1990-03-20 Actel Corporation Universal logic module comprising multiplexers
US4811214A (en) * 1986-11-14 1989-03-07 Princeton University Multinode reconfigurable pipeline computer
IT1207346B (en) 1987-01-20 1989-05-17 Cselt Centro Studi Lab Telecom DISCREET DISCREET COSE COEFFI CIRCUIT FOR THE CALCULATION OF THE QUANTITIES OF NUMERICAL SIGNAL SAMPLES
US4831321A (en) * 1987-03-03 1989-05-16 Robert Cooper Trickle jumping charging device
CA1309519C (en) * 1987-03-17 1992-10-27 Antonio Cantoni Transfer of messages in a multiplexed system
NL8700843A (en) * 1987-04-10 1988-11-01 Philips Nv TELEVISION TRANSFER SYSTEM WITH TRANSFORM CODING.
US4855947A (en) * 1987-05-27 1989-08-08 Amdahl Corporation Microprogrammable pipeline interlocks based on the validity of pipeline states
DE3718521A1 (en) * 1987-06-03 1988-12-22 Du Pont Deutschland METHOD FOR RECORDING QUASI HALFTONE IMAGES AND DEVICE FOR CONVERTING IMAGE POINT TONE VALUE DATA
US4975595A (en) * 1987-06-12 1990-12-04 National Semiconductor Corporation Scannable register/latch circuit
US5297263A (en) 1987-07-17 1994-03-22 Mitsubishi Denki Kabushiki Kaisha Microprocessor with pipeline system having exception processing features
JPS6477391A (en) * 1987-09-18 1989-03-23 Victor Company Of Japan System and device for predictive coding
GB8722394D0 (en) * 1987-09-23 1987-10-28 British Telecomm Video coder
US5163132A (en) * 1987-09-24 1992-11-10 Ncr Corporation Integrated controller using alternately filled and emptied buffers for controlling bi-directional data transfer between a processor and a data storage device
DE3877105D1 (en) 1987-09-30 1993-02-11 Siemens Ag, 8000 Muenchen, De
US5122873A (en) * 1987-10-05 1992-06-16 Intel Corporation Method and apparatus for selectively encoding and decoding a digital motion video signal at multiple resolution levels
US4785349A (en) * 1987-10-05 1988-11-15 Technology Inc. 64 Digital video decompression system
US4811413A (en) * 1987-10-22 1989-03-07 International Business Machines Corp. System of reconfigurable pipelines of generalized neighborhood function morphic image processors
US4866637A (en) * 1987-10-30 1989-09-12 International Business Machines Corporation Pipelined lighting model processing system for a graphics workstation's shading function
US5134697A (en) * 1987-11-16 1992-07-28 Prime Computer Remote memory-mapped display with interactivity determination
US4823201A (en) * 1987-11-16 1989-04-18 Technology, Inc. 64 Processor for expanding a compressed video signal
US4897803A (en) * 1987-11-23 1990-01-30 Xerox Corporation Address token based image manipulation
NL8702905A (en) * 1987-12-03 1989-07-03 Philips Nv METHOD AND DEVICE FOR RECORDING INFORMATION, A RECORD BRACKET, A DEVICE FOR READING THE RECORDED INFORMATION, AND A CODING AND DECODING CIRCUIT FOR USE IN THE RECORDING AND READING DEVICE.
FR2625340B1 (en) * 1987-12-23 1990-05-04 Labo Electronique Physique GRAPHIC SYSTEM WITH GRAPHIC CONTROLLER AND DRAM CONTROLLER
US4891784A (en) * 1988-01-08 1990-01-02 Hewlett-Packard Company High capacity tape drive transparently writes and reads large packets of blocked data between interblock gaps
JPH01188951A (en) * 1988-01-23 1989-07-28 Sharp Corp Control system for execution of data flow program
GB8805742D0 (en) * 1988-03-10 1988-04-07 Emi Plc Thorn Bandwidth reduction system for television signals
US5214770A (en) 1988-04-01 1993-05-25 Digital Equipment Corporation System for flushing instruction-cache only when instruction-cache address and data-cache address are matched and the execution of a return-from-exception-or-interrupt command
US4949280A (en) * 1988-05-10 1990-08-14 Battelle Memorial Institute Parallel processor-based raster graphics system architecture
US5121498A (en) * 1988-05-11 1992-06-09 Massachusetts Institute Of Technology Translator for translating source code for selective unrolling of loops in the source code
US5200925A (en) 1988-07-29 1993-04-06 Mitsubishi Denki Kabushiki Kaisha Serial access semiconductor memory device and operating method therefor
US5010401A (en) * 1988-08-11 1991-04-23 Mitsubishi Denki Kabushiki Kaisha Picture coding and decoding apparatus using vector quantization
US4989138A (en) * 1988-09-02 1991-01-29 Tektronix, Inc. Single bus graphics data processing pipeline with decentralized bus arbitration
US4991110A (en) * 1988-09-13 1991-02-05 Silicon Graphics, Inc. Graphics processor with staggered memory timing
US4866510A (en) * 1988-09-30 1989-09-12 American Telephone And Telegraph Company Digital video encoder
US5241635A (en) 1988-11-18 1993-08-31 Massachusetts Institute Of Technology Tagged token data processing system with operand matching in activation frames
US5148524A (en) * 1988-11-29 1992-09-15 Solbourne Computer, Inc. Dynamic video RAM incorporating on chip vector/image mode line modification
US5161221A (en) 1988-12-12 1992-11-03 Eastman Kodak Company Multi-memory bank system for receiving continuous serial data stream and monitoring same to control bank switching without interrupting continuous data flow rate
US5091721A (en) * 1988-12-22 1992-02-25 Hughes Aircraft Company Acoustic display generator
GB2226471A (en) 1988-12-23 1990-06-27 Philips Electronic Associated Displaying a stored image in expanded format
SG126671A1 (en) * 1989-02-10 2006-11-29 Canon Kk Apparatus for image reading or processing
US5060242A (en) * 1989-02-24 1991-10-22 General Electric Company Non-destructive lossless image coder
JP2925157B2 (en) * 1989-02-28 1999-07-28 キヤノン株式会社 Data storage device
JP2765918B2 (en) * 1989-03-06 1998-06-18 株式会社日立製作所 Presentation device
JP2766302B2 (en) * 1989-04-06 1998-06-18 株式会社東芝 Variable length code parallel decoding method and apparatus
JPH02280462A (en) * 1989-04-20 1990-11-16 Fuji Photo Film Co Ltd Picture data compression method
AU612543B2 (en) 1989-05-11 1991-07-11 Panasonic Corporation Moving image signal encoding apparatus and decoding apparatus
US5172011A (en) * 1989-06-30 1992-12-15 Digital Equipment Corporation Latch circuit and method with complementary clocking and level sensitive scan capability
US5233690A (en) 1989-07-28 1993-08-03 Texas Instruments Incorporated Video graphics display memory swizzle logic and expansion circuit and method
US5257350A (en) 1989-08-10 1993-10-26 Apple Computer, Inc. Computer with self configuring video circuitry
US5151997A (en) * 1989-08-10 1992-09-29 Apple Computer, Inc. Computer with adaptable video circuitry
US5233545A (en) 1989-09-19 1993-08-03 Hewlett-Packard Company Time interval triggering and hardware histogram generation
US5299025A (en) 1989-10-18 1994-03-29 Ricoh Company, Ltd. Method of coding two-dimensional data by fast cosine transform and method of decoding compressed data by inverse fast cosine transform
US5053985A (en) 1989-10-19 1991-10-01 Zoran Corporation Recycling dct/idct integrated circuit apparatus using a single multiplier/accumulator and a single random access memory
US5142380A (en) * 1989-10-23 1992-08-25 Ricoh Company, Ltd. Image data processing apparatus
EP0428310A3 (en) * 1989-11-06 1992-08-05 Canon Kabushiki Kaisha Image processing apparatus and image transmitting apparatus
US5057793A (en) * 1989-11-13 1991-10-15 Cowley Nicholas P Frequency synthesizer PLL having digital and analog phase detectors
US5227863A (en) 1989-11-14 1993-07-13 Intelligent Resources Integrated Systems, Inc. Programmable digital video processing system
US5146326A (en) * 1989-11-14 1992-09-08 Fujitsu Limited Coded picture information decoding apparatus having means for improving picture distortion
US5027212A (en) * 1989-12-06 1991-06-25 Videologic Limited Computer based video/graphics display system
US5003204A (en) * 1989-12-19 1991-03-26 Bull Hn Information Systems Inc. Edge triggered D-type flip-flop scan latch cell with recirculation capability
US5263316A (en) 1989-12-21 1993-11-23 Sundstrand Corporation Turbine engine with airblast injection
US5226131A (en) 1989-12-27 1993-07-06 The United States Of America As Represented By The United States Department Of Energy Sequencing and fan-out mechanism for causing a set of at least two sequential instructions to be performed in a dataflow processing computer
US5287470A (en) 1989-12-28 1994-02-15 Texas Instruments Incorporated Apparatus and method for coupling a multi-lead output bus to interleaved memories, which are addressable in normal and block-write modes
JP2881886B2 (en) * 1989-12-30 1999-04-12 ソニー株式会社 Video signal encoding method and apparatus therefor
US5221966A (en) 1990-01-17 1993-06-22 Avesco Plc Video signal production from cinefilm originated material
US5113182B1 (en) 1990-01-19 1995-11-07 Prince Corp Vehicle door locking system detecting that all doors are closed
JP2842913B2 (en) 1990-01-24 1999-01-06 株式会社日立製作所 Wide television signal processing circuit
JP2728760B2 (en) 1990-02-13 1998-03-18 株式会社東芝 Data transmission device and received data processing method
JPH03248243A (en) 1990-02-26 1991-11-06 Nec Corp Information processor
US5107345A (en) * 1990-02-27 1992-04-21 Qualcomm Incorporated Adaptive block size image compression method and system
US5081450A (en) * 1990-03-09 1992-01-14 International Business Machines Corporation Apparatus and method for compressing and expanding multibit digital pixel data
US5253078A (en) * 1990-03-14 1993-10-12 C-Cube Microsystems, Inc. System for compression and decompression of video data using discrete cosine transform and coding techniques
US5191548A (en) * 1990-03-14 1993-03-02 C-Cube Microsystems System for compression and decompression of video data using discrete cosine transform and coding techniques
US5136371A (en) * 1990-03-15 1992-08-04 Thomson Consumer Electronics, Inc. Digital image coding using random scanning
US5151875A (en) * 1990-03-16 1992-09-29 C-Cube Microsystems, Inc. MOS array multiplier cell
FR2660138B1 (en) * 1990-03-26 1992-06-12 France Telecom Cnet DEVICE FOR CODING / DECODING IMAGE SIGNALS.
EP0453229B1 (en) * 1990-04-17 1997-06-18 Matsushita Electric Industrial Co., Ltd. Method for transmission of variable length code
CA2014734A1 (en) 1990-04-17 1991-10-17 Marvin Simms Autotracking fume extraction exhaust hood
US5201056A (en) * 1990-05-02 1993-04-06 Motorola, Inc. RISC microprocessor architecture with multi-bit tag extended instructions for selectively attaching tag from either instruction or input data to arithmetic operation output
GB9011700D0 (en) 1990-05-25 1990-07-18 Inmos Ltd Communication interface
US5311309A (en) 1990-06-01 1994-05-10 Thomson Consumer Electronics, Inc. Luminance processing system for compressing and expanding video data
US5179372A (en) * 1990-06-19 1993-01-12 International Business Machines Corporation Video Random Access Memory serial port access
US5057917A (en) * 1990-06-20 1991-10-15 The United States Of America As Represented By The Administrator Of The National Aeronautics And Space Administration Real-time data compression of broadcast video signals
US5122948A (en) * 1990-06-28 1992-06-16 Allen-Bradley Company, Inc. Remote terminal industrial control communication system
DE69130138T2 (en) * 1990-06-29 1999-05-06 Digital Equipment Corp Jump prediction unit for high-performance processor
CA2044051A1 (en) 1990-06-29 1991-12-30 Paul C. Wade System and method for error detection and reducing simultaneous switching noise
JPH0459375A (en) * 1990-06-29 1992-02-26 Seikosha Co Ltd Serial printer
EP0466550B1 (en) 1990-06-29 1998-11-18 Digital Equipment Corporation Conversion of internal processor register commands to I/O space address
US5247612A (en) 1990-06-29 1993-09-21 Radius Inc. Pixel display apparatus and method using a first-in, first-out buffer
FR2664779B1 (en) 1990-07-13 1993-06-11 Europ Rech Electr Lab PROCESS FOR PROCESSING A VIDEO SIGNAL.
IL98700A (en) 1990-07-13 1994-04-12 Minnesota Mining & Mfg Apparatus and method for assembling a composite image from a plurality of data types
US5174641A (en) * 1990-07-25 1992-12-29 Massachusetts Institute Of Technology Video encoding method for television applications
US5202847A (en) 1990-07-31 1993-04-13 Inmos Limited Digital signal processing
US5241658A (en) 1990-08-21 1993-08-31 Apple Computer, Inc. Apparatus for storing information in and deriving information from a frame buffer
US5189526A (en) * 1990-09-21 1993-02-23 Eastman Kodak Company Method and apparatus for performing image compression using discrete cosine transform
US5297271A (en) 1990-09-21 1994-03-22 Chips And Technologies, Inc. Method and apparatus for performing a read-write-modify operation in a VGA compatible controller
US5367636A (en) 1990-09-24 1994-11-22 Ncube Corporation Hypercube processor network in which the processor indentification numbers of two processors connected to each other through port number n, vary only in the nth bit
US5038209A (en) * 1990-09-27 1991-08-06 At&T Bell Laboratories Adaptive buffer/quantizer control for transform video coders
JPH06500655A (en) 1990-10-03 1994-01-20 スィンキング　マシンズ　コーポレーション parallel computer system
JPH04142812A (en) 1990-10-04 1992-05-15 Toshiba Corp Phase locked loop circuit
US5130568A (en) * 1990-11-05 1992-07-14 Vertex Semiconductor Corporation Scannable latch system and method
US5229863A (en) 1990-12-24 1993-07-20 Xerox Corporation High speed CCITT decompressor
JPH04242860A (en) 1990-12-28 1992-08-31 Sony Corp Arithmetic device
US5184124A (en) * 1991-01-02 1993-02-02 Next Computer, Inc. Method and apparatus for compressing and storing pixels
GB2252002B (en) * 1991-01-11 1995-01-04 Sony Broadcast & Communication Compression of video signals
JPH04236664A (en) 1991-01-18 1992-08-25 Sony Corp Arithmetic circuit
US5301344A (en) 1991-01-29 1994-04-05 Analogic Corporation Multibus sequential processor to perform in parallel a plurality of reconfigurable logic operations on a plurality of data sets
US5055841A (en) * 1991-02-01 1991-10-08 Bell Communications Research, Inc. High-speed feedforward variable word length decoder
US5257213A (en) 1991-02-20 1993-10-26 Samsung Electronics Co., Ltd. Method and circuit for two-dimensional discrete cosine transform
US5111292A (en) * 1991-02-27 1992-05-05 General Electric Company Priority selection apparatus as for a video signal processor
US5168356A (en) * 1991-02-27 1992-12-01 General Electric Company Apparatus for segmenting encoded video signal for transmission
US5122875A (en) * 1991-02-27 1992-06-16 General Electric Company An HDTV compression system
US5333212A (en) 1991-03-04 1994-07-26 Storm Technology Image compression technique with regionally selective compression ratio
US5457482A (en) 1991-03-15 1995-10-10 Hewlett Packard Company Method and apparatus for utilizing off-screen memory as a simultaneously displayable channel
US5870497A (en) 1991-03-15 1999-02-09 C-Cube Microsystems Decoder for compressed video signals
US5421028A (en) 1991-03-15 1995-05-30 Hewlett-Packard Company Processing commands and data in a common pipeline path in a high-speed computer graphics system
JP2866754B2 (en) * 1991-03-27 1999-03-08 三菱電機株式会社 Arithmetic processing unit
US5203003A (en) 1991-03-28 1993-04-13 Echelon Corporation Computer architecture for conserving power by using shared resources and method for suspending processor execution in pipeline
US5164819A (en) * 1991-04-03 1992-11-17 Music John D Method and system for coding and compressing color video signals
US5287193A (en) 1991-04-10 1994-02-15 Industrial Technology Research Institute Parallel processing architecture of run-length codes
US5457780A (en) 1991-04-17 1995-10-10 Shaw; Venson M. System for producing a video-instruction set utilizing a real-time frame differential bit map and microblock subimages
US5182642A (en) 1991-04-19 1993-01-26 General Dynamics Lands Systems Inc. Apparatus and method for the compression and transmission of multiformat data
JP3109854B2 (en) 1991-04-23 2000-11-20 キヤノン株式会社 Image coding method and apparatus
US5212549A (en) 1991-04-29 1993-05-18 Rca Thomson Licensing Corporation Error concealment apparatus for a compressed video signal processing system
US5185819A (en) * 1991-04-29 1993-02-09 General Electric Company Video signal compression apparatus for independently compressing odd and even fields
US5146325A (en) * 1991-04-29 1992-09-08 Rca Thomson Licensing Corporation Video signal decompression apparatus for independently compressed even and odd field data
US5263136A (en) 1991-04-30 1993-11-16 Optigraphics Corporation System for managing tiled images using multiple resolutions
JP3532932B2 (en) 1991-05-20 2004-05-31 モトローラ・インコーポレイテッド Randomly accessible memory with time overlapping memory access
AU657510B2 (en) 1991-05-24 1995-03-16 Apple Inc. Improved image encoding/decoding method and apparatus
EP0514663A3 (en) 1991-05-24 1993-07-14 International Business Machines Corporation An apparatus and method for motion video encoding employing an adaptive quantizer
US5212742A (en) 1991-05-24 1993-05-18 Apple Computer, Inc. Method and apparatus for encoding/decoding image data
WO1992022029A1 (en) 1991-05-24 1992-12-10 British Technology Group Usa, Inc. Optimizing compiler for computers
JPH04354287A (en) 1991-05-30 1992-12-08 Sony Corp Image interpolation circuit
JP2569496Y2 (en) 1991-06-07 1998-04-22 日東工器株式会社 Diaphragm pump
US5228098A (en) * 1991-06-14 1993-07-13 Tektronix, Inc. Adaptive spatio-temporal compression/decompression of video image signals
US5184347A (en) 1991-07-09 1993-02-02 At&T Bell Laboratories Adaptive synchronization arrangement
US5404550A (en) 1991-07-25 1995-04-04 Tandem Computers Incorporated Method and apparatus for executing tasks by following a linked list of memory packets
US5254991A (en) 1991-07-30 1993-10-19 Lsi Logic Corporation Method and apparatus for decoding Huffman codes
GB2258781B (en) 1991-08-13 1995-05-03 Sony Broadcast & Communication Data compression
US5321806A (en) 1991-08-21 1994-06-14 Digital Equipment Corporation Method and apparatus for transmitting graphics command in a computer graphics system
JP3108479B2 (en) 1991-08-28 2000-11-13 株式会社リコー Encoding / decoding method and apparatus therefor
US5319460A (en) 1991-08-29 1994-06-07 Canon Kabushiki Kaisha Image signal processing device including frame memory
JP2507204B2 (en) 1991-08-30 1996-06-12 松下電器産業株式会社 Video signal encoder
US5309563A (en) 1991-09-09 1994-05-03 Compaq Computer Corporation Computer implemented method for transferring command messages between a system manager for a computer system and a network operating system associated therewith
US5168375A (en) * 1991-09-18 1992-12-01 Polaroid Corporation Image reconstruction by use of discrete cosine and related transforms
GB2260053B (en) 1991-09-27 1995-03-08 Sony Broadcast & Communication Image signal processing
US5261047A (en) 1991-10-29 1993-11-09 Xerox Corporation Bus arbitration scheme for facilitating operation of a printing apparatus
US5231484A (en) 1991-11-08 1993-07-27 International Business Machines Corporation Motion video compression system with adaptive bit allocation and quantization
US5214507A (en) * 1991-11-08 1993-05-25 At&T Bell Laboratories Video signal quantization for an mpeg like coding environment
US5257223A (en) 1991-11-13 1993-10-26 Hewlett-Packard Company Flip-flop circuit with controllable copying between slave and scan latches
US5227878A (en) * 1991-11-15 1993-07-13 At&T Bell Laboratories Adaptive coding and decoding of frames and fields of video
US5237413A (en) * 1991-11-19 1993-08-17 Scientific-Atlanta, Inc. Motion filter for digital television system
US5175617A (en) * 1991-12-04 1992-12-29 Vision Applications, Inc. Telephone line picture transmission
US5307180A (en) 1991-12-18 1994-04-26 Xerox Corporation Method and apparatus for controlling the processing of digital image signals
US5307449A (en) 1991-12-20 1994-04-26 Apple Computer, Inc. Method and apparatus for simultaneously rendering multiple scanlines
US5241222A (en) * 1991-12-20 1993-08-31 Eastman Kodak Company Dram interface adapter circuit
US5517603A (en) 1991-12-20 1996-05-14 Apple Computer, Inc. Scanline rendering device for generating pixel values for displaying three-dimensional graphical images
US5237432A (en) 1991-12-23 1993-08-17 Xerox Corporation Image scaling apparatus
US5159449A (en) 1991-12-26 1992-10-27 Workstation Technologies, Inc. Method and apparatus for data reduction in a video image data reduction system
US5446866A (en) 1992-01-30 1995-08-29 Apple Computer, Inc. Architecture for transferring pixel streams, without control information, in a plurality of formats utilizing addressable source and destination channels associated with the source and destination components
JPH06153069A (en) * 1992-10-30 1994-05-31 Sony Corp Converter, duplicating device, reproduction device and display device of image
US5225832A (en) * 1992-02-13 1993-07-06 Industrial Technology Research Institute High speed variable length decoder
US5490257A (en) 1992-02-24 1996-02-06 Advanced Micro Devices, Inc. RAM based FIFO memory half-full detection apparatus and method
US5357606A (en) 1992-02-25 1994-10-18 Apple Computer, Inc. Row interleaved frame buffer
JP3323950B2 (en) 1992-03-17 2002-09-09 サン・マイクロシステムズ・インコーポレーテッド Method of performing IDCT in a digital image processing system and IDCT processor therefor
US5333266A (en) 1992-03-27 1994-07-26 International Business Machines Corporation Method and apparatus for message handling in computer systems
WO1993020652A1 (en) 1992-04-01 1993-10-14 Intel Corporation Method and apparatus for compressing and decompressing a sequence of digital video images using sync frames
US5329313A (en) 1992-04-01 1994-07-12 Intel Corporation Method and apparatus for real time compression and decompression of a digital motion video signal using a fixed Huffman table
US5253058A (en) 1992-04-01 1993-10-12 Bell Communications Research, Inc. Efficient coding scheme for multilevel video transmission
KR0160610B1 (en) 1992-04-07 1999-01-15 강진구 Method and device for variable length image compression and decompression
US5287420A (en) 1992-04-08 1994-02-15 Supermac Technology Method for image compression on a personal computer
US5283646A (en) 1992-04-09 1994-02-01 Picturetel Corporation Quantizer control method and apparatus
GB2267194B (en) 1992-05-13 1995-10-04 Sony Broadcast & Communication Apparatus and method for processing image data
US5241383A (en) 1992-05-13 1993-08-31 Bell Communications Research, Inc. Pseudo-constant bit rate video coding with quantization parameter adjustment
US5305438A (en) 1992-05-19 1994-04-19 Sony Electronics Inc. Video storage, processing, and distribution system using recording format independent hierarchical storages and processors
US5369405A (en) 1992-05-19 1994-11-29 Goldstar Co., Ltd. Coefficient generation apparatus for variable length decoder
US5450599A (en) 1992-06-04 1995-09-12 International Business Machines Corporation Sequential pipelined processing for the compression and decompression of image data
US5289577A (en) * 1992-06-04 1994-02-22 International Business Machines Incorporated Process-pipeline architecture for image/video processing
KR940010433B1 (en) 1992-06-09 1994-10-22 대우전자 주식회사 Apparatus for decoding variable length code
US5276513A (en) 1992-06-10 1994-01-04 Rca Thomson Licensing Corporation Implementation architecture for performing hierarchical motion analysis of video images in real time
US5289276A (en) 1992-06-19 1994-02-22 General Electric Company Method and apparatus for conveying compressed video data over a noisy communication channel
JPH0695986A (en) 1992-06-19 1994-04-08 Westinghouse Electric Corp <We> Real-time data imaging network system and operating method thereof
US5276681A (en) 1992-06-25 1994-01-04 Starlight Networks Process for fair and prioritized access to limited output buffers in a multi-port switch
EP0575675B1 (en) 1992-06-26 1998-11-25 Discovision Associates Method and apparatus for transformation of signals from a frequency to a time domaine
US5842033A (en) 1992-06-30 1998-11-24 Discovision Associates Padding apparatus for passing an arbitrary number of bits through a buffer in a pipeline system
DE69229338T2 (en) 1992-06-30 1999-12-16 Discovision Ass Data pipeline system
US5412782A (en) 1992-07-02 1995-05-02 3Com Corporation Programmed I/O ethernet adapter with early interrupts for accelerating data transfer
US5253055A (en) 1992-07-02 1993-10-12 At&T Bell Laboratories Efficient frequency scalable video encoding with coefficient selection
US5287178A (en) 1992-07-06 1994-02-15 General Electric Company Reset control network for a video signal encoder
US5278620A (en) 1992-07-08 1994-01-11 Xerox Corporation Cleaning blade equipped with a vibration sensor
US5231486A (en) * 1992-07-27 1993-07-27 General Electric Company Data separation processing in a dual channel digital high definition television system
US5278647A (en) 1992-08-05 1994-01-11 At&T Bell Laboratories Video decoder using adaptive macroblock leak signals
FR2695278B1 (en) 1992-08-26 1994-10-14 Euro Cp Sarl Method for exchanging information, in particular between equipment in a room, and functional unit and installation relating thereto.
US5406279A (en) 1992-09-02 1995-04-11 Cirrus Logic, Inc. General purpose, hash-based technique for single-pass lossless data compression
US5301019A (en) 1992-09-17 1994-04-05 Zenith Electronics Corp. Data compression system having perceptually weighted motion vectors
US5351047A (en) 1992-09-21 1994-09-27 Laboratory Automation, Inc. Data decoding method and apparatus
US5294894A (en) 1992-10-02 1994-03-15 Compaq Computer Corporation Method of and apparatus for startup of a digital computer system clock
US5298992A (en) 1992-10-08 1994-03-29 International Business Machines Corporation System and method for frame-differencing based video compression/decompression with forward and reverse playback capability
US5300949A (en) 1992-10-22 1994-04-05 International Business Machines Corporation Scalable digital video decompressor
US5278520A (en) 1992-10-26 1994-01-11 Codex, Corp. Phase lock detection in a phase lock loop
US5329619A (en) 1992-10-30 1994-07-12 Software Ag Cooperative processing interface and communication broker for heterogeneous computing environments
US5497498A (en) 1992-11-05 1996-03-05 Giga Operations Corporation Video processing module using a second programmable logic device which reconfigures a first programmable logic device for data transformation
JP3007235B2 (en) 1992-11-10 2000-02-07 富士写真フイルム株式会社 Variable length code decompression device and compression / decompression device
US5420801A (en) 1992-11-13 1995-05-30 International Business Machines Corporation System and method for synchronization of multimedia streams
US5301272A (en) 1992-11-25 1994-04-05 Intel Corporation Method and apparatus for address space aliasing to identify pixel types
JP3255308B2 (en) 1992-12-18 2002-02-12 ソニー株式会社 Data playback device
US5517670A (en) 1992-12-30 1996-05-14 International Business Machines Corporation Adaptive data transfer channel employing extended data block capability
JP3257643B2 (en) * 1993-01-18 2002-02-18 ソニー株式会社 Image encoding device and image decoding device
GB2274956B (en) 1993-02-05 1997-04-02 Sony Broadcast & Communication Image data compression
US5396497A (en) 1993-02-26 1995-03-07 Sony Corporation Synchronization of audio/video information
US5298896A (en) 1993-03-15 1994-03-29 Bell Communications Research, Inc. Method and system for high order conditional entropy coding
US5410355A (en) 1993-04-02 1995-04-25 Rca Thomson Licensing Corporation Video signal processor including input codeword buffer for providing stored codewords to codeword priority analysis circuit
US5345408A (en) 1993-04-19 1994-09-06 Gi Corporation Inverse discrete cosine transform processor
US5572691A (en) 1993-04-21 1996-11-05 Gi Corporation Apparatus and method for providing multiple data streams from stored data using dual memory buffers
US5448310A (en) 1993-04-27 1995-09-05 Array Microsystems, Inc. Motion estimation coprocessor
US5486876A (en) 1993-04-27 1996-01-23 Array Microsystems, Inc. Video interface unit for mapping physical image data to logical tiles
US5699460A (en) * 1993-04-27 1997-12-16 Array Microsystems Image compression coprocessor with data flow control and multiple processing units
US5486864A (en) 1993-05-13 1996-01-23 Rca Thomson Licensing Corporation Differential time code method and apparatus as for a compressed video signal
EP0625858B1 (en) 1993-05-19 1998-06-24 Alcatel Video server memory management method
FR2705805B1 (en) 1993-05-27 1996-06-28 Sgs Thomson Microelectronics Image processing system.
US5304953A (en) 1993-06-01 1994-04-19 Motorola, Inc. Lock recovery circuit for a phase locked loop
EP0627835B1 (en) 1993-06-03 2001-08-29 International Business Machines Corporation Improved network layer packet structure
US5425061A (en) 1993-06-07 1995-06-13 Texas Instruments Incorporated Method and apparatus for bit stream synchronization
JP2560988B2 (en) 1993-07-16 1996-12-04 日本電気株式会社 Information processing apparatus and processing method
US5581280A (en) 1993-07-29 1996-12-03 Cirrus Logic, Inc. Video processing apparatus, systems and methods
US5598514A (en) 1993-08-09 1997-01-28 C-Cube Microsystems Structure and method for a multistandard video encoder/decoder
JP3247804B2 (en) 1993-08-17 2002-01-21 株式会社リコー Data compression method, data compression / decompression method, code word data number limiting device
US5568165A (en) 1993-10-22 1996-10-22 Auravision Corporation Video processing technique using multi-buffer video memory
US5398072A (en) 1993-10-25 1995-03-14 Lsi Logic Corporation Management of channel buffer in video decoders
US5452006A (en) 1993-10-25 1995-09-19 Lsi Logic Corporation Two-part synchronization scheme for digital video decoders
US5490247A (en) 1993-11-24 1996-02-06 Intel Corporation Video subsystem for computer-based conferencing system
US5528238A (en) 1993-11-24 1996-06-18 Intel Corporation Process, apparatus and system for decoding variable-length encoded signals
US5634025A (en) * 1993-12-09 1997-05-27 International Business Machines Corporation Method and system for efficiently fetching variable-width instructions in a data processing system having multiple prefetch units
CA2145363C (en) * 1994-03-24 1999-07-13 Anthony Mark Jones Ram interface
US5541595A (en) 1994-05-19 1996-07-30 Matsushita Electric Corporation Of America Variable length code decoder for simultaneous decoding the most significant bits and the least significant bits of a variable length code
US5477263A (en) 1994-05-26 1995-12-19 Bell Atlantic Network Services, Inc. Method and apparatus for video on demand with fast forward, reverse and channel pause
US5495291A (en) 1994-07-22 1996-02-27 Hewlett-Packard Company Decompression system for compressed video data for providing uninterrupted decompressed video data output
US5559999A (en) 1994-09-09 1996-09-24 Lsi Logic Corporation MPEG decoding system including tag list for associating presentation time stamps with encoded data units
US5566089A (en) 1994-10-26 1996-10-15 General Instrument Corporation Of Delaware Syntax parser for a video decompression processor
US5517250A (en) 1995-02-28 1996-05-14 General Instrument Corporation Of Delaware Acquisition of desired data from a packetized data stream and synchronization thereto

Patent Citations (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4236228A (en) * 1977-03-17 1980-11-25 Tokyo Shibaura Electric Co., Ltd. Memory device for processing picture images data
US4135242A (en) * 1977-11-07 1979-01-16 Ncr Corporation Method and processor having bit-addressable scratch pad memory
GB2039106A (en) * 1979-01-02 1980-07-30 Honeywell Inf Systems Number format conversion in computer Fig 3A
EP0075893A2 (en) * 1981-09-30 1983-04-06 Siemens Aktiengesellschaft Memory operand alignment circuit arrangement for decimal and logical instructions
US4677500A (en) * 1984-02-08 1987-06-30 U.S. Philips Corporation System for playing back a film recorded as a video signal on a disc-shaped record carrier
US4807028A (en) * 1986-11-10 1989-02-21 Kokusai Denshin Denwa Co., Ltd. Decoding device capable of producing a decoded video signal with a reduced delay
EP0280573A2 (en) * 1987-02-26 1988-08-31 Pioneer Electronic Corporation Device for reproducing still pictures with an audio portion
US4875196A (en) * 1987-09-08 1989-10-17 Sharp Microelectronic Technology, Inc. Method of operating data buffer apparatus
EP0321628A1 (en) * 1987-12-23 1989-06-28 International Business Machines Corporation Shared memory interface for a data processing system
US5089992A (en) * 1988-06-30 1992-02-18 Mitsubishi Denki Kabushiki Kaisha Semiconductor memory device and a data path using the same
DE3832563A1 (en) * 1988-09-24 1990-03-29 Bosch Gmbh Robert Memory arrangement for a narrow-band picture transmission device
EP0446956A2 (en) * 1990-03-16 1991-09-18 Fujitsu Limited An ATM transmission system with a variable transmission rate
US5319724A (en) * 1990-04-19 1994-06-07 Ricoh Corporation Apparatus and method for compressing still images
EP0460751A2 (en) * 1990-06-05 1991-12-11 Koninklijke Philips Electronics N.V. Method of transmitting audio and/or video signals
US5173695A (en) * 1990-06-29 1992-12-22 Bell Communications Research, Inc. High-speed flexible variable-length-code decoder
EP0468480A2 (en) * 1990-07-25 1992-01-29 Oki Electric Industry Co., Ltd. Synchronous burst-access memory and word-line driving circuit therefor
US5253053A (en) * 1990-12-31 1993-10-12 Apple Computer, Inc. Variable length decoding using lookup tables
EP0503956A2 (en) * 1991-03-15 1992-09-16 C-Cube Microsystems Decompression of video signal
EP0506294A2 (en) * 1991-03-28 1992-09-30 AT&T Corp. Hierarchical variable length decoder for digital video data
EP0542195A2 (en) * 1991-11-13 1993-05-19 Sony Corporation Interframe predicting method and picture signal coding/decoding apparatus
US5280349A (en) * 1992-02-13 1994-01-18 Industrial Technology Research Institute HDTV decoder
EP0562419A2 (en) * 1992-03-26 1993-09-29 General Instrument Corporation Of Delaware Variable length code word decoder for use in digital communication systems
US5293229A (en) * 1992-03-27 1994-03-08 Matsushita Electric Corporation Of America Apparatus and method for processing groups of fields in a video data compression system
EP0572263A2 (en) * 1992-05-28 1993-12-01 C-Cube Microsystems, Inc. Variable length code decoder for video decompression operations
EP0577329A2 (en) * 1992-07-02 1994-01-05 AT&T Corp. Timing recovery for variable bit-rate video on asynchronous transfer mode (ATM) networks
US5325092A (en) * 1992-07-07 1994-06-28 Ricoh Company, Ltd. Huffman decoder architecture for high speed operation and reduced memory
EP0587443A2 (en) * 1992-09-10 1994-03-16 Texas Instruments Incorporated Image processor and method
EP0618772A1 (en) * 1992-10-23 1994-10-12 Schaartec Gmbh & Co Kg Deboning device and method.
EP0600446A2 (en) * 1992-11-30 1994-06-08 Sony Corporation Decoder for a compressed digital video signal using a common memory for decoding and 2/3 pull-down conversion
EP0618728A2 (en) * 1993-02-26 1994-10-05 Sony Corporation Synchronization of audio/video information
US5561465A (en) * 1993-03-31 1996-10-01 U.S. Philips Corporation Video decoder with five page memory for decoding of intraframes, predicted frames and bidirectional frames
US5430485A (en) * 1993-09-30 1995-07-04 Thomson Consumer Electronics, Inc. Audio/video synchronization in a digital transmission system
EP0674266A2 (en) * 1994-03-24 1995-09-27 Discovision Associates Method and apparatus for interfacing with ram
EP0695095A2 (en) * 1994-07-29 1996-01-31 Discovision Associates Video decompression

Non-Patent Citations (16)

* Cited by examiner, † Cited by third party
Title
Chang, Shih Fu and David Messerschmitt, Design High Throughput VLC Decoder Part 1 Concurrent VLSI Architectures. IEEE Transactions on Circuits and Systems for Video Technology. vol. 2, No. 2 Jun. 1, 1992. pp 187 196. *
Chang, Shih-Fu and David Messerschmitt, "Design High-Throughput VLC Decoder Part 1-Concurrent VLSI Architectures." IEEE Transactions on Circuits and Systems for Video Technology. vol. 2, No. 2 Jun. 1, 1992. pp 187-196.
Goto, Junichi et al. "250-MHZ BICMOS Super-High-Speed Video Signal Processor ULSI" IEEE Journal of Solid State Circuits vol. 26 No. 12 Dec. 1991 pp 1876-1883.
Goto, Junichi et al. 250 MHZ BICMOS Super High Speed Video Signal Processor ULSI IEEE Journal of Solid State Circuits vol. 26 No. 12 Dec. 1991 pp 1876 1883. *
Jones, Fred "A New Era of Fast Dynamic Rams." IEEE Spectrum vol. 49 No. 10 Oct. 1992 pp 43-49.
Jones, Fred A New Era of Fast Dynamic Rams. IEEE Spectrum vol. 49 No. 10 Oct. 1992 pp 43 49. *
Macinnis, Alexander G. "The MPEG Systems Coding Specification." Signal Processing: Image Communication 4 (1992) pp 153-159.
Macinnis, Alexander G. The MPEG Systems Coding Specification. Signal Processing: Image Communication 4 (1992) pp 153 159. *
McCarthy, Charles L. "A Low-Cost Audio/Video Decoder Solution for MPEG System Streams." IEEE Jun. 21, 1994, pp 312-313.
McCarthy, Charles L. A Low Cost Audio/Video Decoder Solution for MPEG System Streams. IEEE Jun. 21, 1994, pp 312 313. *
Puri, A. R Aravind et al. "Video Coding with Motion-Compensated Interpolation for CD-ROM Applications." Signal Processing Image Communication vol. 2, No. 2 Aug. 1990 pp 127-144.
Puri, A. R Aravind et al. Video Coding with Motion Compensated Interpolation for CD ROM Applications. Signal Processing Image Communication vol. 2, No. 2 Aug. 1990 pp 127 144. *
Salters, R.H.W. "Fast DRAMS for Sharper TV". IEEE Spectrum vol. 29 No. 10 Oct. 1992, pp 40-42.
Salters, R.H.W. Fast DRAMS for Sharper TV . IEEE Spectrum vol. 29 No. 10 Oct. 1992, pp 40 42. *
Yang, Kun Min. VLSI Architecture Design of a Versatile Variable Length Decoding Chip for Real Time Video Codecs, Tencon 1990 IEEE Region 10 Conference on Computer and Communication . . . , IEEE Publications Feb. 1990, pp 551 554. *
Yang, Kun-Min. VLSI Architecture Design of a Versatile Variable Length Decoding Chip for Real-Time Video Codecs, Tencon 1990 IEEE Region 10 Conference on Computer and Communication . . . , IEEE Publications Feb. 1990, pp 551-554.

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7385949B1 (en) 2001-06-05 2008-06-10 Broadcom Corporation System and method for de-interleaving data in a wireless receiver
US20030044075A1 (en) * 2001-08-30 2003-03-06 Jae-Woo Roh Block coding/decoding method and apparatus for increasing code rate
US6954554B2 (en) * 2001-08-30 2005-10-11 Daewoo Electronics Corporation Block coding/decoding method and apparatus for increasing code rate
US7606081B1 (en) 2001-10-15 2009-10-20 Altera Corporation Device programmable to operate as a multiplexer, demultiplexer, or memory device
US7463544B1 (en) * 2001-10-15 2008-12-09 Altera Corporation Device programmable to operate as a multiplexer, demultiplexer, or memory device
US20030218921A1 (en) * 2002-02-28 2003-11-27 Peter Schrogmeier Latency time switch for an S-DRAM
US6804165B2 (en) * 2002-02-28 2004-10-12 Infineon Technologies Ag Latency time switch for an S-DRAM
US6853588B2 (en) * 2002-10-31 2005-02-08 Electronics And Telecommunications Research Institute First-in first-out memory circuit and method for executing same
US20040085817A1 (en) * 2002-10-31 2004-05-06 Youngwoo Kim First-in first-out memory circuit and method for executing same
US20090214175A1 (en) * 2003-07-03 2009-08-27 Mccrossan Joseph Recording medium, reproduction apparatus, recording method, integrated circuit, program, and reproduction method
US20090220211A1 (en) * 2003-07-03 2009-09-03 Mccrossan Joseph Recording medium, reproduction apparatus, recording method, integrated circuit, program, and reproduction method
US8280230B2 (en) 2003-07-03 2012-10-02 Panasonic Corporation Recording medium, reproduction apparatus, recording method, integrated circuit, program and reproduction method
US8369690B2 (en) 2003-07-03 2013-02-05 Panasonic Corporation Recording medium, reproduction apparatus, recording method, integrated circuit, program, and reproduction method
US8682146B2 (en) 2003-07-03 2014-03-25 Panasonic Corporation Recording medium, reproduction apparatus, recording method, integrated circuit, program, and reproduction method
US20050071582A1 (en) * 2003-09-26 2005-03-31 Samsung Electronics Co., Ltd. Circuits and methods for providing variable data I/O width for semiconductor memory devices
US7480776B2 (en) * 2003-09-26 2009-01-20 Samsung Electronics Co., Ltd. Circuits and methods for providing variable data I/O width for semiconductor memory devices
US20070190686A1 (en) * 2006-02-13 2007-08-16 Advanced Semiconductor Engineering, Inc. Method of fabricating substrate with embedded component therein

Also Published As

Publication number Publication date
CN1144434A (en) 1997-03-05
AU701335B2 (en) 1999-01-28
JPH08172624A (en) 1996-07-02
AU701335C (en) 2001-11-08
US5768629A (en) 1998-06-16
MXPA99001886A (en) 2004-09-07
US5984512A (en) 1999-11-16
US5740460A (en) 1998-04-14
EP0892556A3 (en) 2000-01-12
EP0891097A3 (en) 2000-01-12
EP0891096A2 (en) 1999-01-13
US5829007A (en) 1998-10-27
US5798719A (en) 1998-08-25
EP0695095A3 (en) 1999-03-17
US5878273A (en) 1999-03-02
EP0891098A2 (en) 1999-01-13
EP0891094A3 (en) 2000-01-12
EP0892556A2 (en) 1999-01-20
US6799246B1 (en) 2004-09-28
US6217234B1 (en) 2001-04-17
EP0891095A2 (en) 1999-01-13
EP0891098A3 (en) 2000-01-12
US5963154A (en) 1999-10-05
EP0891097A2 (en) 1999-01-13
EP0891099A3 (en) 2000-01-12
JP2001168728A (en) 2001-06-22
AU2501995A (en) 1996-02-08
US5821885A (en) 1998-10-13
US5995727A (en) 1999-11-30
US5703793A (en) 1997-12-30
EP0891094A2 (en) 1999-01-13
US5835792A (en) 1998-11-10
US5828907A (en) 1998-10-27
JP2001094437A (en) 2001-04-06
US5801973A (en) 1998-09-01
EP0891099A2 (en) 1999-01-13
SG108204A1 (en) 2005-01-28
CA2154962A1 (en) 1996-01-30
EP0695095A2 (en) 1996-01-31
EP0891095A3 (en) 2000-01-12
AU2400799A (en) 1999-06-24
EP0891096A3 (en) 2000-01-12
AU2400499A (en) 1999-06-03

Similar Documents

Publication Publication Date Title
US6141721A (en) Method of asynchronous memory access
US6119213A (en) Method for addressing data having variable data width using a fixed number of bits for address and width defining fields
US5379356A (en) Decompression processor for video applications
US5774206A (en) Process for controlling an MPEG decoder
US5815646A (en) Decompression processor for video applications
EP0674446A2 (en) Reconfigurable data processing stage
KR100291532B1 (en) An information processing system comprising a reconfigurable processing stage
KR100304511B1 (en) Video restoration and decoding system
JPH08228346A (en) Start cord detector
KR100270798B1 (en) Video dccompression
WO1996036178A1 (en) Multiple sequence mpeg decoder and process for controlling same

Legal Events

Date Code Title Description
STCF Information on status: patent grant

Free format text: PATENTED CASE

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FPAY Fee payment

Year of fee payment: 4

FPAY Fee payment

Year of fee payment: 8

REMI Maintenance fee reminder mailed
AS Assignment

Owner name: COASES INVESTMENTS BROS. L.L.C., DELAWARE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DISCOVISION ASSOCIATES;REEL/FRAME:022951/0518

Effective date: 20080402

FEPP Fee payment procedure

Free format text: PAYER NUMBER DE-ASSIGNED (ORIGINAL EVENT CODE: RMPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: INTELLECTUAL VENTURES DRAM 1 LLC, DELAWARE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:COASES INVESTMENTS BROS. L.L.C.;REEL/FRAME:026968/0832

Effective date: 20110921

AS Assignment

Owner name: TALON RESEARCH, LLC, TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:INTELLECTUAL VENTURES DRAM 1 LLC;REEL/FRAME:027427/0327

Effective date: 20110926

REMI Maintenance fee reminder mailed
FPAY Fee payment

Year of fee payment: 12

SULP Surcharge for late payment

Year of fee payment: 11