WO2006109988A1 - Method and apparatus for decoding video signal using reference pictures - Google Patents

Method and apparatus for decoding video signal using reference pictures Download PDF

Info

Publication number
WO2006109988A1
WO2006109988A1 PCT/KR2006/001341 KR2006001341W WO2006109988A1 WO 2006109988 A1 WO2006109988 A1 WO 2006109988A1 KR 2006001341 W KR2006001341 W KR 2006001341W WO 2006109988 A1 WO2006109988 A1 WO 2006109988A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
base layer
offset
current
frame
Prior art date
Application number
PCT/KR2006/001341
Other languages
French (fr)
Inventor
Byeong Moon Jeon
Seung Wook Park
Ji Ho Park
Doe Hyun Yoon
Original Assignee
Lg Electronics Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020050066622A external-priority patent/KR20060109247A/en
Application filed by Lg Electronics Inc. filed Critical Lg Electronics Inc.
Priority to KR1020077023436A priority Critical patent/KR100883603B1/en
Priority to JP2008506373A priority patent/JP2008536438A/en
Priority to EP06732851A priority patent/EP1880553A4/en
Publication of WO2006109988A1 publication Critical patent/WO2006109988A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H04N19/615Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding using motion compensated temporal filtering [MCTF]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103Selection of coding mode or of prediction mode
    • H04N19/105Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/187Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scalable video layer
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/63Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding using sub-band based transform, e.g. wavelets
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/70Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by syntax aspects related to video coding, e.g. related to compression standards
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/44Receiver circuitry for the reception of television signals according to analogue transmission standards
    • H04N5/445Receiver circuitry for the reception of television signals according to analogue transmission standards for displaying additional information
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]

Definitions

  • the present invention relates to scalable encoding and decoding of a video signal, and more particularly to a method ' and apparatus for encoding a video signal, wherein a base layer in the video signal is additionally used to code an enhanced layer in the video signal, and a method and apparatus for decoding such encoded video data.
  • Scalable Video Codec is a method which encodes video into a sequence of pictures with the highest image quality while ensuring that part of the encoded picture sequence (specifically, a partial sequence of frames intermittently selected from the total sequence of frames) can also be decoded and used to represent the video with a low image quality.
  • Motion Compensated Temporal Filtering MCTF is an encoding scheme that has been suggested for use in the scalable video codec.
  • auxiliary picture sequence for low bitrates for example, a sequence of pictures that have a small screen size and/or a low frame rate, so that each decoder can select and decode a sequence suitable for its capabilities and characteristics.
  • One example is to encode and transmit not only a main picture sequence of 4CIF (Common Intermediate Format) but also an auxiliary picture sequence of CIF and an auxiliary picture sequence of QCIF (Quarter CIF) to decoders.
  • Each sequence is referred to as a layer, and the higher of two given layers is referred to as an enhanced layer and the lower is referred to as a base layer.
  • Such picture sequences have redundancy since the same video signal source is encoded into the sequences.
  • To increase the coding efficiency of each sequence there is a need to reduce the amount of coded information of the higher sequence by performing inter-sequence picture prediction of video frames in the higher sequence from video frames in the lower sequence temporally coincident with the video frames in the higher sequence.
  • video frames in sequences of different layers may have different aspect ratios.
  • video frames of the higher sequence i.e. , the enhanced layer
  • video frames of the lower sequence i.e., the base layer
  • the present invention relates to decoding a video signal .
  • At least a portion of a current image in a current layer is predicted based on at least a portion of a reference image, offset information and dimension information.
  • the offset information may indicate a position offset between at least one boundary pixel of the reference image and at least one boundary pixel of the current image
  • the dimension information may indicate at least one dimension of the current image .
  • the reference image is based on a base image in a base layer.
  • the reference image may be at least an up-sampled portion of the base image.
  • the offset information includes left offset information indicating a position offset between at least one left side pixel of the reference image and at least one left side pixel of the current image.
  • the offset information includes top offset information indicating a position offset between at least one top side pixel of the reference image and at least one top side pixel of the current image.
  • the dimension information includes width information indicating a width of the current image.
  • the dimension information includes height information indicating a height of the current image.
  • the offset information may be obtained from a header for at least a portion of a picture (e.g., a slice, frame, etc.) in the current layer. Also, it maybe determined that the offset information is present based on an indicator in the header.
  • a picture e.g., a slice, frame, etc.
  • Other embodiments include apparatuses for decoding a video signal .
  • FIG. 1 is a block diagram of a video signal encoding apparatus to which a scalable video signal coding method according to the present invention is applied;
  • FIG. 2 is a block diagram of part of an MCTF encoder shown in FIG. 1 responsible for carrying out image estimation/prediction and update operations;
  • FIGS . 3a and 3b illustrate the relationship between enhanced layer frames and base layer frames which can be used as reference frames for converting an enhanced layer frame to an H frame having a predictive image
  • FIG. 4 illustrates how part of a base layer picture is selected and enlarged to be used for a prediction operation of an enhanced layer picture according to an embodiment of the present invention
  • FIGS. 5a and 5b illustrate embodiments of the structure of information regarding a positional relationship of a base layer picture to an enhanced layer picture, which is transmitted to the decoder, according to the present invention
  • FIG. 6 illustrates how an area including a base layer picture is enlarged to be used for a prediction operation of an enhanced layer picture according to another embodiment of the present invention
  • FIG. 7 illustrates how a base layer picture is enlarged to a larger area than an enhanced layer picture so as to be used for a prediction operation of the enhanced layer picture according to yet another embodiment of the present invention
  • FIG. 8 is a block diagram of an apparatus for decoding a data stream encoded by the apparatus of FIG. 1;
  • FIG. 9 is a block diagram of part of an MCTF decoder shown in FIG. 8 responsible for carrying out inverse prediction and update operations.
  • FIG. 1 is a block diagram of a video signal encoding apparatus to which a scalable video signal coding method according to the present invention is applied.
  • the apparatus of FIG. 1 is implemented to code an input video signal in two layers, principles of the present invention described below can also be applied when a video signal is coded in three or more layers.
  • the present invention can also be applied to any scalable video coding scheme, without being limited to an MCTF scheme which is described below as an example.
  • the video signal encoding apparatus shown in FIG. 1 comprises an MCTF encoder 100 to which the present invention is applied, a texture coding unit 110, a motion coding unit 120, a base layer encoder 150, and a muxer (or multiplexer) 130.
  • the MCTF encoder 100 is an enhanced layer encoder which encodes an input video signal on a per macroblock basis according to an MCTF scheme and generates suitable management information.
  • the texture coding unit 110 converts information of encoded macroblocks into a compressed bitstream.
  • the motion coding unit 120 codes motion vectors of image blocks obtained by the MCTF encoder 100 into a compressed bitstream according to a specified scheme.
  • the base layer encoder 150 encodes an input video signal according to a specified scheme, for example, according to the MPEG-I, 2 or 4 standard or the H.261, H.263 or H.264 standard, and produces a small -screen picture sequence, for example, a sequence of pictures scaled down to 25% of their original size.
  • the muxer 130 encapsulates the output data of the texture coding unit 110, the small-screen picture sequence output from the base layer encoder 150, and the output vector data of the motion coding unit 120 into a desired format.
  • the muxer 130 then multiplexes and outputs the encapsulated data into a desired transmission format.
  • the base layer encoder 150 can provide a low-bitrate data stream not only by encoding an input video signal into a sequence of pictures having a smaller screen size than pictures of the enhanced layer, but also by encoding an input video signal into a sequence of pictures having the same screen size as pictures of the enhanced layer at a lower frame rate than the enhanced layer.
  • the base layer is encoded into a small -screen picture sequence, and the small-screen picture sequence is referred to as a base layer sequence and the frame sequence output from the MCTF encoder 100 is referred to as an enhanced layer sequence.
  • the MCTF encoder 100 performs motion estimation and prediction operations on each target macroblock in a video frame.
  • the MCTF encoder 100 also performs an update operation for each target macroblock by adding an image difference of the target macroblock from a corresponding macroblock in a neighbor frame to the corresponding macroblock in the neighbor frame.
  • FIG. 2 illustrates some elements of the MCTF encoder 100 for carrying out these operations.
  • the elements of the MCTF encoder 100 shown in FIG. 2 include an estimator/predictor 102, an updater 103, and a decoder 105.
  • the decoder 105 decodes an encoded stream received from the base layer encoder 150, and enlarges decoded small-screen frames to the size of frames in the enhanced layer using an internal sealer 105a.
  • the estimator/predictor 102 searches for a reference block of each macroblock in a current frame, which is to be coded into residual data, in adjacent frames prior to or subsequent to the current frame and in frames enlarged by the sealer 105a.
  • the estimator/predictor 102 then obtains an image difference (i.e., a pixel-to-pixel difference) of each macroblock in the current frame from the reference block or from a corresponding block in a temporally coincident frame enlarged by the sealer 105a, and codes the image difference into the macroblock.
  • the estimator/predictor 102 also obtains a motion vector originating from the macroblock and extending to the reference block.
  • the updater 103 performs an update operation for a macroblock in the current frame, whose reference block has been found in frames prior to or subsequent to the current frame, by multiplying the image difference of the macroblock by an appropriate constant (for example, 1/2 or 1/4) and adding the resulting value to the reference block.
  • the operation carried out by the updater 103 is referred to as a 'U' operation, and a frame produced by the 1 U' operation is referred to as an 'L 1 frame.
  • the estimator/predictor 102 and the updater 103 of FIG. 2 may perform their operations on a plurality of slices, which are produced by dividing a single frame, simultaneously and in parallel instead of performing their operations on the video frame.
  • a frame (or slice) having an image difference, which is produced by the estimator/predictor 102, is referred to as an 1 H' frame
  • the 1 H' frame (or slice) contains data having high frequency components of the video signal .
  • the term 'picture 1 is used to indicate a slice or a frame, provided that the use of the term is technically feasible.
  • the estimator/predictor 102 divides each of the input video frames (or L frames obtained at the previous level) into macroblocks of a desired size. For each divided macroblock, the estimator/predictor 102 searches for a block, whose image is most similar to that of each divided macroblock, in previous/next neighbor frames of the enhanced layer and/or in base layer frames enlarged by the sealer 105a. That is, the estimator/predictor 102 searches for a macroblock temporally correlated with each divided macroblock. A block having the most similar image to a target image block has the smallest image difference from the target image block. The image difference of two image blocks is defined, for example, as the sum or average of pixel-to-pixel differences of the two image blocks.
  • a block having the smallest image difference from the target macroblock is referred to as a reference block.
  • a picture including the reference block is referred to as a reference picture.
  • two reference blocks may be present in a frame
  • the estimator/predictor 102 calculates and outputs a motion vector from the current block to the reference block.
  • the estimator/predictor "102 also calculates and outputs pixel error values (i.e., pixel difference values) of the current block from pixel values of the reference block, which is present in either the prior frame or the subsequent frame, or from average pixel values of the two reference blocks, which are present in the prior and subsequent frames.
  • pixel error values i.e., pixel difference values
  • the image or pixel difference values are also referred top as residual data.
  • the estimator/predictor 102 determines whether or not a frame in the same time zone as the current frame (hereinafter also referred to as a "temporally coincident frame") or a frame in a close time zone to the current frame (hereinafter also referred to as a “temporally close frame”) is present in the base layer sequence.
  • the estimator/predictor 102 obtains the image difference (i.e., residual data) of the current macroblock from a corresponding macroblock in the temporally coincident or close frame based on pixel values of the two macroblocks, and does not obtain a motion vector of the current macroblock with respect to the corresponding macroblock.
  • the close time zone to the current frame corresponds to a time interval including frames that can be regarded as having the same image as the current frame . Information of this time interval is carried within an encoded stream.
  • the above operation of the estimator/predictor 102 is referred to as a 'P' operation.
  • the estimator/predictor 102 performs the 'P' operation to produce an H frame by searching for a reference block of each macroblock in the current frame and coding each macroblock into residual data
  • the estimator/predictor 102 can selectively use, as reference pictures, enlarged pictures of the base layer received from the sealer l ' 05a, in addition to neighbor L frames of the enhanced layer prior to and subsequent to the current frame, as shown in FIG. 3a.
  • FIG. 3b shows five frames that can be used to produce an H frame.
  • a current L frame 400L has L frames 401 prior to and L frames 402 subsequent to the current L frame 400L.
  • the current L frame 400L also has a base layer frame 405 in the same time zone.
  • One or two frames from among the L frames 401 and 402 in the same MCTF level as a current L frame 400L, the frame 405 of the base layer in the same time zone as the L frame 400L, and base layer frames 403 and 404 prior to and subsequent to the frame 405 are used as reference pictures to produce an H frame 400H from the current L frame 400L.
  • a picture of the base layer is selected as a reference picture for prediction of a picture of the enhanced layer in the reference picture selection method as shown in FIG. 3b
  • all or part of the base layer picture can be used for prediction of the enhanced layer picture.
  • a base layer picture has an aspect ratio of 4:3
  • an actual image portion 502 of the base layer picture has an aspect ratio of 16:9
  • an enhanced layer picture 500 has an aspect ratio of 16:9
  • upper and lower horizontal portions 501a and 501b of the base layer picture contain invalid data.
  • only the image portion 502 of the base layer picture is used for prediction of the enhanced layer picture 500.
  • the sealer 105a selects (or crops) the image portion 502 of the base layer picture (S41) , up-samples the selected image portion 502 to enlarge it to the size of the enhanced layer picture 500 (S42) , and provides the enlarged image portion to the estimator/predictor 102.
  • the MCTF encoder 100 incorporates position information of the selected portion of the base layer picture into a header of the current picture coded into residual data.
  • the MCTF encoder 100 also sets and inserts a flag "flag_base_layer_cropping" , which indicates that part of the base layer picture has been selected and used, in the picture header at an appropriate position so that the flag is delivered to the decoder.
  • the position information is not transmitted when the flag "flag_base_layer_cropping" is reset.
  • FIGS. 5a and 5b illustrate embodiments of the structure of information regarding a selected portion 512 of a base layer picture.
  • the selected portion 512 of the base layer picture is specified by offsets (left_offset , right_offset , top_offset, and bottom_offset) from the left, right, top and bottom boundaries of the base layer picture.
  • the left offset indicates a position offset between left side pixels (or, for example, at least one pixel) in the base layer image and left side pixels in the selected portion 512.
  • the top offset indicates a position offset between top side pixels (or, for example, at least one pixel) in the base layer image and top side pixels in the selected portion 512.
  • the right offset indicates a position offset between right side pixels (or, for example, at least one pixel) in the base layer image and right side pixels in the selected portion 512.
  • the bottom side offset indicates a position offset between bottom side pixels (or, for example, at least one pixel) in the base layer image and bottom side pixels in the selected portion 512.
  • the selected portion 512 of the base layer picture is specified by offsets
  • the offsets in the information of the selected portion shown in FIGS. 5a and 5b may have negative values.
  • a base layer picture has an aspect ratio of 4:3
  • an enhanced layer picture 600 has an aspect ratio of 16:9
  • an actual image portion of the picture has an aspect ratio of 4:3
  • the left and right offset values have negative values -d L and -d R .
  • Portions 601a and 601b extended from the base layer picture are specified by the negative values -d L and -d R .
  • the extended portions 601a and 601b are padded with offscreen data, and a picture 610 including the extended portions 601a and 601b is upsampled to have the same size as that of the enhanced layer picture 600. Accordingly, data of an area 611 in the enlarged base layer picture, which corresponds to an actual image portion of the enhanced layer picture 600, can be used for prediction of the actual image portion of the enhanced layer picture 600.
  • the same advantages as described above in the example of FIG. 4 can be achieved by using the information of FIGS. 5a and 5b as position information of an area overlapping with the enhanced layer picture, which is to be associated with the enlarged base layer picture, instead of using the information of FIGS. 5a and 5b for specifying the selected area in the base layer picture.
  • a base layer picture 702 when a base layer picture 702 is upsampled so that an actual image area 701 of the base layer picture 702 is enlarged to the size of an enhanced layer picture 700, the enlarged (e.g., up-sampled) picture corresponds to an area larger than the enhanced layer picture 700.
  • top and bottom offsets top_offset and bottom_offset are included in the position information of an area overlapping with the enhanced layer picture 700. These offsets correspond to the enlarged base layer picture, and are assigned negative values -d ⁇ and -d B so that only an actual image area of the enlarged base layer picture is used for prediction of the enhanced layer picture 700.
  • left and right offsets of the position information of the area corresponding to the enlarged base layer picture are zero.
  • the left and right offsets may be non-zero, and also correspond to the enlarged base layer picture.
  • a portion of the image in the enlarged base layer picture may not be used in determining the enhanced layer picture.
  • the offset information corresponds to the base layer picture, as opposed to the up-sample base layer picture, a portion of the image in the base layer picture may not be used in determining the enhanced layer picture.
  • the left offset indicates a position offset between left side pixels (or, for example, at least one pixel) in the up-sampled base layer image and left side pixels in the enhanced layer image.
  • the top offset indicates a position offset between top side pixels (or, for example, at least one pixel) in the up-sampled base layer image and top side pixels in the enhanced layer image.
  • the right offset indicates a position offset between right side pixels (or, for example, at least one pixel) in the up-sampled base layer image and right side pixels in the enhanced layer image.
  • the bottom side offset indicates a position offset between bottom side pixels (or, for example, at least one pixel) in the up-sampled base layer image and bottom side pixels in the enhanced layer image.
  • the information of FIGS. 5a and 5b can be used as information for selection of a portion of a base layer picture, which is to be used for prediction of an enhanced layer picture, or can be used as position information of an area overlapping with an enhanced layer picture, which is to be associated with a base layer picture for use in prediction of the enhanced layer picture .
  • Information of the size and aspect ratio of the base layer picture, mode information of an actual image of the base layer picture, etc. can be determined by decoding, for example, from a sequence header of the encoded base layer stream. Namely, the information may be recorded in the sequence header of the encoded base layer stream. Accordingly, the position of an area overlapping with the enhanced layer picture, which corresponds to the base layer picture or the selected area in the base layer picture described above, are determined based on position or offset information, and all or part of the base layer picture is used to suit this determination.
  • the MCTF encoder 100 generates a sequence of H frames and a sequence of L frames, respectively, by performing the 'P' and 1 U' operations described above on a certain-length sequence of pictures, for example, on a group of pictures (GOP) . Then, an estimator/predictor and an updater at a next serially-connected stage (not shown) generates a sequence of H frames and a sequence of L frames by repeating the 'P' and
  • the 'P' and 'U' operations are performed an appropriate number of times (for example, until one L frame is produced per GOP) to produce a final enhanced layer sequence .
  • FIG. 8 is a block diagram of an apparatus for decoding a data stream encoded by the apparatus of FIG. 1.
  • the decoding apparatus of FIG. 8 includes a demuxer (or demultiplexer) 200, a texture decoding unit 210, a motion decoding unit 220, an MCTF decoder 230, and a base layer decoder 240.
  • the demuxer 200 separates a received data stream into a compressed motion vector stream, a compressed macroblock information stream, and a base layer stream.
  • the texture decoding unit 210 reconstructs the compressed macroblock information stream to its original uncompressed state.
  • the motion decoding unit 220 reconstructs the compressed motion vector stream to its original uncompressed state.
  • the MCTF decoder 230 is an enhanced layer decoder which converts the uncompressed macroblock information stream and the uncompressed motion vector stream back to an original video signal according to an MCTF scheme.
  • the base layer decoder 240 decodes the base layer stream according to a specified scheme, for example, according to the MPEG-4 or H.264 standard.
  • the MCTF decoder 230 includes, as an internal element, an inverse filter that has a structure as shown in FIG. 9 for reconstructing an input stream to its original frame sequence.
  • FIG. 9 shows some elements of the inverse filter for reconstructing a sequence of H and L frames of MCTF level N to a sequence of L frames of level N-I.
  • the elements of the inverse filter of FIG. 9 include an inverse updater 231, an inverse predictor 232, a motion vector decoder 235, an arranger 234, and a sealer 230a.
  • the inverse updater 231 subtracts pixel difference values of input H frames from corresponding pixel values of input L frames.
  • the inverse predictor 232 reconstructs input H frames to frames having original images with reference to the L frames, from which the image differences of the H frames have been subtracted, and/or with reference to enlarged pictures output from the sealer 240a.
  • the motion vector decoder 235 decodes an input motion vector stream into motion vector information of each 5 block and provides the motion vector information to an inverse predictor (for example, the inverse predictor 232) of each stage.
  • the arranger 234 interleaves the frames completed by the inverse predictor 232 between the L frames output from the inverse updater 231, thereby producing a normal video frame sequence.
  • 10 230a enlarges small -screen pictures of the base layer to the enhanced layer picture size, for example, according to the information as shown in FIGS. 5a and 5b.
  • the L frames output from the arranger 234 constitute an L frame sequence 601 of level N-I.
  • level N-I reconstructs the L frame sequence 601 and an input H frame sequence 602 of level N-I to an L frame sequence.
  • This decoding process is performed the same number of times as the number of MCTF levels employed in the encoding procedure, thereby reconstructing an original video frame
  • the inverse predictor 232 specifies an L frame of the enhanced layer and/or an enlarged frame of the base layer which has been used as a reference frame to code the macroblock to
  • the inverse predictor 232 determines a reference block in the specified frame based on a motion vector provided from the motion vector decoder 235, and then adds pixel values of the reference block (or average pixel values of two macroblocks used as reference blocks of the macroblock) to pixel difference
  • the sealer 230a selects and enlarges an area in the base layer picture (in the example of FIG. 4) or enlarges a larger area than the base layer picture (in the example of FIG. 6) based on positional relationship information as shown in FIGS. 5a and 5b included in a header analyzed by the MCTF decoder 230 so that the enlarged area of the base layer picture is used for reconstructing macroblocks containing residual data in the current H frame to original image blocks as described above .
  • the positional relationship information is extracted from the header and is then referred to when information indicating whether or not the positional relationship information is included (specifically, the flag "flag_base_layer_cropping" in the example of FIGS . 5a and 5b) indicates that the positional relationship information is included.
  • the inverse predictor 232 uses an enlarged one of the base layer picture received from the sealer 230a for prediction of the enhanced layer picture by associating the entirety of the enlarged base layer picture with all or part of the current H frame or with a larger area than the current H frame according to the values (positive or negative) of the offset information.
  • the inverse predictor 232 uses an enlarged one of the base layer picture received from the sealer 230a for prediction of the enhanced layer picture by associating the entirety of the enlarged base layer picture with all or part of the current H frame or with a larger area than the current H frame according to the values (positive or negative) of the offset information.
  • the predictor 232 uses only an area in the enlarged base layer picture, which corresponds to the H frame, for reconstructing macroblocks in the current H frame to their original images .
  • the offset information included negative values.
  • the MCTF decoding is performed in specified units, for example, in units of slices in a parallel fashion, so that the macroblocks in the frame have their original images reconstructed and the reconstructed macroblocks are then combined to constitute a complete video frame.
  • the above decoding method reconstructs an MCTF-encoded data stream to a complete video frame sequence.
  • the decoding apparatus decodes and outputs a base layer sequence or decodes and outputs an enhanced layer sequence using the base layer depending on its processing and presentation capabilities.
  • the decoding apparatus described above may be incorporated into a mobile communication terminal, a media player, or the like.
  • a method and apparatus for encoding/decoding a video signal uses pictures of a base layer provided for low-performance decoders, in addition to pictures of an enhanced layer, when encoding a video signal in a scalable fashion, so that the total amount of coded data is reduced, thereby increasing coding efficiency.
  • part of a base layer picture which can be used for a prediction operation of an enhanced layer picture, is specified so that the prediction operation can be performed normally without performance degradation even when a picture enlarged from the base layer picture cannot be directly used for the prediction operation of the enhanced layer picture.

Abstract

In the method for decoding a video signal, at least a portion of a current image in a current layer is predicted based on at least a portion of a reference image, offset information and dimension information. The offset information may indicate a position offset between at least one boundary pixel of the reference image and at least one boundary pixel of the current image, and the dimension information may indicate at least one dimension of the current image.

Description

D E S C R I P T I O N
METHOD AND APPARATUS FOR DECODING VIDEO SIGNAL USING REFERENCE PICTURES
1 . Technical Field The present invention relates to scalable encoding and decoding of a video signal, and more particularly to a method 'and apparatus for encoding a video signal, wherein a base layer in the video signal is additionally used to code an enhanced layer in the video signal, and a method and apparatus for decoding such encoded video data.
2. Background Art
Scalable Video Codec (SVC) is a method which encodes video into a sequence of pictures with the highest image quality while ensuring that part of the encoded picture sequence (specifically, a partial sequence of frames intermittently selected from the total sequence of frames) can also be decoded and used to represent the video with a low image quality. Motion Compensated Temporal Filtering (MCTF) is an encoding scheme that has been suggested for use in the scalable video codec.
Although it is possible to represent low image-quality video by receiving and processing part of the sequence of pictures encoded in a scalable fashion as described above, there is still a problem in that the image quality is significantly reduced if the bitrate is lowered. One solution to this problem is to hierarchically provide an auxiliary picture sequence for low bitrates, for example, a sequence of pictures that have a small screen size and/or a low frame rate, so that each decoder can select and decode a sequence suitable for its capabilities and characteristics. One example is to encode and transmit not only a main picture sequence of 4CIF (Common Intermediate Format) but also an auxiliary picture sequence of CIF and an auxiliary picture sequence of QCIF (Quarter CIF) to decoders. Each sequence is referred to as a layer, and the higher of two given layers is referred to as an enhanced layer and the lower is referred to as a base layer.
Such picture sequences have redundancy since the same video signal source is encoded into the sequences. To increase the coding efficiency of each sequence, there is a need to reduce the amount of coded information of the higher sequence by performing inter-sequence picture prediction of video frames in the higher sequence from video frames in the lower sequence temporally coincident with the video frames in the higher sequence. However, video frames in sequences of different layers may have different aspect ratios. For example, video frames of the higher sequence (i.e. , the enhanced layer) may have a wide aspect ratio of 16:9, whereas video frames of the lower sequence (i.e., the base layer) may have a narrow aspect ratio of 4:3. In this case, there is a need to determine which part of a base layer picture is to be used for an enhanced layer picture or for which part of the enhanced layer picture the base layer picture is to be used when performing prediction of the enhanced layer picture.
3. Disclosure of Invention
The present invention relates to decoding a video signal .
In one embodiment of the method for decoding a video signal ,
In the method for decoding a video signal, at least a portion of a current image in a current layer is predicted based on at least a portion of a reference image, offset information and dimension information. The offset information may indicate a position offset between at least one boundary pixel of the reference image and at least one boundary pixel of the current image, and the dimension information may indicate at least one dimension of the current image .
In one embodiment, the reference image is based on a base image in a base layer. For example, the reference image may be at least an up-sampled portion of the base image.
In one embodiment, the offset information includes left offset information indicating a position offset between at least one left side pixel of the reference image and at least one left side pixel of the current image. In another embodiment, the offset information includes top offset information indicating a position offset between at least one top side pixel of the reference image and at least one top side pixel of the current image.
In one embodiment, the dimension information includes width information indicating a width of the current image.
In another embodiment, the dimension information includes height information indicating a height of the current image.
In one embodiment, the offset information may be obtained from a header for at least a portion of a picture (e.g., a slice, frame, etc.) in the current layer. Also, it maybe determined that the offset information is present based on an indicator in the header.
Other embodiments include apparatuses for decoding a video signal .
4. Brief Description of Drawings
The above and other objects, features and other advantages of the present invention will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a block diagram of a video signal encoding apparatus to which a scalable video signal coding method according to the present invention is applied; FIG. 2 is a block diagram of part of an MCTF encoder shown in FIG. 1 responsible for carrying out image estimation/prediction and update operations;
FIGS . 3a and 3b illustrate the relationship between enhanced layer frames and base layer frames which can be used as reference frames for converting an enhanced layer frame to an H frame having a predictive image;
FIG. 4 illustrates how part of a base layer picture is selected and enlarged to be used for a prediction operation of an enhanced layer picture according to an embodiment of the present invention;
FIGS. 5a and 5b illustrate embodiments of the structure of information regarding a positional relationship of a base layer picture to an enhanced layer picture, which is transmitted to the decoder, according to the present invention;
FIG. 6 illustrates how an area including a base layer picture is enlarged to be used for a prediction operation of an enhanced layer picture according to another embodiment of the present invention; FIG. 7 illustrates how a base layer picture is enlarged to a larger area than an enhanced layer picture so as to be used for a prediction operation of the enhanced layer picture according to yet another embodiment of the present invention;
FIG. 8 is a block diagram of an apparatus for decoding a data stream encoded by the apparatus of FIG. 1; and
FIG. 9 is a block diagram of part of an MCTF decoder shown in FIG. 8 responsible for carrying out inverse prediction and update operations.
5. Modes for Carrying out the Invention
Example embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
FIG. 1 is a block diagram of a video signal encoding apparatus to which a scalable video signal coding method according to the present invention is applied. Although the apparatus of FIG. 1 is implemented to code an input video signal in two layers, principles of the present invention described below can also be applied when a video signal is coded in three or more layers. The present invention can also be applied to any scalable video coding scheme, without being limited to an MCTF scheme which is described below as an example.
The video signal encoding apparatus shown in FIG. 1 comprises an MCTF encoder 100 to which the present invention is applied, a texture coding unit 110, a motion coding unit 120, a base layer encoder 150, and a muxer (or multiplexer) 130. The MCTF encoder 100 is an enhanced layer encoder which encodes an input video signal on a per macroblock basis according to an MCTF scheme and generates suitable management information. The texture coding unit 110 converts information of encoded macroblocks into a compressed bitstream. The motion coding unit 120 codes motion vectors of image blocks obtained by the MCTF encoder 100 into a compressed bitstream according to a specified scheme. The base layer encoder 150 encodes an input video signal according to a specified scheme, for example, according to the MPEG-I, 2 or 4 standard or the H.261, H.263 or H.264 standard, and produces a small -screen picture sequence, for example, a sequence of pictures scaled down to 25% of their original size. The muxer 130 encapsulates the output data of the texture coding unit 110, the small-screen picture sequence output from the base layer encoder 150, and the output vector data of the motion coding unit 120 into a desired format. The muxer 130 then multiplexes and outputs the encapsulated data into a desired transmission format. The base layer encoder 150 can provide a low-bitrate data stream not only by encoding an input video signal into a sequence of pictures having a smaller screen size than pictures of the enhanced layer, but also by encoding an input video signal into a sequence of pictures having the same screen size as pictures of the enhanced layer at a lower frame rate than the enhanced layer. In the embodiments of the present invention described below, the base layer is encoded into a small -screen picture sequence, and the small-screen picture sequence is referred to as a base layer sequence and the frame sequence output from the MCTF encoder 100 is referred to as an enhanced layer sequence.
The MCTF encoder 100 performs motion estimation and prediction operations on each target macroblock in a video frame. The MCTF encoder 100 also performs an update operation for each target macroblock by adding an image difference of the target macroblock from a corresponding macroblock in a neighbor frame to the corresponding macroblock in the neighbor frame. FIG. 2 illustrates some elements of the MCTF encoder 100 for carrying out these operations.
The elements of the MCTF encoder 100 shown in FIG. 2 include an estimator/predictor 102, an updater 103, and a decoder 105. The decoder 105 decodes an encoded stream received from the base layer encoder 150, and enlarges decoded small-screen frames to the size of frames in the enhanced layer using an internal sealer 105a. The estimator/predictor 102 searches for a reference block of each macroblock in a current frame, which is to be coded into residual data, in adjacent frames prior to or subsequent to the current frame and in frames enlarged by the sealer 105a. The estimator/predictor 102 then obtains an image difference (i.e., a pixel-to-pixel difference) of each macroblock in the current frame from the reference block or from a corresponding block in a temporally coincident frame enlarged by the sealer 105a, and codes the image difference into the macroblock. The estimator/predictor 102 also obtains a motion vector originating from the macroblock and extending to the reference block. The updater 103 performs an update operation for a macroblock in the current frame, whose reference block has been found in frames prior to or subsequent to the current frame, by multiplying the image difference of the macroblock by an appropriate constant (for example, 1/2 or 1/4) and adding the resulting value to the reference block. The operation carried out by the updater 103 is referred to as a 'U' operation, and a frame produced by the 1U' operation is referred to as an 'L1 frame.
The estimator/predictor 102 and the updater 103 of FIG. 2 may perform their operations on a plurality of slices, which are produced by dividing a single frame, simultaneously and in parallel instead of performing their operations on the video frame.
A frame (or slice) having an image difference, which is produced by the estimator/predictor 102, is referred to as an 1H' frame
(or slice) . The 1H' frame (or slice) contains data having high frequency components of the video signal . In the following description of the embodiments, the term 'picture1 is used to indicate a slice or a frame, provided that the use of the term is technically feasible.
The estimator/predictor 102 divides each of the input video frames (or L frames obtained at the previous level) into macroblocks of a desired size. For each divided macroblock, the estimator/predictor 102 searches for a block, whose image is most similar to that of each divided macroblock, in previous/next neighbor frames of the enhanced layer and/or in base layer frames enlarged by the sealer 105a. That is, the estimator/predictor 102 searches for a macroblock temporally correlated with each divided macroblock. A block having the most similar image to a target image block has the smallest image difference from the target image block. The image difference of two image blocks is defined, for example, as the sum or average of pixel-to-pixel differences of the two image blocks. Of blocks having a threshold image difference or less from a target macroblock in the current frame, a block having the smallest image difference from the target macroblock is referred to as a reference block. A picture including the reference block is referred to as a reference picture. For each macroblock of the current frame, two reference blocks (or two reference pictures) may be present in a frame
(including a base layer frame) prior to the current frame, in a frame (including a base layer frame) subsequent thereto, or one in a prior frame and one in a subsequent frame .
If the reference block is found, the estimator/predictor 102 calculates and outputs a motion vector from the current block to the reference block. The estimator/predictor "102 also calculates and outputs pixel error values (i.e., pixel difference values) of the current block from pixel values of the reference block, which is present in either the prior frame or the subsequent frame, or from average pixel values of the two reference blocks, which are present in the prior and subsequent frames. The image or pixel difference values are also referred top as residual data.
If no macroblock having a desired threshold image difference or less from the current macroblock is found in the two neighbor frames (including base layer frames) via the motion estimation operation, the estimator/predictor 102 determines whether or not a frame in the same time zone as the current frame (hereinafter also referred to as a "temporally coincident frame") or a frame in a close time zone to the current frame (hereinafter also referred to as a "temporally close frame") is present in the base layer sequence. If such a frame is present in the base layer sequence, the estimator/predictor 102 obtains the image difference (i.e., residual data) of the current macroblock from a corresponding macroblock in the temporally coincident or close frame based on pixel values of the two macroblocks, and does not obtain a motion vector of the current macroblock with respect to the corresponding macroblock. The close time zone to the current frame corresponds to a time interval including frames that can be regarded as having the same image as the current frame . Information of this time interval is carried within an encoded stream.
The above operation of the estimator/predictor 102 is referred to as a 'P' operation. When the estimator/predictor 102 performs the 'P' operation to produce an H frame by searching for a reference block of each macroblock in the current frame and coding each macroblock into residual data, the estimator/predictor 102 can selectively use, as reference pictures, enlarged pictures of the base layer received from the sealer l'05a, in addition to neighbor L frames of the enhanced layer prior to and subsequent to the current frame, as shown in FIG. 3a.
In an example embodiment of the present invention, five frames are used to produce each H frame. FIG. 3b shows five frames that can be used to produce an H frame. As shown, a current L frame 400L has L frames 401 prior to and L frames 402 subsequent to the current L frame 400L. The current L frame 400L also has a base layer frame 405 in the same time zone. One or two frames from among the L frames 401 and 402 in the same MCTF level as a current L frame 400L, the frame 405 of the base layer in the same time zone as the L frame 400L, and base layer frames 403 and 404 prior to and subsequent to the frame 405 are used as reference pictures to produce an H frame 400H from the current L frame 400L. As will be appreciated from the above discussion, there are various reference block selection modes. To inform the decoder of which mode is employed, the MCTF encoder 100 transmits
'reference block selection mode' information to the texture coding unit 110 after inserting/writing it into a field at a specified position of a header area of a corresponding macroblock.
When a picture of the base layer is selected as a reference picture for prediction of a picture of the enhanced layer in the reference picture selection method as shown in FIG. 3b, all or part of the base layer picture can be used for prediction of the enhanced layer picture. For example, as shown in Fig. 4, when a base layer picture has an aspect ratio of 4:3, an actual image portion 502 of the base layer picture has an aspect ratio of 16:9, and an enhanced layer picture 500 has an aspect ratio of 16:9, upper and lower horizontal portions 501a and 501b of the base layer picture contain invalid data. In this case, only the image portion 502 of the base layer picture is used for prediction of the enhanced layer picture 500. To accomplish this, the sealer 105a selects (or crops) the image portion 502 of the base layer picture (S41) , up-samples the selected image portion 502 to enlarge it to the size of the enhanced layer picture 500 (S42) , and provides the enlarged image portion to the estimator/predictor 102.
The MCTF encoder 100 incorporates position information of the selected portion of the base layer picture into a header of the current picture coded into residual data. The MCTF encoder 100 also sets and inserts a flag "flag_base_layer_cropping" , which indicates that part of the base layer picture has been selected and used, in the picture header at an appropriate position so that the flag is delivered to the decoder. The position information is not transmitted when the flag "flag_base_layer_cropping" is reset.
FIGS. 5a and 5b illustrate embodiments of the structure of information regarding a selected portion 512 of a base layer picture. In the embodiment of FIG. 5a, the selected portion 512 of the base layer picture is specified by offsets (left_offset , right_offset , top_offset, and bottom_offset) from the left, right, top and bottom boundaries of the base layer picture. The left offset indicates a position offset between left side pixels (or, for example, at least one pixel) in the base layer image and left side pixels in the selected portion 512. The top offset indicates a position offset between top side pixels (or, for example, at least one pixel) in the base layer image and top side pixels in the selected portion 512. The right offset indicates a position offset between right side pixels (or, for example, at least one pixel) in the base layer image and right side pixels in the selected portion 512. The bottom side offset indicates a position offset between bottom side pixels (or, for example, at least one pixel) in the base layer image and bottom side pixels in the selected portion 512. In the embodiment of FIG. 5b, the selected portion 512 of the base layer picture is specified by offsets
(left_offset and top_offset) from the left and top boundaries of the base layer picture and by the width and height (crop_width and crop_height) of the selected portion 512. Various other specifying methods are also possible.
The offsets in the information of the selected portion shown in FIGS. 5a and 5b may have negative values. For example, as shown in Fig. 6, when a base layer picture has an aspect ratio of 4:3, an enhanced layer picture 600 has an aspect ratio of 16:9, and an actual image portion of the picture has an aspect ratio of 4:3, the left and right offset values (left_offset and right_offset) have negative values -dL and -dR. Portions 601a and 601b extended from the base layer picture are specified by the negative values -dL and -dR. The extended portions 601a and 601b are padded with offscreen data, and a picture 610 including the extended portions 601a and 601b is upsampled to have the same size as that of the enhanced layer picture 600. Accordingly, data of an area 611 in the enlarged base layer picture, which corresponds to an actual image portion of the enhanced layer picture 600, can be used for prediction of the actual image portion of the enhanced layer picture 600.
Since the offset fields of the information illustrated in FIGS. 5a and 5b may have negative values, the same advantages as described above in the example of FIG. 4 can be achieved by using the information of FIGS. 5a and 5b as position information of an area overlapping with the enhanced layer picture, which is to be associated with the enlarged base layer picture, instead of using the information of FIGS. 5a and 5b for specifying the selected area in the base layer picture.
Specifically, with reference to Fig. 7, when a base layer picture 702 is upsampled so that an actual image area 701 of the base layer picture 702 is enlarged to the size of an enhanced layer picture 700, the enlarged (e.g., up-sampled) picture corresponds to an area larger than the enhanced layer picture 700. In this example, top and bottom offsets top_offset and bottom_offset are included in the position information of an area overlapping with the enhanced layer picture 700. These offsets correspond to the enlarged base layer picture, and are assigned negative values -dτ and -dB so that only an actual image area of the enlarged base layer picture is used for prediction of the enhanced layer picture 700. In the example of FIG. 7, left and right offsets of the position information of the area corresponding to the enlarged base layer picture are zero. However, it will be understood that the left and right offsets may be non-zero, and also correspond to the enlarged base layer picture. It will also be appreciated that a portion of the image in the enlarged base layer picture may not be used in determining the enhanced layer picture. Similarly, when the offset information corresponds to the base layer picture, as opposed to the up-sample base layer picture, a portion of the image in the base layer picture may not be used in determining the enhanced layer picture.
Furthermore, in this embodiment, the left offset indicates a position offset between left side pixels (or, for example, at least one pixel) in the up-sampled base layer image and left side pixels in the enhanced layer image. The top offset indicates a position offset between top side pixels (or, for example, at least one pixel) in the up-sampled base layer image and top side pixels in the enhanced layer image. The right offset indicates a position offset between right side pixels (or, for example, at least one pixel) in the up-sampled base layer image and right side pixels in the enhanced layer image. The bottom side offset indicates a position offset between bottom side pixels (or, for example, at least one pixel) in the up-sampled base layer image and bottom side pixels in the enhanced layer image.
As described above, the information of FIGS. 5a and 5b can be used as information for selection of a portion of a base layer picture, which is to be used for prediction of an enhanced layer picture, or can be used as position information of an area overlapping with an enhanced layer picture, which is to be associated with a base layer picture for use in prediction of the enhanced layer picture .
Information of the size and aspect ratio of the base layer picture, mode information of an actual image of the base layer picture, etc., can be determined by decoding, for example, from a sequence header of the encoded base layer stream. Namely, the information may be recorded in the sequence header of the encoded base layer stream. Accordingly, the position of an area overlapping with the enhanced layer picture, which corresponds to the base layer picture or the selected area in the base layer picture described above, are determined based on position or offset information, and all or part of the base layer picture is used to suit this determination.
Returning to Figs. 1 and 2, the MCTF encoder 100 generates a sequence of H frames and a sequence of L frames, respectively, by performing the 'P' and 1U' operations described above on a certain-length sequence of pictures, for example, on a group of pictures (GOP) . Then, an estimator/predictor and an updater at a next serially-connected stage (not shown) generates a sequence of H frames and a sequence of L frames by repeating the 'P' and
'U' operations on the generated L frame sequence. The 'P' and 'U' operations are performed an appropriate number of times (for example, until one L frame is produced per GOP) to produce a final enhanced layer sequence .
The data stream encoded in the method described above is transmitted by wire or wirelessly to a decoding apparatus or is delivered via recording media. The decoding apparatus reconstructs the original video signal in the enhanced and/or base layer according to the method described below. FIG. 8 is a block diagram of an apparatus for decoding a data stream encoded by the apparatus of FIG. 1. The decoding apparatus of FIG. 8 includes a demuxer (or demultiplexer) 200, a texture decoding unit 210, a motion decoding unit 220, an MCTF decoder 230, and a base layer decoder 240. The demuxer 200 separates a received data stream into a compressed motion vector stream, a compressed macroblock information stream, and a base layer stream. The texture decoding unit 210 reconstructs the compressed macroblock information stream to its original uncompressed state. The motion decoding unit 220 reconstructs the compressed motion vector stream to its original uncompressed state. The MCTF decoder 230 is an enhanced layer decoder which converts the uncompressed macroblock information stream and the uncompressed motion vector stream back to an original video signal according to an MCTF scheme. The base layer decoder 240 decodes the base layer stream according to a specified scheme, for example, according to the MPEG-4 or H.264 standard.
The MCTF decoder 230 includes, as an internal element, an inverse filter that has a structure as shown in FIG. 9 for reconstructing an input stream to its original frame sequence. FIG. 9 shows some elements of the inverse filter for reconstructing a sequence of H and L frames of MCTF level N to a sequence of L frames of level N-I. The elements of the inverse filter of FIG. 9 include an inverse updater 231, an inverse predictor 232, a motion vector decoder 235, an arranger 234, and a sealer 230a. The inverse updater 231 subtracts pixel difference values of input H frames from corresponding pixel values of input L frames. The inverse predictor 232 reconstructs input H frames to frames having original images with reference to the L frames, from which the image differences of the H frames have been subtracted, and/or with reference to enlarged pictures output from the sealer 240a. The motion vector decoder 235 decodes an input motion vector stream into motion vector information of each 5 block and provides the motion vector information to an inverse predictor (for example, the inverse predictor 232) of each stage. The arranger 234 interleaves the frames completed by the inverse predictor 232 between the L frames output from the inverse updater 231, thereby producing a normal video frame sequence. The sealer
10 230a enlarges small -screen pictures of the base layer to the enhanced layer picture size, for example, according to the information as shown in FIGS. 5a and 5b.
The L frames output from the arranger 234 constitute an L frame sequence 601 of level N-I. A next-stage inverse updater
15 and predictor of level N-I reconstructs the L frame sequence 601 and an input H frame sequence 602 of level N-I to an L frame sequence. This decoding process is performed the same number of times as the number of MCTF levels employed in the encoding procedure, thereby reconstructing an original video frame
20 sequence. With reference to ' reference_selection_code ' information carried in a header of each macroblock of an input H frame, the inverse predictor 232 specifies an L frame of the enhanced layer and/or an enlarged frame of the base layer which has been used as a reference frame to code the macroblock to
25 residual data. The inverse predictor 232 determines a reference block in the specified frame based on a motion vector provided from the motion vector decoder 235, and then adds pixel values of the reference block (or average pixel values of two macroblocks used as reference blocks of the macroblock) to pixel difference
30 values of the macroblock of the H frame; thereby reconstructing the original image of the macroblock of the H frame .
When a base layer picture has been used as a reference frame of a current H frame, the sealer 230a selects and enlarges an area in the base layer picture (in the example of FIG. 4) or enlarges a larger area than the base layer picture (in the example of FIG. 6) based on positional relationship information as shown in FIGS. 5a and 5b included in a header analyzed by the MCTF decoder 230 so that the enlarged area of the base layer picture is used for reconstructing macroblocks containing residual data in the current H frame to original image blocks as described above . The positional relationship information is extracted from the header and is then referred to when information indicating whether or not the positional relationship information is included (specifically, the flag "flag_base_layer_cropping" in the example of FIGS . 5a and 5b) indicates that the positional relationship information is included.
In the case where the information of FIGS. 5a and 5b has been used as information indicating the position of an area overlapping with an enhanced layer picture, to use in prediction of the enhanced layer picture, the inverse predictor 232 uses an enlarged one of the base layer picture received from the sealer 230a for prediction of the enhanced layer picture by associating the entirety of the enlarged base layer picture with all or part of the current H frame or with a larger area than the current H frame according to the values (positive or negative) of the offset information. In the case of FIG. 7 where the enlarged base layer picture is associated with a larger area than the current H frame, the predictor 232 uses only an area in the enlarged base layer picture, which corresponds to the H frame, for reconstructing macroblocks in the current H frame to their original images . In this example, the offset information included negative values.
For one H frame, the MCTF decoding is performed in specified units, for example, in units of slices in a parallel fashion, so that the macroblocks in the frame have their original images reconstructed and the reconstructed macroblocks are then combined to constitute a complete video frame. The above decoding method reconstructs an MCTF-encoded data stream to a complete video frame sequence. The decoding apparatus decodes and outputs a base layer sequence or decodes and outputs an enhanced layer sequence using the base layer depending on its processing and presentation capabilities.
The decoding apparatus described above may be incorporated into a mobile communication terminal, a media player, or the like.
As is apparent from the above description, a method and apparatus for encoding/decoding a video signal according to the present invention uses pictures of a base layer provided for low-performance decoders, in addition to pictures of an enhanced layer, when encoding a video signal in a scalable fashion, so that the total amount of coded data is reduced, thereby increasing coding efficiency. In addition, part of a base layer picture, which can be used for a prediction operation of an enhanced layer picture, is specified so that the prediction operation can be performed normally without performance degradation even when a picture enlarged from the base layer picture cannot be directly used for the prediction operation of the enhanced layer picture.
Although this invention has been described with reference to the example embodiments, it will be apparent to those skilled in the art that various improvements, modifications, replacements, and additions can be made in the invention without departing from the scope and spirit of the invention. Thus, it is intended that the invention cover the improvements, modifications, replacements, and additions of the invention.

Claims

C L A I M S
1. A method for decoding a video signal, comprising: predicting at least a portion of a current image in a current layer based on at least a portion of a reference image, offset information and dimension information, the offset information indicating a position offset between at least one boundary pixel of the reference image and at least one boundary pixel of the current image, and the dimension information indicating at least one dimension of the current image.
2. The method of claim 1, wherein the reference image is based on a base image in a base layer.
3. The method of claim 2, wherein the reference image is at least an up-sampled portion of the base image.
4. The method of claim 1, wherein the offset information includes left offset information indicating a position offset between at least one left side pixel of the reference image and at least one left side pixel of the current image.
5. The method of claim 4, wherein the reference image is based on a base image in a base layer.
6. The method of claim 5, wherein the reference image is at least an up-sampled portion of the base image.
7. The method of claim 4, wherein the offset information includes top offset information indicating a position offset between at least one top side pixel of the reference image and at least one top side pixel of the current image.
8. The method of claim 7, wherein the reference image is based on a base image in a base layer.
9. The method of claim 8, wherein the reference image is at least an up-sampled portion of the base image.
10. The method of claim 7, wherein the dimension information includes width information indicating a width of the current image .
11. The method of claim 7, wherein the dimension information includes height information indicating a height of the current image .
12. The method of claim 7, wherein the dimension information includes width information indicating a width of the current image and height information indicating a height of the current image.
13. The method of claim 1, wherein the offset information includes top offset information indicating a top position offset between at least one top side pixel of the reference image and at least one top side pixel of the current image.
14. The method of claim 13, wherein the reference image is based on a base image in a base layer.
15. The method of claim 14, wherein the reference image is at least an up-sampled portion of the base image.
16. The method of claim 1, wherein the dimension information includes width information indicating a width of the current image .
17. The method of claim 1, wherein the dimension information includes height information indicating a height of the current image .
18. The method of claim 1, wherein the dimension information includes width information indicating a width of the current image and height information indicating a height of the current image.
19. The method of claim 1, wherein the predicting step obtains the offset information from a header for at least a portion of a picture in the current layer.
20. The method of claim 19, wherein the predicting step determines that the offset information is present based on an indicator in the header.
21. The method of claim 20, wherein the picture is a slice.
22. The method of claim 20, wherein the picture is a frame.
23. The method of claim 1, further comprising: up-sampling at least a portion of a base image in a base layer to produce an up-sampled image as the reference image.
24. The method of claim 23, wherein the offset information includes left offset information indicating a position offset between at least one left side pixel of the reference image and at least one left side pixel of the current image, and top offset information indicating a top position offset between at least one top side pixel of the reference image and at least one top side pixel of the current image ; and the dimension information includes width information indicating a width of the current image and height information indicating a height of the current image.
25. The method of claim 24, wherein the predicting step obtains the offset information from a header for at least a portion of a pictur e in the current layer.
26. An apparatus for decoding a video signal, comprising: a decoder predicting at least a portion of a current image in a current layer based on at least a portion of a reference image, offset information and dimension information, the offset information indicating a position offset between at least one boundary pixel of the reference image and at least one boundary pixel of the current image, and the dimension information indicating at least one dimension of the current image.
PCT/KR2006/001341 2005-04-13 2006-04-12 Method and apparatus for decoding video signal using reference pictures WO2006109988A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
KR1020077023436A KR100883603B1 (en) 2005-04-13 2006-04-12 Method and apparatus for decoding video signal using reference pictures
JP2008506373A JP2008536438A (en) 2005-04-13 2006-04-12 Method and apparatus for decoding video signal using reference picture
EP06732851A EP1880553A4 (en) 2005-04-13 2006-04-12 Method and apparatus for decoding video signal using reference pictures

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US67067605P 2005-04-13 2005-04-13
US60/670,676 2005-04-13
KR10-2005-0066622 2005-07-22
KR1020050066622A KR20060109247A (en) 2005-04-13 2005-07-22 Method and apparatus for encoding/decoding a video signal using pictures of base layer

Publications (1)

Publication Number Publication Date
WO2006109988A1 true WO2006109988A1 (en) 2006-10-19

Family

ID=37087227

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/KR2006/001341 WO2006109988A1 (en) 2005-04-13 2006-04-12 Method and apparatus for decoding video signal using reference pictures
PCT/KR2006/001339 WO2006109986A1 (en) 2005-04-13 2006-04-12 Method and apparatus for encoding/decoding video signal using reference pictures

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/KR2006/001339 WO2006109986A1 (en) 2005-04-13 2006-04-12 Method and apparatus for encoding/decoding video signal using reference pictures

Country Status (3)

Country Link
EP (2) EP1880553A4 (en)
KR (2) KR100883603B1 (en)
WO (2) WO2006109988A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2015083575A1 (en) * 2013-12-06 2015-06-11 シャープ株式会社 Image decoding device, image encoding device, and encoded data conversion device
US9357218B2 (en) 2010-05-04 2016-05-31 Lg Electronics Inc. Method and apparatus for processing a video signal

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101455083B (en) * 2006-03-24 2012-04-11 韩国电子通信研究院 Coding method of reducing interlayer redundancy using mition data of fgs layer and device thereof
US9451284B2 (en) 2011-10-10 2016-09-20 Qualcomm Incorporated Efficient signaling of reference picture sets
KR101513707B1 (en) * 2011-12-14 2015-04-20 삼성전기주식회사 Method for taking 3D image and camera module using thereof
US20160286225A1 (en) * 2012-09-27 2016-09-29 Dolby Laboratories Licensing Corporation Inter-layer reference picture processing for coding standard scalability
US9584808B2 (en) * 2013-02-22 2017-02-28 Qualcomm Incorporated Device and method for scalable coding of video information
US10284842B2 (en) * 2013-03-05 2019-05-07 Qualcomm Incorporated Inter-layer reference picture construction for spatial scalability with different aspect ratios
KR102553316B1 (en) * 2015-03-06 2023-07-10 한국전자통신연구원 Apparatus for generating broadcasting signal frame using layered division multiplexing and method using the same
KR102553322B1 (en) * 2015-04-20 2023-07-10 한국전자통신연구원 Apparatus for generating broadcasting signal frame using layered division multiplexing and method using the same

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5973739A (en) * 1992-03-27 1999-10-26 British Telecommunications Public Limited Company Layered video coder
US6510177B1 (en) * 2000-03-24 2003-01-21 Microsoft Corporation System and method for layered video coding enhancement
US6535559B2 (en) * 1997-04-01 2003-03-18 Sony Corporation Image encoder, image encoding method, image decoder, image decoding method, and distribution media
US20040114689A1 (en) * 2002-12-13 2004-06-17 Huipin Zhang Wavelet based multiresolution video representation with spatially scalable motion vectors

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6549575B1 (en) * 1996-11-07 2003-04-15 International Business Machines Corporation. Efficient, flexible motion estimation architecture for real time MPEG2 compliant encoding
US6788347B1 (en) * 1997-03-12 2004-09-07 Matsushita Electric Industrial Co., Ltd. HDTV downconversion system
KR100343766B1 (en) * 1999-06-08 2002-07-20 마쯔시다덴기산교 가부시키가이샤 Video signal shuffling, encoding, decoding apparatus and program recording medium
US7020195B1 (en) * 1999-12-10 2006-03-28 Microsoft Corporation Layered coding and decoding of image data

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5973739A (en) * 1992-03-27 1999-10-26 British Telecommunications Public Limited Company Layered video coder
US6535559B2 (en) * 1997-04-01 2003-03-18 Sony Corporation Image encoder, image encoding method, image decoder, image decoding method, and distribution media
US6510177B1 (en) * 2000-03-24 2003-01-21 Microsoft Corporation System and method for layered video coding enhancement
US20040114689A1 (en) * 2002-12-13 2004-06-17 Huipin Zhang Wavelet based multiresolution video representation with spatially scalable motion vectors

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP1880553A4 *

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9357218B2 (en) 2010-05-04 2016-05-31 Lg Electronics Inc. Method and apparatus for processing a video signal
US9363520B2 (en) 2010-05-04 2016-06-07 Lg Electronics Inc. Method and apparatus for processing a video signal
US9936202B2 (en) 2010-05-04 2018-04-03 Lg Electronics Inc. Method and apparatus for processing a video signal
WO2015083575A1 (en) * 2013-12-06 2015-06-11 シャープ株式会社 Image decoding device, image encoding device, and encoded data conversion device
CN105981386A (en) * 2013-12-06 2016-09-28 华为技术有限公司 Image decoding device, image encoding device, and encoded data conversion device
US10142653B2 (en) 2013-12-06 2018-11-27 Huawei Technologies Co., Ltd. Image decoding apparatus, image coding apparatus, and coded data transformation apparatus
CN105981386B (en) * 2013-12-06 2019-02-26 华为技术有限公司 Picture decoding apparatus, picture coding device and coded data converting device

Also Published As

Publication number Publication date
KR100883603B1 (en) 2009-02-13
WO2006109986A1 (en) 2006-10-19
KR20080000588A (en) 2008-01-02
EP1880553A4 (en) 2011-03-02
EP1880552A1 (en) 2008-01-23
KR100883602B1 (en) 2009-02-13
KR20080000587A (en) 2008-01-02
EP1880553A1 (en) 2008-01-23
EP1880552A4 (en) 2011-03-09

Similar Documents

Publication Publication Date Title
US7746933B2 (en) Method and apparatus for encoding/decoding video signal using reference pictures
US9288486B2 (en) Method and apparatus for scalably encoding and decoding video signal
US7924917B2 (en) Method for encoding and decoding video signals
US7787540B2 (en) Method for scalably encoding and decoding video signal
US8755434B2 (en) Method and apparatus for scalably encoding and decoding video signal
US8660180B2 (en) Method and apparatus for scalably encoding and decoding video signal
WO2006109988A1 (en) Method and apparatus for decoding video signal using reference pictures
US20060133482A1 (en) Method for scalably encoding and decoding video signal
US20060133677A1 (en) Method and apparatus for performing residual prediction of image block when encoding/decoding video signal
EP1878250A1 (en) Method for scalably encoding and decoding video signal
US20080008241A1 (en) Method and apparatus for encoding/decoding a first frame sequence layer based on a second frame sequence layer
WO2006104365A1 (en) Method for scalably encoding and decoding video signal
EP1878248A1 (en) Method for scalably encoding and decoding video signal
US20070223573A1 (en) Method and apparatus for encoding/decoding a first frame sequence layer based on a second frame sequence layer
US20070280354A1 (en) Method and apparatus for encoding/decoding a first frame sequence layer based on a second frame sequence layer
US20070242747A1 (en) Method and apparatus for encoding/decoding a first frame sequence layer based on a second frame sequence layer
WO2006104363A1 (en) Method for scalably encoding and decoding video signal

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 200680017250.X

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: 1020077023436

Country of ref document: KR

ENP Entry into the national phase

Ref document number: 2008506373

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2006732851

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: RU

WWP Wipo information: published in national office

Ref document number: 2006732851

Country of ref document: EP