US20080159387A1

US20080159387A1 - Entropy deficiency based image

Info

Publication number: US20080159387A1
Application number: US11/987,639
Authority: US
Inventors: Ira Dvir; Nitzan Rabinowitz
Original assignee: Human Monitoring Ltd
Current assignee: Human Monitoring Ltd
Priority date: 2007-01-03
Filing date: 2007-12-03
Publication date: 2008-07-03
Also published as: US8019167B2; US20110317931A1; WO2008081461A2; WO2008081461A3; EP2116058A2; WO2008081460A2; WO2008081458A2; WO2008081460A3; KR20090116728A; US20080159639A1; WO2008081458A3; JP2010515397A; US8467617B2; EP2116057A2

Abstract

A method for obtaining a quantization factor for image compression by quantization of coefficients of a transformation of an image or part thereof, comprising determining an entropy-related sensitivity to quantization of at least a part of the image; and determining a quantization factor based on the entropy-related sensitivity.

Description

RELATED APPLICATIONS

The present application claims the benefit under 35 USC 119(e) of U.S. Provisional Patent Application No. 60/878,062 filed on Jan. 3, 2007 entitled “A SYSTEM AND APPARATUS FOR ENTROPY DEFICIENCY DCT BASED COMPRESSION-DECOMPRESSION OF IMAGES”, and U.S. Provisional Patent Application No. 60/878,063 filed on Jan. 3, 2007 entitled “A SYSTEM AND APPARATUS FOR COMPRESSING HIGH RESOLUTION STILL IMAGES OVER LOWER RESOLUTION VIDEO ENCODERS, FOR HANDSETS, CAMERAS, CAMCORDERS, AND CAMERA EQUIPPED PORTABLE MEDIA PLAYERS”; the present application relates also to U.S. patent application Ser. No. 11/882,811 filed on Aug. 6, 2007 entitled “COMPRESSING HIGH RESOLUTION IMAGES IN A LOW RESOLUTION VIDEO”. The disclosures of the above-mentioned applications are incorporated herein by reference.

FIELD OF THE INVENTION

This invention relates to image compression. Some embodiments relate to methods for image compression responsive to the sensitivity to quantization of an image or part thereof.

BACKGROUND OF THE INVENTION

Image compression is an application frequently employed in cameras, mobile handsets or personal computers, usually with respect to storage or transmission capacity.
While a good compression ratio is desirable for reducing storage space or communications rate, it is also desirable that the compressed image provides a good, or at least sufficient, visual quality. Generally there is a tradeoff between the compression ratio and the visual quality of the compressed image.
Some contemporary image compression techniques are based on quantization, i.e., a reduction of the range of values the image or its transformation comprises, allowing an effective compression. For example, JPEG and MPEG utilize a quantization of the coefficients of a DCT transform, JPEG2000 utilizes a quantization of wavelet transforms. Therefore, with missing or reduced elements the quality of a compressed image is inferior, at least to a certain extent, relative to the original image.
Beyond a human judgment, some metrics were devised to assess the quality of a compressed image. These metrics typically relate to a statistical deviation between the original image and the reconstructed image, for example, the commonly used PSNR (peak-signal-to-noise-ratio). Recently some more complicated metrics were developed, for example VQM (visual quality measures) and SSIM (structural similarity). Basically, all these metrics are evaluated after the compression.
Some approaches for image compression that try to improve the visual quality of the compressed image are known. For example, using variable sizes of pixels (e.g. U.S. Pat. No. 5,107,345), or using different levels of quantization to edges and surfaces (e.g. U.S. Pat. No. 5,793,892), or scalable DCT based compression schemes (e.g. U.S. Pat. No. 6,826,232, U.S. Pat. No. 7,020,342, U.S. Pat. No. 6,853,318), the disclosures of all of which patents are incorporated herein by reference.

SUMMARY OF THE INVENTION

A broad aspect of exemplary embodiments of the invention relates to a method for image compression by quantization that achieves high compression ratio while maintaining good visual quality which, at least typically, may be better than other contemporary methods utilized by JPEG or MPEG, AVC or a video inter-intra compression.
In exemplary embodiments of the invention, the high compression ratio is obtained by high quantization of regions with high complexity without sacrificing substantial details, and on the other hand, using low quantification for low complexity regions, preserving the gradually varying shades (such as faces, sky, walls, etc.).
In exemplary embodiments of the invention, the quantization is based on measure of the responsiveness, or sensitivity, of a group of pixels to quantization.
In exemplary embodiments of the invention, the responsiveness to quantization of a group of pixels is defined as the change in entropy of a quantized group of pixel relative to the entropy of the original pixels, or a derivation thereof. Preferably, without limitation, the group of pixels is quantized by a small factor relative to the range of the pixels values. The relative change in entropy due to such a small quantization resembles a differential of the entropy with respect to quantization.
It was found, quite unexpectedly, that a plot of the responsiveness to quantization of a group of pixels with respect to a complexity measure of the group of pixels (such as the standard mean deviation) exhibits a characteristic distribution. Additionally, using over a thousand examples, it was found that the plot resembles and approximates an exponential function. Additionally, it was found that the exponential function is characterized by identical or similar parameters. Therefore, in exemplary embodiments of the invention, the responsiveness of a group of pixels can be determined from the function that expresses the responsiveness with respect to the complexity of the group of pixels.
In exemplary embodiments of the invention, the image compression is achieved by quantization of the coefficients of a transformation of the group of pixels, such as the coefficients of a DCT transformation as typically used in temporal and spatial compression, for example, in JPEG, MPEG and AVC.
In exemplary embodiments of the invention, the method for determining the quantization factor by responsiveness to quantization applies as well to a difference between a reconstructed compressed group of pixels and the original one.
In the specifications and claims, unless otherwise specified, and without limiting the generality, the term ‘pixel’ denotes a visual pixel, or a derived element of a pixel (such as a difference between a reconstructed compressed group of pixels and the original one).
An aspect of exemplary embodiments of the invention relates to a method for image compression by quantization factors that are based on the responsiveness to quantization of group of pixels. Such group of pixels may be as typically used in the MPEG and AVC standards of 16×16 or 8×8 or 16×8 or 8×16 or 4×8 or 8×4 or 4×4 pixels.
In exemplary embodiments of the invention, the quantization is adapted to a desired bit-rate, or the image target compression ratio, or the compressed image quality, or a combination thereof.
In exemplary embodiments of the invention, the quantization factor is determined, according to the responsiveness, within a range of quantization factors. Optionally, the range of quantization factors is determined subject to requirements and/or constraints of an application or a usage of the compressed image. For example, the target bit-rate, or the image target compression ratio, or the compressed image quality, or a combination thereof.
In exemplary embodiments of the invention, based on the curvature and/or the slope of the function of the responsiveness with respect to complexity of a group of pixels, at least two characteristic regions are identified in the function.
In exemplary embodiments of the invention, in one region corresponding to low complexity and/or where the curve is slightly curved and/or diagonal, the quantization is determined according to the responsiveness such that the pixels would compress by the minimum allowable, at least approximately, quantization factor and preserve, at least approximately, the visual quality of the low complexity pixels.
In exemplary embodiments of the invention, in a second region corresponding to high complexity and/or where the curve is approximately asymptotic and/or approximately horizontal, the quantization factor is practically insensitive to the complexity; that is, the visual quality of the compressed image is, at least approximately, not affected by using a larger quantization than the one derived from the function.
In exemplary embodiments of the invention, a threshold value where the insensitivity region begins can be determined. Optionally, the threshold is constant, at least approximately, for any image or part thereof, without impairing, at least approximately, the visual quality of the image compressed by a quantization factor derived from the function according to the constant threshold.
In the discussions that follow, unless otherwise specified, the terms ‘image partition’, or ‘partition’ denote a group of pixels of an image.
According to an aspect of some embodiments of the present invention there is provided a method for obtaining a quantization factor for image compression by quantization of coefficients of a transformation of the image or part thereof, comprising:
(a) determining an entropy-related sensitivity to quantization of at least a part of the image; and
(b) determining a quantization factor based on the entropy-related sensitivity.
According to some embodiments of the invention, determining a sensitivity to quantization comprises determining a change in entropy of a quantized at least a part of the image relative to the entropy of the at least part of the image, or a derivation thereof.
According to some embodiments of the invention, the derivation comprises the ratio between the entropy of a quantized at least a part of an image and the entropy of the at least part of the image (ED).
According to some embodiments of the invention, the method comprises determining a threshold of ED above which the visual quality of a compressed at least a part of an image is insensitive, at least approximately, to ED.
According to some embodiments of the invention, the threshold is independent of the image.
According to some embodiments of the invention, the method comprises determining a threshold of a complexity quantification of the at least a part of an image above which the visual quality of a compressed at least a part of an image is insensitive, at least approximately, to the complexity quantification. According to some embodiments of the invention, the threshold is independent of the image.
According to some embodiments of the invention, determining a quantization factor based on the entropy-related sensitivity comprises a linear function of the entropy-related sensitivity.
According to some embodiments of the invention, determining a quantization factor comprises determination according to a range of quantization factors.
According to some embodiments of the invention, determination according to a range of quantization factors comprises a linear function of the rounded range of quantization factors.
According to some embodiments of the invention, the quantization factor is adjusted according to on at least one of the image target bit-rate, the image target compression ratio, the target visual quality of the compressed image, or a combination thereof.
According to some embodiments of the invention, the range of quantization factors is based on at least one of the image target bit-rate, the image target compression ratio, the target visual quality of the compressed image, or a combination thereof.
According to some embodiments of the invention, the at least part of an image comprises is arbitrary.
According to some embodiments of the invention, the at least part of the image comprises a difference between a decompression of a previously compressed part of the image and the original part of the image.
According to an aspect of some embodiments of the present invention there is provided a method for evaluation of entropy-related sensitivity to quantization of an image or part thereof to obtain a quantization factor for image compression by quantization of coefficients of a transformation of the image or part thereof, comprising:
(a) determining a function of entropy-related quantification with respect to a complexity quantification of at least a part of the image; and
(b) determining a quantization factor based on the function.
According to some embodiments of the invention, the entropy-related quantification comprises, at least approximately, a ratio between the entropy of a quantized at least a part of an image and the entropy of the at least part of the image (ED).
According to some embodiments of the invention, a quantification of the complexity comprises a standard deviation of the at least part of the image.
According to some embodiments of the invention, the function comprises an exponential function of the complexity quantification.
According to some embodiments of the invention, the function is, at least approximately, independent of the image.
According to some embodiments of the invention, the function comprises an approximation of an exponential function.
According to some embodiments of the invention, determining a quantization factor comprises determination according to a range of quantization factors.
According to some embodiments of the invention, the range of quantization factors is based on at least one of the image target bit-rate, the image target compression ratio, the target visual quality of the compressed image, or a combination thereof.
According to some embodiments of the invention, the method comprises determining, according to at least one of the curvature or the slope of the function, a threshold of ED above which the visual quality of a compressed at least a part of an image is insensitive, at least approximately, to ED.
According to some embodiments of the invention, the method comprises determining, according to at least one of the curvature or the slope of the function, a threshold of a complexity quantification of the at least a part of an image above which the visual quality of a compressed at least a part of an image is insensitive, at least approximately, to the complexity quantification.
According to an aspect of some embodiments of the present invention there is provided an apparatus configured to carry-out the methods recited above.

BRIEF DESCRIPTION OF THE DRAWINGS

Non-limiting examples of embodiments of the present invention are described with reference to figures listed below. In the drawings which follow, identical or equivalent or similar structures, elements, or parts that appear in more than one drawing are generally labeled with the same numeral in the drawings in which they appear. Dimensions of components and features shown in the figures are chosen for convenience and clarity of presentation and are not necessarily shown to scale.

FIG. 1 illustrates a chart of pixels values of a partition before and after quantization, in accordance with exemplary embodiments of the invention;

FIG. 2A shows an image, in accordance with exemplary embodiments of the invention;

FIG. 2B illustrates entropy deficiency of the partitions of the image of FIG. 2A, in accordance with exemplary embodiments of the invention;

FIG. 2C illustrates the distribution of the image deficiencies of the partitions of FIG. 2B and a fitted exponential curve with respect to the standard deviation, in accordance with exemplary embodiments of the invention;

FIG. 3A shows an image, in accordance with exemplary embodiments of the invention;

FIG. 3B illustrates the entropy deficiency of the partitions of the image of FIG. 3A, in accordance with exemplary embodiments of the invention;

FIG. 3C illustrates the distribution of the image deficiencies of the partitions of FIG. 3B and a fitted exponential curve, with respect to the standard deviation in accordance with exemplary embodiments of the invention;

FIG. 4A shows an image, in accordance with exemplary embodiments of the invention;

FIG. 4B illustrates the entropy deficiency of the partitions of the image of FIG. 2A, in accordance with exemplary embodiments of the invention;

FIG. 4C illustrates the distribution of the image deficiencies of the partitions of FIG. 2B and a fitted exponential curve, with respect to the standard deviation, in accordance with exemplary embodiments of the invention;

FIG. 5 illustrates a chart with respect to complexities of image partitions and a range of quantization factors with (a) a graph of the entropy deficiencies of the partitions, and (b) a graph of quantization factors, in accordance with exemplary embodiments of the invention;

FIG. 6 is a flowchart that outlines a sequence of operations for determining a quantization factor for image partitions and their subsequent compression, in accordance with exemplary embodiments of the invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

The discussion below is divided into sections with headers which are intended for clarity and readability only.

Responsiveness to Quantization—Entropy Deficiency (ED)

In exemplary embodiments of the invention, the responsiveness to quantization is defined as a change in the entropy of a group of pixels quantized by a preferably (without limiting) a small quantization, relative to the entropy of the original pixels. Optionally, the responsiveness to quantization is obtained by a convenient derivation of the latter definition, namely, as the ratio of the entropy of a quantized group of pixels to the entropy of the original pixels.
In exemplary embodiments of the invention, the responsiveness is defined according to the following formula.
R=(E(qP)−E(qP))/E(P)=1−E(qP)/E(P) (1)
Wherein
R is the responsiveness according to the definition,
E is the entropy −Σp_ilog₂(1/p _i), where p_iis the probability of each pixel in the group,
P is a group of pixels,
qP is the quantized group of pixels.
According to formula (1) the responsiveness is bounded by a range between 1 (full responsiveness) and 0 (no responsiveness).
In exemplary embodiments of the invention, a convenient derivation of responsiveness for a group of pixels is derived according to the following formula.
ED=1−R (2)
So that:
ED=E(qP)/E(P) (3)
Where ED is the responsiveness, denoted as Entropy Deficiency. As such, the ED is in a range between 0 (full responsiveness) and 1 (no responsiveness).
In exemplary embodiments of the invention, the complexity is quantified as the entropy of the pixels, or as the standard deviation of the pixels, or other complexity quantification such as contrast (or non-uniformity) measures.
In the following discussion, without limiting the generality, and unless otherwise specified, the responsiveness to quantization refers to responsiveness to quantization where the complexity is the entropy of a group of pixels. Such responsiveness will be referred to as ‘entropy deficiency’.
FIG. 1 illustrates a chart of pixels values of a group of 64 pixels before quantization (102) and after quantization (104), in accordance with exemplary embodiments of the invention. Originally there were about 250 different values, and a quantization a factor of 32 reduced the range to only 8 distinct values. The entropy of the original image is 6.0, and after quantization the entropy is 3.0, yielding an entropy deficiency of: 3.0/6.0=0.50.
In exemplary embodiments of the invention, quantizing comprises dividing the pixels by a uniform factor. Optionally and alternatively, the factor is different for different values range and/or relations between values (e.g. smaller factor for edge pixels).
In exemplary embodiments of the invention, the quantization factor for calculating ED according to formula (3) is a small value relative to the range of pixels values in the group. Preferably, but not limited to, the quantization factor is 4.

Entropy Deficiency Curve (Function)

It was unexpectedly found that the entropy deficiency of a group of pixels exhibits a characteristic distribution with respect to the complexity of the group of pixels, such as standard deviation or entropy or other quantification of the pixel complexity. The distribution aggregates in a pattern resembling a curve with an initial steep increase followed by a gradual decline, resembling a negative exponential.
The entropy deficiency of over a thousand different image partitions were plotted and/or evaluated with respect to the corresponding complexity, and a fitted curve was found to exhibit an exponential function, as follows:
ED=A+B×Exp(−C×P) (4)
wherein
a) ED is the entropy deficiency,
b) A, B and C are constants, and
c) P comprises a complexity quantification of the group of pixels (e.g. standard deviation).
In exemplary embodiments of the invention, for different groups of pixels the constants may vary, yet they typically aggregate in ranges of close values. Optionally, the ranges are as follows:
A is approximately between 0.3 and 1.0,
B is approximately between −0.2 and −1.0, and
C is approximately between 0.2 and 0.3.
Furthermore, it was found that a curve according to formula (4) where A is 0.7, B is −0.4 and C is 0.25 yields a sufficient fit for a determination of quantization factor for an efficient compression with high visual quality as described later on.
The distribution of entropy deficiencies and the corresponding fitted curve for a few images are illustrated in FIGS. 2, 3, and 4.
In accordance with exemplary embodiments of the invention, FIG. 2A shows an image 202, and FIG. 2B illustrates the entropy deficiencies 204 of partitions 206 of image 202. FIG. 2C illustrates a distribution 208 of the entropy deficiencies 204 of partitions 206 of image 202, together with a fitted exponential curve 210, with respect to the standard deviation 212 of corresponding partitions 206. Entropy deficiencies 204 of partitions 206 of image 202 are depicted in a gray scale, together with a reference scale 214. Coordinate 216 represents entropy deficiencies of distribution 208 and graph 212.
In accordance with exemplary embodiments of the invention, entropy deficiencies 204 of partitions 206 of image 202 were derived according to formula (3). Regions of image 202 having a constant or low variation, such the lower background (218 a), the white shoulder strip of the shirt (218 b), or the sky (218 c) have low entropy deficiencies (220 a, 220 b and 220 c, respectively), whereas complex regions such as plants (222 a) or grass (222 b) or illuminated hair (222 c) have high entropy deficiency (224 a, 224 b and 224 c, respectively).
FIG. 2C illustrates how distribution 208 of the entropy deficiencies 204 of partitions 206 resembles an exponential with respect to the partitions complexity (standard deviation 210). Fitted graph 212, which takes into account the dispersion of the distribution, is an exponential according to formula (4).
In order to illustrates how the distribution of entropy deficiencies and curve fitting are similar for various images and partitions, different images with respective entropy deficiencies and distributions and fitted curves are shown.
Similar to FIG. 2A, FIGS. 3A and 4A show other images (302/402), and FIGS. 3B and 4B illustrate the entropy deficiency (304/404) of partitions (306/406) of the images (302/402). FIGS. 3C and 4C illustrate the distribution (308/408) of the entropy deficiencies (304/404) of partitions (306/406), together with fitted exponential curves (310/410), with respect to the standard deviation (312/412). The image deficiencies (304/406) of partitions (306/406) of the images (302/402) are depicted in gray scale, with reference scales (314/414). The coordinates of the distributions (308/408) and graphs (312/412) correspond to the entropy deficiencies (304/404).
The additional images 302/402, and the accompanying entropy deficiencies 304/404, and particularly the accompanying distributions (308/408) and graphs (310/410) illustrate that the shape of the distributions of the entropy deficiencies with respect to complexity is a common property of different kinds of images.
Consequently, based on hundreds of different images, in accordance with exemplary embodiments of the invention formula (4) can be used to determine, at least approximately, the entropy deficiency of a group of pixels directly from the complexity of the pixels.
Furthermore, formula (4) enables to determine a quantization factor for compression of a group of pixels, as described below.

Quantization According to Entropy Deficiency

In exemplary embodiments of the invention, a function (or curve) according to formula (4) enables to determine an effective quantization factor for compression of image partitions in terms of compression ratio and/or image visual quality.
FIG. 5 and the following discussion describe some properties of the function, and how they relate to, and enable the determination of quantization factors for compression of a partition, in accordance with exemplary embodiments of the invention.
FIG. 5 illustrates a chart 500 with respect to complexities 512 of image partitions. Graph (curve) 510 depicts the entropy deficiencies of the partitions according to formula (4), in accordance with exemplary embodiments of the invention.
In exemplary embodiments of the invention, a range of quantization factors 530 between a minimal value 532 (‘MinQ’), and a maximal value 534 (‘MaxQ’) is provided. Range 530 is optionally preset and/or determined by the compression application and/or the intended use of the compression and/or the bit-rate and/or the intended visual quality of the compressed partitions (‘rate control’).
In exemplary embodiments of the invention, MaxQ is derived from and/or equals the image target bit-rate, or the image target compression ratio, or the compressed image quality, or a combination thereof. Optionally, MinQ is also related to the image target bit-rate, or the image target compression ratio, or the compressed image quality, or a combination thereof.
In order to determine a quantization factor (‘Q’) for a partition, in exemplary embodiments of the invention the entropy deficiency is mapped (transformed) onto the quantization factors in range 530. Optionally, the mapping is a linear mapping, as is illustrated by line 560 that maps a complexity of 22.0 (562), or its corresponding entropy deficiency 0.7 (566), to a quantization factor Q of 0.86 (564). Optionally, the factor is rounded to an integer. Optionally, the mapping comprises a linear mapping with additional terms, as illustrated below.
In exemplary embodiments of the invention, a quantization factor Q for compressing an image partition is determined by mapping of the entropy deficiency of a partition on quantization factors range 530 according to the following formula.
Q=Min Q−A+Round(((Max Q−Min Q)+B)×ED) (5)
Where A and B are constants and ED is the entropy deficiency, and wherein Q is bounded by MinQ; that is, if Q evaluates to a value lower than MinQ, Q is set to MinQ.
In exemplary embodiments of the invention, at least A or B is 3. Consequently, in exemplary embodiments of the invention, Q is evaluated according to formula (5) with substitutions for A and B, namely:
Q=Min Q−3+Round(((Max Q−Min Q)+3)×ED) (6)
FIG. 5 graphically illustrates relations between the entropy deficiency function and the quantization factor, in accordance with exemplary embodiments of the invention. Graph 510 of the entropy deficiencies of image partitions is plotted, according to formula (4), with respect to corresponding complexities 512. Graph 540 depicts the mapped quantization factors Q for the corresponding entropy deficiencies according to formula (6), where MinQ is 0 and MaxQ is 20, with respect to a secondary coordinate 538 of Q. For example, for a partition complexity of 3 (572), or entropy deficiency is 0.46 (574), the corresponding quantization factor is 8 (576), as shown with dotted lines in chart 500.
Graph 510 of the entropy deficiency function may be divided into two regions:
(a) A rising region 542 with partitions of low complexities and corresponding low entropy deficiencies or large responsiveness (formula (3)), where the partitions are sensitive to quantization (sensitivity region). The quantization factor Q vary rapidly in the sensitivity region 542, as shown in chart 500 where Q vary between 4 and 13 for entropy deficiencies between 0 and 11, respectively.
(b) An asymptotic region (saturation region) 544 with partitions of high complexities and corresponding high entropy deficiencies and small responsiveness (formula (3)), where the partitions are insensitive (or less sensitive) to quantization (insensitivity region). The quantization factor Q is approximately constant in the saturation region 544, as shown in chart 500 where Q is 13 for entropy deficiencies between 11 and 50, or higher.
In exemplary embodiments of the invention, a partition in the sensitivity region 542 is sensitive to quantization such that increasing the quantization above Q will decrease the visual quality of the compressed partition (e.g. blockiness or abrupt changes), and decreasing the quantization will decrease the compression ratio without gaining in visual quality. Yet, using a quantization factor Q according to formulas (5) or (6) (or chart 500) will typically yield a sufficient compression (with respect to application requirements) with a good visual quality (e.g. preserving the variation of shades).
In exemplary embodiments of the invention, a partition in the saturation region 544 is insensitive, at least approximately, to the complexity of the partition, such that a partition corresponding of a low complexity in the saturation region 544 may be quantized by a (possibly larger) factor Q corresponding to a higher complexity, yielding a better compression ratio without affecting the visual quality of the compressed partition. As such, region 544 may be referred to as the insensitive region.
The determination of a quantization factor responsive to entropy deficiency as described above is new and unique, providing a quantization adapted to the contents of the partition and comprises two exceptional aspects. Firstly, partitions within the sensitivity region (i.e. low complexity) are quantized such that the mild variations are maintained in the compressed image, and being of low complexity the low quantization does not overload the volume of the compressed image. Secondly, partitions with low complexity within the insensitivity region may be compressed with high quantization providing high compression ratio without affecting the visual quality of the compressed image. In other words, the same quantization may be used for all partitions in the insensitivity region without adverse visual affect in the compressed image.
In exemplary embodiments of the invention, insensitivity region 544 begins about a complexity measure of about 11.0 (550) or about a corresponding entropy deficiency value of about 0.67 (552). Optionally, threshold point 550 for the division of graph 540 to sensitivity region 542 and insensitivity region 544, may vary. For example, the determination of the threshold value 550 can be determined according to the curvature and/or slope of graph 510 or the corresponding entropy deficiency function (4). Optionally the threshold value 550 depends on the constants used in formula (4). Optionally or alternatively, the divisions may be affected to the evaluation of complexity function used to determine the responsiveness and/or the complexity function used in formula (4) (e.g. not a standard deviation). For example, insensitivity region 544 may begin at a threshold value 550 of 15.0 or the corresponding entropy deficiency value of 0.69. Typically, for given parameters of function 4 and a given complexity quantification, threshold value 550 is constant, at least approximately, for any image.
In exemplary embodiments of the invention, entropy deficiency graph 510 (or function) may be approximated. Optionally, the approximation is by a piece-wise linear approximation, for example, linear sections 554, 556 and 558, such that determining the quantization factor by the approximation will not affect visual quality, or only negligibly affect the visual quality. Optionally, graph 510 may be fitted with two linear sections. Optionally, other curve approximations may be used, such as sigmoid or Heaviside step functions, optionally yielding better approximation to graph 510 (or entropy deficiency function) relative to a linear approximation.
In exemplary embodiments of the invention, using an approximation for graph 510 can boost computation time for finding the quantization factor Q. For example, using a linear approximation, the factor Q may be determined by simple arithmetic operation, avoiding more complex operations such as exponentials.
In exemplary embodiments of the invention, the entropy deficiency function (formula (4), graph 510) may be pre-calculated into a table, which consequently can be used as a lookup table, optionally with interpolations.
In exemplary embodiments of the invention, a plurality of ranges of factors MinQ to MaxQ are preset and stored. Subsequently, according to the bit-rate and/or intended quality of the compressed partition an appropriate range is selected and used to determine the quantization factors.

Adjusting the Quantization Factor

In exemplary embodiments of the invention, the quantization factor Q obtained in saturation region 544 is lower than a factor which will still maintain, at least approximately, the image quality as by using Q; that is, a better compression ratio could be achieved without sacrificing quality. Additionally, a quantization factor Q obtained in the sensitivity region might be somewhat larger for than desired for a desired visual quality.
In exemplary embodiments of the invention, the determination of an adjusted quantization factor requires sub-dividing sensitivity region 442. According to the slope and/or curvature of sensitivity region 442, region 442 is divided into two regions: (a) a steep semi-linear region 546 and (b) an inflection region 548. The dividing point in terms of complexity or corresponding entropy deficiency, such as complexity value 6 in chart 500, is optionally determined about where the steep part begins to inflect, or the slope begins to decrease or, when the curvature is increasing beyond a certain value.
In exemplary embodiments of the invention, the adjustment of the quantization factor pertains to the coefficients of a transformation of a partition, and comprises the following operations, wherein the order of the operations is not mandatory where applicable.
a) Determining the number of non-zero coefficients of the transformation. Optionally, coefficients of low values, such as lower than the median of the non-zero coefficients values or lower than the average of non-zero coefficients (e.g. low than 10% or lower than 5% or 1%) are considered as zero.
b) Obtaining a quantization factor Q according to formula (5) or (6) (or according to chart 500 described above), denoted as Q₀.
c) Quantizing (dividing) coefficients of the transformed partition by Q₀.
d) Determining the number of non-zero coefficients after quantization by Q₀. Optionally, coefficients of low values, such as lower than the median of the non-zero coefficients values and/or lower than the average of non-zero coefficients (e.g. low than 10% or lower than 5% or 1%) are considered as zero.
e) Determining a ratio RQ between the number of non-zero un-quantized coefficients and the number of non-zero quantized coefficients;
f) Subtracting the ratio RQ from the image target bit-rate, or the image target compression ratio, or a value derived therefrom (e.g. with a proportionality factor), to obtain a difference DQ;
g) Determining a new quantization factor Q, the determination comprising:
i) If the partition entropy deficiency or complexity is about inflection region 548 then Q equals Q₀(i.e. the quantization factor does not change).
ii) If DQ>0 and the partition entropy deficiency or complexity is in insensitivity region 544 then Q₀is increased by |DQ| or by a value depending on |DQ|, obtaining a new Q.
Optionally, the increase is limited to a range of values. Optionally, the increase is in a range between 1 and 10. Optionally, the range is between 0 and 5. Optionally, the range is between 1 and 4. Optionally, the increase is according to |DQ|, so that the larger |DQ| the larger is the increase. For example, for a range between 1 and 4, the increase is by |DQ|, but limited by 4.
iii) If DQ<0 and the partition entropy deficiency or complexity is below inflection region 548 (i.e. in region 546) then Q₀is decreased by |DQ| or by a value depending on |DQ|, obtaining a new Q.
Optionally, the decrease is limited to a range of values. Optionally, the decrease is in a bounded range between 0 and 6. Optionally, the range is between 1 and 5. Optionally, the range is between 1 and 2. Optionally, the decrease is according to |DQ|, so that the larger |DQ| the larger is the decrease. For example, for a range between 1 and 2, the increase is by |DQ|, but limited by 2.
In exemplary embodiments of the invention, the compression according to the modified quantization factor Q is limited so that it does not effect exceeding the image target bit-rate or the target compression ratio of the image, or a combination thereof.
In exemplary embodiments of the invention, when the quantization factor is decreased a better visual quality is achieved on the expense of some decrease in the compression ratio. Yet, since region 456 belong to partitions of low complexities, the reduction in compression ratio is, typically, insignificant.

Partitions and Blocks

In exemplary embodiments of the invention, the determination of the responsiveness of a group of pixels to quantization (e.g. entropy deficiency), and a subsequent compression by quantization, can be performed on a collection of pixels with no geometrical constraints.
In exemplary embodiments of the invention, the group of pixels comprises a partition of an image. In exemplary embodiments of the invention, a partition comprises a rectangular shape. Optionally, a dimension of a partition is one of 2, 4, 8, 16, 32 or 64 pixels. Optionally, a partition dimension is larger than 64 pixels. Optionally or alternatively, a partition comprises a non-rectangular shape. Optionally, the partition shape is according to the values of the pixels and/or the complexity of the pixels and/or geometry of features and/or computational considerations. For example, the partition shape may be adapted to comprise pixels of the same or similar complexity, or adapted to comprise a limited range of values.
In exemplary embodiments of the invention, a partition comprises one or more blocks. Optionally, a dimension of a block is one of 2, 4, 8, 16 or 32 or 64 pixels. Optionally, a block dimension is larger than 64 pixels. Optionally, a block comprises one or more blocks. Optionally, a block comprises a non-rectangular shape, for example, such as to comprise pixels of the same or similar range of values and/or similar complexity.
In exemplary embodiments of the invention, a partition comprises disjointed blocks, that is, the blocks are separated by one or more pixels not belonging to the partition.
In exemplary embodiments of the invention, a partition is divided into blocks based on the complexity of the partition. In that manner, a better quantization (in terms of compression ratio and/or visual quality) may be determined for each block separately rather than the whole partition. Optionally, the division into blocks is such that above a certain complexity the partition is divided into a plurality of blocks having relative high and low complexities, each optionally resulting in different quantization factors.
For example, a standard deviation of a partition is found to be 28.1 which may be considered as too complex. Therefore, the partition is divided into two blocks having standard deviations of 4.4 and 26.9, respectively. Thus, the first block falls within the sensitivity region (542 of FIG. 5) and quantized by a small factor (9), while the second block is falls in the insensitivity region (544 of FIG. 5) and quantized by a larger factor (13).
In exemplary embodiments of the invention, a partition and/or a block dimension is according to a method of the image compression. For example, when MPEG (e.g. h.264) is used the partition dimensions comprise 16×16 pixels (‘macro-block’), or the dimensions comprise 8×8 pixels frequently used in DCT transformation such as JPEG.
In exemplary embodiments of the invention, the quantization is independent of the size and/or shape of a partition or block, since only the collection of pixels is considered.
In exemplary embodiments of the invention, the methods and/or embodiments described for a partition apply, at least partially, to a block within a partition.

Image Compression and Decompression

In exemplary embodiments of the invention, an image, or part thereof, such as a group of pixels, or a partition, or a block, is quantized according to a quantization factor that is determined as described above and, optionally, is subsequently compressed. Optionally or alternatively, the image pixels are used to determine the quantization factors as described, and the coefficients of a transformation, such as DCT, of the respective pixels are quantized (divided) by the factors. Optionally or alternatively, the transformed pixels are used to determine the quantization factors and are quantized accordingly. Optionally, the quantization is by a modified factor, such as by limiting the value of the factor according to the compression method.
In exemplary embodiments of the invention, the quantized pixels or quantized coefficients are encoded, for example, the entropy encoding or arithmetic encoding.
In exemplary embodiments of the invention, the image is a gray-scale. Optionally, the image is a color image separated into channels, such as RGB, YIQ, YUV, etc., and each channel is quantized and/or compressed separately according to exemplary methods and embodiments of the invention. Optionally, the image comprises of pixels packing one or more colors such as RGB, or luminance (brightness) and one or more color components (e.g. YUC).
In exemplary embodiments of the invention, an image is compressed in a video or pseudo-video sequence, wherein a video frame comprises one or more image partitions or blocks. Optionally, the frames are compressed according to intra- or inter-predictive methods. Optionally or additionally, motion and/or temporal compressions are used to compress the frames.
In exemplary embodiments of the invention, the quantization factors obtained as discussed above can be used within the framework of compression standards, such as the spatial compression in JPEG, MPEG or H.264.
In exemplary embodiments of the invention, a compressed image is decompressed using techniques of the art. Optionally or alternatively, when the compression is non-standard, a matching decoder can be devised.

Operation Outline

An exemplary procedure for obtaining a quantization factor according to the entropy deficiency, or responsiveness to quantization, and using the factor for compression is described with respect to FIG. 6.
FIG. 6 is a flowchart that outlines a sequence of operations for determining a quantization factor for image partitions and their subsequent compression, in accordance with exemplary embodiments of the invention.
According to the application requirements or system constraints, a target bit-rate is set (602). Optionally, the requirement is set by the bit-rate, or image target compression ratio, or the compressed image quality, or a combination thereof.
In order to determine quantization factor, the entropy deficiency function is established (604), such as by formula (4). Optionally, an approximation of the function is established, such as by linear segments or a lookup table, in order to simplify and speed up the determination of the quantization factors. Optionally, the regions of the function are established, that is, the sensitivity region 542, the insensitivity region 544 or the inflection region 548.
The image is divided into partitions. Optionally, the partitions are determined according to the application, such as 8×8 pixels for JPEG or 16×16 for video macro-blocks, as discussed above.
A partition is obtained or selected in the image (606), and the partition complexity or the entropy deficiency is determined (608). According to the complexity and/or the entropy deficiency the quantization factor is found (610).
Optionally, when the quantization pertains to coefficients of a transformation, such as DCT, the quantization factor is adjusted with respect to the bit-rate (612). Optionally, the adjustment is with respect to the bit-rate, or image target compression ratio, or the compressed image quality, or a combination thereof.
Subsequently the partition or its transformation (e.g. DCT coefficients), are quantized (614), that is, optionally, dividing the pixels values or the coefficients by the quantization factor (or a derivation thereof). The quantized values are compressed (616) by a method of the art, such as by entropy encoding. Then the next partition is obtained (616) and the sequence is repeated (618) until the image, or the part of the image intended for compression, is compressed.

Derived Image—Difference

In exemplary embodiments of the invention, image compression comprises a multi-step spatial compression. For example, a compressed partition is decompressed, resulting in a partition which is different from the original. Diff may comprise a part of the compressed partition so that during compression, the Diff is optionally added to the decompressed partition to yield a better visual quality, that is, with more details. In exemplary embodiments of the invention, in order to improve the compression ratio, Diff is compressed too, and as a part of decompression the decompressed Diff is added to the decompressed partition. In exemplary embodiments of the invention, Diff is quantized by a factor determined according to formula (4), or optionally, according to formula (3), similar to or as the original partition is quantized and compressed.
In exemplary embodiments of the invention, additional Diff partitions are obtained and quantized and compressed as described above. For example, the first uncompressed Diff is added to the first uncompressed partition and the combined partitions are subtracted from the original partition to yield another Diff with finer details. In this manner more Diff partitions may be obtained.
In exemplary embodiments of the invention, Diff pixels are optionally scaled and/or shifted and/or otherwise manipulated (e.g. contrast enhancement) before the quantization, and reverse operations are applied to the uncompressed Diff.

Apparatus

In exemplary embodiments of the invention, existing equipment for image and/or video compression (coder) is used, optionally with provisions to set the quantization factors for particular image partitions.
In exemplary embodiments of the invention, the coder comprises one or more software modules and/or libraries. Optionally, the coder comprises hardware and/or firmware. Optionally, the coder comprises a chip-set with internal or external memory and/or one or more processors. Optionally, the coder is part of a device.
In exemplary embodiments of the invention, the chipset is used on mobile devices such as cameras or cellular phones or PDAs. Optionally, the coder (or codec) is part of the device.
In exemplary embodiments of the invention, off-the-shelf or proprietary tools for constructing the compressed image and/or the video sequence are utilized. Optionally, the tools comprise SDK (software development kit) using techniques such API or procedure calls to compress and construct the image and/or video. Optionally or additionally, hardware modules are used. Optionally, a combination of software, hardware or firmware is used.
In exemplary embodiments of the invention, the coder is linked to an imaging sensor. Optionally, the sensor transfers the image to a memory. Optionally, the sensor may be tapped for the image. Optionally, the sensor is a part of the chip-set.
In exemplary embodiments of the invention, processors may be used in coding, such as a general purpose or image co-processor (ICP), an application processor or a communications processor. Optionally, the processor is a dedicated processor. Optionally, the processor comprises a DSP.
In exemplary embodiments of the invention, the coder may accept, in addition to the particular quantization factors, operation parameters such as bit-rate, a range of allowed or desired quantization factors, target compression level, or one or more visual quality metrics. For example, by setting a configuration file or using API (application programming interface) to set the parameters.
In exemplary embodiments of the invention, the image or video sequence may be stored in memory, optionally for subsequent decoding or transfer to another device. Optionally, the image or video sequence may be ‘streamed’, that is, transferred as it is constructed.
In exemplary embodiments of the invention, the coder is adapted for compression standards such as JPEG, or video standards such as AVC (H.264), MPEG 1/2/4 or other formats. Optionally, a coder for non-standard formats is used.
In exemplary embodiments of the invention, equipment such as described above is used to decompress the image and/or video (decoder).

General

In the description and claims of the present application, each of the verbs “comprise”, “include” and “have” as well as any conjugates thereof, are used to indicate that the object or objects of the verb are not necessarily a complete listing of members, components, elements or parts of the subject or subjects of the verb.
The present invention has been described using detailed descriptions of embodiments thereof that are provided by way of example and are not intended to necessarily limit the scope of the invention. In particular, numerical values may be higher or lower than ranges of numbers set forth above and still be within the scope of the invention. The described embodiments comprise different features, not all of which are required in all embodiments of the invention. Some embodiments of the invention utilize only some of the features or possible combinations of the features. Alternatively and additionally, portions of the invention described/depicted as a single unit may reside in two or more separate physical entities which act in concert to perform the described/depicted function. Alternatively and additionally, portions of the invention described/depicted as two or more separate physical entities may be integrated into a single physical entity to perform the described/depicted function. Variations of embodiments of the present invention that are described and embodiments of the present invention comprising different combinations of features noted in the described embodiments can be combined in all possible combinations including, but not limited to use of features described in the context of one embodiment in the context of any other embodiment. The scope of the invention is limited only by the following claims.
All publications and/or patents and/or product descriptions cited in this document are fully incorporated herein by reference to the same extent as if each had been individually incorporated herein by reference or if they were reproduced in full herein.

Claims

1. A method for obtaining a quantization factor for image compression by quantization of coefficients of a transformation of the image or part thereof, comprising:

(a) determining an entropy-related sensitivity to quantization of at least a part of the image; and

(b) determining a quantization factor based on the entropy-related sensitivity.

2. A method according to claim 1, wherein determining a sensitivity to quantization comprises determining a change in entropy of a quantized at least a part of the image relative to the entropy of the at least part of the image, or a derivation thereof.

3. A method according to claim 2, wherein the derivation comprises the ratio between the entropy of a quantized at least a part of an image and the entropy of the at least part of the image (ED).

4. A method according to claim 3, comprising determining a threshold of ED above which the visual quality of a compressed at least a part of an image is insensitive, at least approximately, to ED.

5. A method according to claim 4, wherein the threshold is independent of the image.

6. A method according to claim 1, comprising determining a threshold of a complexity quantification of the at least a part of an image above which the visual quality of a compressed at least a part of an image is insensitive, at least approximately, to the complexity quantification.

7. A method according to claim 6, wherein the threshold is independent of the image.

8. A method according to claim 1, wherein determining a quantization factor based on the entropy-related sensitivity comprises a linear function of the entropy-related sensitivity.

9. A method according to claim 1, wherein determining a quantization factor comprises determination according to a range of quantization factors.

10. A method according to claim 9, wherein determination according to a range of quantization factors comprises a linear function of the rounded range of quantization factors.

11. A method according to claim 10, wherein the quantization factor is adjusted according to on at least one of the image target bit-rate, the image target compression ratio, the target visual quality of the compressed image, or a combination thereof.

12. A method according to claim 9, wherein the range of quantization factors is based on at least one of the image target bit-rate, the image target compression ratio, the target visual quality of the compressed image, or a combination thereof.

13. A method according to claim 1, wherein the at least part of an image comprises is arbitrary.

14. A method according to claim 1, wherein the at least part of the image comprises a difference between a decompression of a previously compressed part of the image and the original part of the image.

15. A method for evaluation of entropy-related sensitivity to quantization of an image or part thereof to obtain a quantization factor for image compression by quantization of coefficients of a transformation of the image or part thereof, comprising:

(a) determining a function of entropy-related quantification with respect to a complexity quantification of at least a part of the image; and

(b) determining a quantization factor based on the function.

16. A method according to claim 15, wherein the entropy-related quantification comprises, at least approximately, a ratio between the entropy of a quantized at least a part of an image and the entropy of the at least part of the image (ED).

17. A method according to claim 15, wherein a quantification of the complexity comprises a standard deviation of the at least part of the image.

18. A method according to claim 15, wherein the function comprises an exponential function of the complexity quantification.

19. A method according to claim 18, wherein the function is, at least approximately, independent of the image.

20. A method according to claim 18, wherein the function comprises an approximation of an exponential function.

21. A method according to claim 15, wherein determining a quantization factor comprises determination according to a range of quantization factors.

22. A method according to claim 21, wherein the range of quantization factors is based on at least one of the image target bit-rate, the image target compression ratio, the target visual quality of the compressed image, or a combination thereof.

23. A method according to claim 16, comprising determining, according to at least one of the curvature or the slope of the function, a threshold of ED above which the visual quality of a compressed at least a part of an image is insensitive, at least approximately, to ED.

24. A method according to claim 15, comprising determining, according to at least one of the curvature or the slope of the function, a threshold of a complexity quantification of the at least a part of an image above which the visual quality of a compressed at least a part of an image is insensitive, at least approximately, to the complexity quantification.

25. Apparatus configured to carry out the method of claim 1.