DIGITAL WATERMARKS
RELATED APPLICATION DATA
[0001] This application is a continuation of U.S. patent application Ser. No. 10/379,393, filed Mar. 3, 2003. The Ser. No. 10/379,393 application is a continuation of application Ser. No. 09/292,569, filed Apr. 15, 1999, which claims the benefit of provisional application No. 60/082,228, filed Apr. 16, 1998.
[0002] The Ser. No. 10/379,393 application is also a continuation of application Ser. No. 09/998,763, which is a division of application Ser. No. 09/292,569, filed Apr. 15, 1999, which claims the benefit of provisional application No. 60/082,228, filed Apr. 16, 1998.
[0003] The Ser. No. 10/379,393 application is also a continuation-in-part of application Ser. No. 09/186,962, filed Nov. 5, 1998, which is a continuation of application Ser. No. 08/649,419, filed May 16, 1996 (now U.S. Pat. No. 5,862,260), which claims priority to application PCT/US96/ 06618, filed May 7, 1996 (now published as W09636163).
[0004] The Ser. No. 10/379,393 application is also a continuation-in-part of application Ser. No. 09/442,440, filed Nov. 17, 1999, which is a continuation of application Ser. No. 08/951,858, filed Oct. 16, 1997 (now U.S. Pat. No. 6,026,193), which is a continuation of application Ser. No. 08/436,134, filed May 8, 1995 (now U.S. Pat. No. 5,748, 763).
FIELD OF THE INVENTION
[0005] The present application relates to improvements in the field of digital watermarking.
BACKGROUND AND SUMMARY OF THE
INVENTION
[0006] Digital watermarking ("watermarking") is a quickly growing field of endeavor, with several different approaches. The present assignee's work is reflected in the earlier-cited related applications, as well as in U.S. Pat. Nos. 5,841,978, 5,768,426, 5,748,783, 5,748,763, 5,745,604, 5,710,834, 5,636,292, 5,721,788, and laid-open PCT applications W097/43736 and WO99/10837 (corresponding to pending U.S. application Ser. No. 08/746,613 (now U.S. Pat. No. 6,122,403) and Ser. No. 09/138,061). Other work is illustrated by U.S. Pat. Nos. 5,734,752, 5,646,997, 5,659, 726, 5,664,018, 5,671,277, 5,687,191, 5,687,236, 5,689,587, 5,568,570, 5,572,247, 5,574,962, 5,579,124, 5,581,500, 5,613,004, 5,629,770, 5,461,426, 5,743,631, 5,488,664, 5,530,759, 5,539,735, 4,943,973, 5,337,361, 5,404,160, 5,404,377, 5,315,098, 5,319,735, 5,337,362, 4,972,471, 5,161,210, 5,243,423, 5,091,966, 5,113,437, 4,939,515, 5,374,976, 4,855,827, 4,876,617, 4,939,515, 4,963,998, 4,969,041, and published foreign applications WO 98/02864, EP 822,550, WO 97/39410, WO 96/36163, GB 2,196,167, EP 777,197, EP 736,860, EP 705,025, EP 766, 468, EP 782,322, WO 95/20291, WO 96/26494, WO 96/36935, WO 96/42151, WO 97/22206, WO 97/26733. Some of the foregoing patents relate to visible watermarking techniques. Other visible watermarking techniques (e.g. data glyphs) are described in U.S. Pat. Nos. 5,706,364, 5,689, 620, 5,684,885, 5,680,223, 5,668,636, 5,640,647, 5,594,809.
[0007] Most of the work in watermarking, however, is not in the patent literature but rather in published research. In
addition to the patentees of the foregoing patents, some of the other workers in this field (whose watermark-related writings can by found by an author search in the INSPEC database) include I. Pitas, Eckhard Koch, Jian Zhao, Norishige Morimoto, Laurence Boney, Kineo Matsui, A. Z. Tirkel, Fred Mintzer, B. Macq, Ahmed H. Tewfik, Frederic Jordan, Naohisa Komatsu, and Lawrence O'Gorman.
[0008] The artisan is assumed to be familiar with the foregoing prior art.
[0009] In the present disclosure it should be understood that references to watermarking encompass not only the assignee's watermarking technology, but can likewise be practiced with any other watermarking technology, such as those indicated above.
[0010] The physical manifestation of watermarked information most commonly takes the form of altered signal values, such as slightly changed pixel values, picture luminance, picture colors, DCT coefficients, instantaneous audio amplitudes, etc. However, a watermark can also be manifested in other ways, such as changes in the surface microtopology of a medium, localized chemical changes (e.g. in photographic emulsions), localized variations in optical density, localized changes in luminescence, etc. Watermarks can also be optically implemented in holograms and conventional paper watermarks.)
[0011] In accordance with the present invention, various improvements to digital watermarking are disclosed. For example, an improved watermarking method proceeds on an iterative basis in which the watermark data is encoded in a source signal, the result is then decoded, and the "strengths" of the individual encoded bits are discerned. The watermarking parameters are then adjusted so as to redress any deficiencies determined in the first watermarking operation, and the source signal is re-watermarked—this time with the adjusted parameters. So doing assures reliable detection of all the component bits.
[0012] The foregoing and other features and advantages of the present invention will be more readily apparent from the following Detailed Description, which proceeds with reference to the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] FIGS. 1-9 are screen shots associated with an exemplary embodiment of one aspect of the invention.
DETAILED DESCRIPTION
[0014] An improvement to existing watermark encoding techniques is to add an iterative assessment of the robustness of the mark, with a corresponding adjustment in a rewatermarking operation. Especially when encoding multiple bit watermarks, the characteristics of the underlying content may result in some bits being more robustly (e.g. strongly) encoded than others. In an illustrative technique employing this improvement, a watermark is first embedded in an object. Next, a trial decoding operation is performed. A confidence measure (e.g. signal-to-noise ratio) associated with each bit detected in the decoding operation is then assessed. The bits that appear weakly encoded are identified, and corresponding changes are made to the watermarking parameters to bring up the relative strengths of these bits. The object is then watermarked anew, with the changed
parameters. This process can be repeated, as needed, until all of the bits comprising the encoded data are approximately equally detectable from the encoded object, or meet some predetermined signal-to-noise ratio threshold.
[0015] While the foregoing analysis evaluated confidence on a per-bit basis, related iterative procedures can evaluate confidence on a per-portion basis. That is, the encoded object is considered in portions, and each portion is analyzed for the robustness of the data encoded thereby. In portions evidencing "weak" encoding, the encoding parameters can be adjusted to strengthen the encoding in one or more subsequent re-encoding operations.
[0016] The portions can take different forms, e.g., rectangular patches in a still or moving image; brief temporal excerpts in audio or video; certain DCT/Fourier/wavelet coefficients (or adjoining groups of coefficients) in coefficient-based representations of the object in a transformed domain, etc.
[0017] By this technique, even if the encoded object is spatially or temporally excerpted, or filtered (e.g. spectrally), there is increased assurance that the watermark energy remaining after such processing will permit accurate decoding.
[0018] In an illustrative embodiment, the process is highly automated and essentially transparent to a user. The user simply instructs a computer-controlled system to watermark an object, and the system responds by performing the trial watermarking, decoding, making successive adjustments, and repeating as necessary until a final encoded object meeting desired watermark-quality requirements is produced.
[0019] A more elaborate embodiment is a batch image processing system (implemented, e.g., in software run on a Pentium Ill-based personal computer system) that successively opens, watermarks, JPEG-compresses, and re-stores each image in a collection of images. Such systems are well-suited for use by proprietors of large image collections (e.g. news agencies, photo stock houses, etc.).
[0020] In such systems, any of the foregoing iterativeencoding techniques can be employed. Or other iterative encoding techniques can be used. One other such other iterative encoding technique tests a trial encoding against one or more various forms of corruption, and makes adjustments based on the results of the attempted decoding after such corruption(s). The process repeats as necessary.
[0021] In the exemplary system, the image is compressed after watermarking. Compression commonly weakens a watermark since some of the watermark signal components are typically attenuated. The watermark can be made more durable by increasing the watermark's energy, but so doing can have the effect of visibly degrading image quality. Thus, a trade-off must be struck between watermark durability and image fidelity.
[0022] A user-interface associated with the system can allow a user to select one of three watermarking modes: low, medium, and high. In the low mode, image quality is more important than watermark durability. (Such encoding is best-suited for applications in which the image is not expected to be corrupted after distribution.) In the medium mode, image quality and watermark durability are about
equally important. In the high mode, watermark durability is more important than image quality.
[0023] Generally hidden from the user is a watermark intensity parameter—a parameter used by the watermarking algorithm to determine the intensity (e.g. amplitude) of the watermark signal added to the image. In the illustrative system, the parameter has a value between 1 and 16, with 1 representing the lowest-intensity watermark, and 16 representing the highest-intensity watermark. (The actual watermark energy is typically not a linear function of the parameter but can be, e.g., logarithmically related.) The user interface can also allow the user to specify the degree of JPEG compression desired (e.g. on an arbitrary low/medium/high, or numerically defined scale), and indicate whether the compression can be adjusted—if necessary—to achieve the desired watermark durability/image quality trade-off.
[0024] In operation, the system opens the first image file, and sets an initial intensity parameter (i.e. 1 to 16) corresponding to the user's mode selection (low, medium, or high). The resulting watermarked image is then compressed in accordance with the user's instructions. A copy of the compressed file is next made and decompressed. The watermark is then decoded from the decompressed image to check that the watermarked data can be correctly read, and the strength of the watermark signal in the image is assessed.
[0025] This strength metric (which may be, e.g., signalto-noise ratio expressed on a 1-10 scale) indicates the observed durability of the watermark. If the correct watermark was read, the strength metric is then compared against a threshold value corresponding to the selected mode of operation. (I.e. if "low" mode is selected, a relatively low threshold is set on detected strength, whereas if "high" mode is selected, a relatively high threshold is set.)
[0026] If the strength of watermark detected in the decompressed image meets or exceeds the corresponding threshold value, the trial watermarking meets the user's objectives. Thus, the compressed/watermarked image is written to a disk or other output storage device. Another file from the collection is read, and the process repeats.
[0027] If the strength of the watermark detected in the decompressed image does not meet the corresponding threshold value (or if the correct watermark was not read), the system increases the intensity value and watermarks the original image anew. The watermarked image is again compressed, and a copy made and decompressed. Again, the watermark is decoded from the decompressed image to check for accuracy, and the strength of the watermark signal in the image is assessed.
[0028] If the correct watermark was read, and the strength metric meets or exceeds the corresponding threshold, the watermarked/compressed image is written to the output storage device, and the process continues with a new image file.
[0029] If the strength of the watermark, or its readability, still don't meet spec, the intensity can be increased again and the process repeated anew. However, at some point, the intensity parameter may reach a value at which the image quality suffers unacceptably. Thus, the different modes of operation (low, medium, and optionally high) have corresponding internal intensity threshold values beyond which
the system will not increase. If such threshold is reached, the system will not further increase the embedded watermark intensity. Instead, it will next adjust the compression (if the user has permitted the system—through the UI—this flexibility).
[0030] By lowering the degree of compression, encoded watermark energy better persists. An iterative watermark/ compress/decompress/decode procedure thus begins in which the compression is successively backed-off until decoding yields a watermark that meets the readability requirement and strength threshold. (In an illustrated embodiment, the intensity remains fixed at the highest level permitted for that mode while this iterative procedure proceeds.)
[0031] In some systems, a threshold is set beyond which the degree of compression cannot be lowered (again, for quality concerns). If this threshold is reached, and the decoded watermark still does not meet the readability and strength tests, then the image is set-aside for later manual consideration by the operator, and the process continues anew with the next image. (The same type of exception handling occurs if the user has not permitted the system to adjust the compression, and the strength/readability tests cannot be met with watermark intensity within the permitted range.)
[0032] In these iterative systems, processing speed will depend on the size of the increments by which the parameters (e.g. encoding intensity, compression) are successively adjusted. In some embodiments, a fine level of granularity is employed (e.g. changing the intensity by a unit value on successive iterations). In others, larger increments are used. In still others, binary search-type algorithms can be used to hone-in on a nearly-optimal value in a relatively short time.
[0033] The Figures show illustrative screen shots for use in an embodiment implementing certain of the foregoing features. FIG. 1 shows an initial screen, allowing the user to select between (1) starting the batch embedding; or (2) reviewing exceptions/errors from a previous batch embedding process and taking remedial actions (e.g., per FIG. 9 below). FIG. 2 shows the UI permitting the user to specify the mode of operation desired (i.e., low/medium/high).
[0034] FIG. 3 shows a UI for selecting the desired degree of compression, which can be specified either as low, medium, or high, or by setting a numerical compression parameter. The UI of FIG. 3 also allows the user to request a warning if the compressed image file is larger than a specified size, and permits the user to specify progressive scans.
[0035] FIG. 4 shows a UI permitting the user to specify the watermark payload with which the images are to be encoded. Among the fields are Digimarc ID, Distributor ID, Do Not Copy flag, Restricted Use flag, Adult Content Flag, Copyright Year, Image ID, and Transaction ID. The UI also permits the user to specify that certain IDs are to be incremented for each successive image in the batch.
[0036] FIG. 5 shows a UI permitting the user to specify the location of images to be processed. The UI optionally allows the user to preview images as part of the process.
[0037] FIG. 6 shows a UI permitting the user to specify a destination folder for the watermarked images, and output options (e.g. file format).
[0038] FIG. 7 shows a UI permitting the user to specify error and log file settings (e.g. specifying maximum number of errors before stopping, and maximum consecutive errors before stopping).
[0039] FIG. 8 shows a UI presenting errors and warnings to a user, e.g., after a batch of image files has been processed. Included with the error listings are files that were not processed (e.g. because they were found to already have a watermark). Included with the warning listings are files whose size, after compression, exceeded the earlier-specified maximum file size (FIG. 3). The UI also permits the user to review all or selected files.
[0040] FIG. 9 shows a UI that presents an image for review, both after and before encoding/compression, and indicating the size of each image. The screen also includes UI controls permitting the user to adjust the intensity (on the 1 to 16 scale) and the JPEG compression level (either on a low/medium/high scale, or by numerical parameter). After the user has made any desired changes, the "Next Image" button permits the next image to be displayed and processed.
[0041] (Although the just-described arrangement adjusted the intensity until a threshold was reached, and only then adjusted compression, in other systems this need not be the case. For example, variations in intensity and compression might be alternated until a suitable result is achieved.)
[0042] While the just-discussed system checked robustness of the watermark against JPEG-distortion (i.e. a lossy compression/decompression process), the same procedure can be adapted to other types of expected distortion. For example, if an image is expected to be printed and then scanned, such distortion might be modeled by down-sampling and blur-filtering. The system can check trial encodings against such distortion, and adjust the intensity (or other parameter) as needed to provide a desired degree of durability against such corruption.
[0043] Likewise, while the just-discussed system checked trial encoding for two performance metrics (e.g., accurate decoding, and strength of watermark meeting a threshold), in other embodiments different performance metrics can be employed. Other sample metrics include, but are not limited to: speed of watermark decoding being within a threshold; change (i.e. increase or decrease) in bit-rate (e.g. for MPEG video or MP3/MP4 audio) being within a threshold; change in entropy (in the encoded vs. un-encoded object) being within a threshold, etc. (Changes in bit-rate and entropy resulting from watermarking are reviewed, e.g., in WO 99/10837.)
[0044] In addition to testing for watermark robustness after image compression/decompression, the just-described embodiment also included image compression as part of the batch processing. Of course, this step could be omitted, and/or other processing steps (e.g. filtering, cropping, format translation, etc.) desired by the user could be included.
[0045] Watermarking can be applied to myriad forms of information. These include imagery (including video) and audio—whether represented in digital form (e.g. an image comprised of pixels, digital or MPEG video, MP3/MP4 audio, etc.), or in an analog representation (e.g. non-sampled music, printed imagery, banknotes, etc.) Watermarking can be applied to digital content (e.g. imagery, audio) either before or after compression. Watermarking can also be used
|
