US7277595B1 - Method and apparatus for digital image manipulation to remove image blemishes - Google Patents

Method and apparatus for digital image manipulation to remove image blemishes Download PDF

Info

Publication number
US7277595B1
US7277595B1 US10/338,786 US33878603A US7277595B1 US 7277595 B1 US7277595 B1 US 7277595B1 US 33878603 A US33878603 A US 33878603A US 7277595 B1 US7277595 B1 US 7277595B1
Authority
US
United States
Prior art keywords
circle
blemish
pixels
removal tool
function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US10/338,786
Inventor
Russell Reid
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Apple Inc
Original Assignee
Apple Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Apple Inc filed Critical Apple Inc
Priority to US10/338,786 priority Critical patent/US7277595B1/en
Assigned to APPLE COMPUTER, INC. reassignment APPLE COMPUTER, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: REID, RUSSELL
Priority to US11/852,270 priority patent/US7545994B2/en
Application granted granted Critical
Publication of US7277595B1 publication Critical patent/US7277595B1/en
Assigned to APPLE INC. reassignment APPLE INC. CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: APPLE COMPUTER, INC.
Assigned to APPLE INC. reassignment APPLE INC. CHANGE OF NAME (SEE DOCUMENT FOR DETAILS). Assignors: APPLE COMPUTER, INC.
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/40Picture signal circuits
    • H04N1/409Edge or detail enhancement; Noise or error suppression
    • H04N1/4097Removing errors due external factors, e.g. dust, scratches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/46Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/46Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/469Contour-based spatial representations, e.g. vector-coding
    • G06V10/471Contour-based spatial representations, e.g. vector-coding using approximation functions

Definitions

  • the present invention relates to the field of digital photography.
  • the present invention discloses a system for digital image processing.
  • digital photography is rapidly replacing traditional film photography.
  • digital photography there is no waiting for photographic film to be processed. Instead, most digital cameras allow the photographer to instantly review the digitally captured images. With this ability, the photographer may take another shot if he or she is not satisfied with the first digital images.
  • Digital image manipulation programs can be used to manipulate digital photographs in order to correct flaws, modify colors, etc.
  • the potential of digital image manipulation by amateurs remains largely unrealized.
  • the only image manipulation used by most amateur digital photographers is the rotation of digital images to correct the orientation of the image.
  • Digital image manipulation largely remains the domain of experts since quality image manipulation software is often too expensive for the amateur photographer and too difficult to use for the casual user.
  • the present invention introduces an intuitive system for digital manipulation.
  • the system of the present invention allows a user to simply fix small blemishes in digital images.
  • the user merely places a cursor on the image blemish and activates a cursor control device such as a mouse button.
  • the blemish repair system is implemented using one of Dirichlet's theorems.
  • the theorem provides a method of creating a smooth surface within a circle by using the values of points around the circle. To prevent the result from appearing too smooth, some pixelization may be added.
  • FIG. 1 illustrates a block diagram of a computer system that may be used to implement the present invention.
  • FIG. 2 a illustrates one embodiment of an example graphical user interface for the blemish removal system of the present invention.
  • FIG. 2 b illustrates the graphical user interface embodiment of FIG. 2 a where the blemish removal system is being activated on an area of an image.
  • FIG. 2 c illustrates the graphical user interface embodiment of FIG. 2 b after the blemish has been removed from the image.
  • FIG. 2 d illustrates a flow diagram of one embodiment of the present invention.
  • FIG. 3 a illustrates an imaginary number plotted on a Cartesian coordinate system with the real number portion on the x-axis and the imaginary portion on the y-axis.
  • FIG. 3 b illustrates a vector subtraction of a vector to a point in the unit circle from a vector to a point on the unit circle.
  • FIG. 3 c illustrates a method of determining the magnitude of the difference vector of FIG. 3 b.
  • FIG. 4 conceptually illustrates iteration across all the pixels in a circle.
  • FIG. 5 conceptually illustrates iteration around the border pixels of a circle to generate the value of an inner pixel.
  • FIG. 6 illustrates a circle divided into octants.
  • Digital photography is rapidly replacing traditional film photography.
  • One of the most powerful aspects of digital photography is ability to perform image manipulation upon captured digital images.
  • Digital image manipulation programs can be used to manipulate digital photographs in order to correct image flaws, modify image colors, and perform other image corrections.
  • Most image manipulation can be performed on a typical personal computer system such as the iMac personal computers from Apple Computer, Inc. of Cupertino, Calif.
  • FIG. 1 illustrates an exemplary embodiment of a computer system 100 that may be used to implement the teachings of the present invention.
  • Computer system 100 may be a typical personal computer system with a processor 110 , a memory 120 , and a bus controller 130 for controlling a system bus 135 .
  • Various different Input/Output devices may be coupled to the system bus 135 to allow the computer system 100 to interact with the outside world.
  • computer system 100 includes the input/output devices of a hard disk drive (HDD) 169 , a display device 161 (such as a CRT or flat panel display), a keyboard 151 , a cursor control device 153 (such as a computer mouse, track pad, or trackball), a fixed media drive 180 , and network interface 191 for coupling to a network such as local area network (LAN) 195 .
  • HDD hard disk drive
  • display device 161 such as a CRT or flat panel display
  • keyboard 151 such as a CRT or flat panel display
  • a cursor control device 153 such as a computer mouse, track pad, or trackball
  • fixed media drive 180 such as a fixed media drive
  • network interface 191 for coupling to a network such as local area network (LAN) 195 .
  • LAN local area network
  • the image manipulation system 125 of the present invention resides in the memory 120 of computer system 100 as a set of computer instructions.
  • the image manipulation system may also reside as a program on a fixed media device 181 such as Compact Disc Read-Only-Memory (CD-ROM) or Digital Versatile Disc Read-Only-Memory (DVD-ROM).
  • the fixed media device is used with an associated fixed media drive 180 such as a CD-ROM drive or DVD-ROM drive, respectively.
  • the image manipulation system 125 may be downloaded from across a network such as the global Internet 199 accessible through network interface 191 .
  • the first task of the image blemish removal system of the present invention is to acquire digital images to process.
  • the acquired images are normally stored onto a mass storage device such as hard disk drive (HDD) 169 .
  • the image manipulation system 125 may acquire digital images from a number of different sources.
  • FIG. 1 illustrates a digital camera system 147 coupled to serial bus 145 controlled by serial bus interface 140 of the computer system 100 .
  • Serial bus 145 may be an IEEE 1394 Firewire serial bus, a Universal Serial Bus (USB), or any other type of serial bus.
  • the system is able to automatically recognize when digital camera system 147 is coupled to computer system 100 such that the digital images in digital camera system 147 can be acquired.
  • the system can also acquire images in a number of other manners.
  • film photographs may be placed onto a floppy disc (not shown) or onto a fixed media device 181 by a film developer at the request of the film photographer.
  • Such photos may be acquire by image manipulation system 125 by having the user place the floppy disc (not shown) or the fixed media device 181 into the appropriate drive such as floppy disc drive (not show) or the fixed media drive 180 .
  • Digital images may also be obtained through a computer network interface 191 of computer system 100 .
  • a user may obtain digital images from a World Wide Web (WWW) site on the global Internet 199 .
  • Digital images may also be obtained using a digital scanner (not shown) coupled to computer system 100 as is well known in the art.
  • WWW World Wide Web
  • FIG. 2 d To repair a blemish in a digital image, some embodiments provide a method illustrated in FIG. 2 d .
  • This process (1) identifies (at 250 ) a circle that surrounds the blemish, (2) interpolates (at 252 ) a smooth function defined by a first set of pixels that define a border of the circle, (3) fills (at 254 ) the circle with a second set of pixels created by the function, and (4) reduces (at 256 ) the smoothness of the set of pixels by causing pixelization that is a function of a brightness variance around the circle.
  • Captured digital images do not always appear the way we would like the digital images to appear. For example, acne appearing at an inopportune time can mar the digital image captured of a teen-ager on his Senior Prom. An airplane in the corner sky may disturb any otherwise perfect digital image of a beautiful sunset in a nature setting. A particle of dust on the Charge Coupled Device (CCD) sensor of a digital camera, or on its lens, may result in a black spot on every image. Such small undesirable flaws in a digital image are referred to as ‘blemishes.’
  • CCD Charge Coupled Device
  • Film photographs tend to be even more susceptible to having image blemishes.
  • the source film may become scratched causing a white line or white dot to appear on a photograph.
  • Hair or dust may be present on a film lens or during the scan of a traditional film-based photograph used to bring the photograph into the digital domain. These blemishes distract the viewer from completely enjoying the image.
  • the blemish removal solution should be simple and intuitive such that any computer user could easily use the blemish removal tool.
  • FIG. 2 a illustrates one embodiment of an example graphical user interface for the blemish removal system of the present invention.
  • an image window 210 contains an image of a house to use for a real estate listing web site.
  • Frisbee a child left his Frisbee 230 in the yard thus disturbing the perfect image of the house.
  • a user may select the blemish removal tool by clicking on button 220 .
  • FIG. 2 b illustrates the graphical user interface embodiment of FIG. 2 a after the user has clicked on the blemish removal button 220 .
  • the system displays a small circle 240 on the image that may be moved using a cursor control device such as a mouse.
  • the size of circle 240 is determined as a function of the size of the overall image.
  • One specific embodiment sets the radius of small circle as:
  • radius width * height 125 ( 1 )
  • the user is allowed to determine the size of circle 240 .
  • the blemish removal tool will work best with small circles.
  • FIG. 2 c illustrates the graphical user interface embodiment of FIG. 2 b after the blemish has been repaired.
  • the user is given the option of creating completely new pixels for the blemish area or creating a blend of the existing pixels and new pixels.
  • a number of different methods of performing the blemish removal may be used to implement the invention.
  • one of Dirichiet's Theorems in the imaginary number domain is used to perform blemish removal.
  • the use of the Dirichlet Theorem allows the blemish repair tool to calculate the only unwrinkled surface that fits exactly into the existing photo at the edges of a circle around the blemish point.
  • the Dirichlet Theorem in the imaginary number domain used by the present invention is expressed as follows:
  • Equation (2) operates by computing a value for internal pixel (Z) as a function of all the vales of pixels around the unit circle (f (e i ⁇ )). Substituting equations (3), (4), and (5) into equation (2) yields:
  • equation (6) can be simplified to:
  • Equation (6) serves as the basis for one embodiment of the blemish removal tool of the present invention.
  • Every pixel in the circle surrounding the blemish may be fixed by using equation (7) on every pixel within the blemish circle. This may be performed by iterating across a square and fixing every point within the circle in the square as illustrated in FIG. 4 .
  • the following code can fix every pixel within radius r2 of the center of the blemish defined by point xcenter, ycenter:
  • Equation (7) can be implemented with a discrete integral that integrates around a circle of pixels that surround a blemish as conceptually illustrated by FIG. 5 .
  • the information from all the pixels around the circle edge affects the value of each pixel within the circle.
  • the following code implements a method of determining the red, green, and blue values for a replacement pixel within the blemish area.
  • the blemish removal system described in the previous section will provide a perfectly smooth replacement for the area within the blemish circle. However, a perfectly smooth replacement may appear too perfect. Thus, some pixelization may be created to prevent the repair from appearing unnaturally smooth. The pixelization may be added equally to the red, blue, and green components in order to prevent undesirable color changes.
  • One embodiment operates by first dividing the blemish border into octants as illustrated by FIG. 6 . Next, a brightness average and variance for each octant is calculated. The brightness variance and average of the octants are sorted by the amount of variance and the picks the second highest amount of variance is selected to replicate in the blemish repair area.
  • the blemish repair tool may generate Gaussian noise, the statistical type found in general in digital photos, to modulate the smoothed surface with the computed Gaussian noise.
  • Gaussian noise the statistical type found in general in digital photos.
  • the modulation with Gaussian noise achieves a pixelization that blends with the existing photo.

Abstract

To better realize the great potential of amateur digital photography, the present invention introduces an intuitive system for digital manipulation. The system of the present invention allows a user to simply fix small blemishes in digital images. In one embodiment, the user merely places a cursor on the image blemish and activates a cursor control device such as a mouse button. In one embodiment, the blemish repair system is implemented using one of Dirichlet's theorems. The theorem provides a method of creating a smooth surface within a circle by using the values of points around the circle. To prevent the result from appearing too smooth, some pixelization may be added.

Description

FIELD OF THE INVENTION
The present invention relates to the field of digital photography. In particular the present invention discloses a system for digital image processing.
BACKGROUND OF THE INVENTION
With improved technology and sharply reduced digital camera pricing, digital photography is rapidly replacing traditional film photography. With digital photography, there is no waiting for photographic film to be processed. Instead, most digital cameras allow the photographer to instantly review the digitally captured images. With this ability, the photographer may take another shot if he or she is not satisfied with the first digital images.
Digital photography has given amateur digital photographers new avenues of image distribution. Amateur digital photographs can easily be published on the world-wide web (WWW) to allow anyone on the Internet to view the digital photographs. Internet users can also email digital photographs as attachments or embedded within HTML formatted email messages.
One of the most powerful aspects of digital photography is ability to perform digital image manipulation. Digital image manipulation programs can be used to manipulate digital photographs in order to correct flaws, modify colors, etc. However, the potential of digital image manipulation by amateurs remains largely unrealized. The only image manipulation used by most amateur digital photographers is the rotation of digital images to correct the orientation of the image. Digital image manipulation largely remains the domain of experts since quality image manipulation software is often too expensive for the amateur photographer and too difficult to use for the casual user.
To better realize the great potential of amateur digital photography, it would be very desirable to have improved digital image manipulation tools that are available for amateur digital photography enthusiasts. These digital image manipulation tools should be highly intuitive yet provide powerful features for manipulating digital images.
SUMMARY OF THE INVENTION
To better realize the great potential of amateur digital photography, the present invention introduces an intuitive system for digital manipulation. The system of the present invention allows a user to simply fix small blemishes in digital images. In one embodiment, the user merely places a cursor on the image blemish and activates a cursor control device such as a mouse button.
In one embodiment, the blemish repair system is implemented using one of Dirichlet's theorems. The theorem provides a method of creating a smooth surface within a circle by using the values of points around the circle. To prevent the result from appearing too smooth, some pixelization may be added.
Other objects, features, and advantages of present invention will be apparent from the company drawings and from the following detailed description.
BRIEF DESCRIPTION OF THE DRAWINGS
The objects, features, and advantages of the present invention will be apparent to one skilled in the art in view of the following detailed description in which:
FIG. 1 illustrates a block diagram of a computer system that may be used to implement the present invention.
FIG. 2 a illustrates one embodiment of an example graphical user interface for the blemish removal system of the present invention.
FIG. 2 b illustrates the graphical user interface embodiment of FIG. 2 a where the blemish removal system is being activated on an area of an image.
FIG. 2 c illustrates the graphical user interface embodiment of FIG. 2 b after the blemish has been removed from the image.
FIG. 2 d illustrates a flow diagram of one embodiment of the present invention.
FIG. 3 a illustrates an imaginary number plotted on a Cartesian coordinate system with the real number portion on the x-axis and the imaginary portion on the y-axis.
FIG. 3 b illustrates a vector subtraction of a vector to a point in the unit circle from a vector to a point on the unit circle.
FIG. 3 c illustrates a method of determining the magnitude of the difference vector of FIG. 3 b.
FIG. 4 conceptually illustrates iteration across all the pixels in a circle.
FIG. 5 conceptually illustrates iteration around the border pixels of a circle to generate the value of an inner pixel.
FIG. 6 illustrates a circle divided into octants.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT
A method and apparatus for the digital manipulation of images to removal blemishes is disclosed. In the following description, for purposes of explanation, specific nomenclature is set forth to provide a thorough understanding of the present invention. However, it will be apparent to one skilled in the art that these specific details are not required in order to practice the present invention. For example, the present invention has been described with reference to one of Dirichelt's Theorems. However, the system of the present invention may use other types of function interpolation methods.
Digital Image Blemishes
Digital photography is rapidly replacing traditional film photography. One of the most powerful aspects of digital photography is ability to perform image manipulation upon captured digital images. Digital image manipulation programs can be used to manipulate digital photographs in order to correct image flaws, modify image colors, and perform other image corrections. Most image manipulation can be performed on a typical personal computer system such as the iMac personal computers from Apple Computer, Inc. of Cupertino, Calif.
Image Processing Computer System
FIG. 1 illustrates an exemplary embodiment of a computer system 100 that may be used to implement the teachings of the present invention. Computer system 100 may be a typical personal computer system with a processor 110, a memory 120, and a bus controller 130 for controlling a system bus 135. Various different Input/Output devices may be coupled to the system bus 135 to allow the computer system 100 to interact with the outside world. For example, computer system 100 includes the input/output devices of a hard disk drive (HDD) 169, a display device 161 (such as a CRT or flat panel display), a keyboard 151, a cursor control device 153 (such as a computer mouse, track pad, or trackball), a fixed media drive 180, and network interface 191 for coupling to a network such as local area network (LAN) 195.
The image manipulation system 125 of the present invention resides in the memory 120 of computer system 100 as a set of computer instructions. The image manipulation system may also reside as a program on a fixed media device 181 such as Compact Disc Read-Only-Memory (CD-ROM) or Digital Versatile Disc Read-Only-Memory (DVD-ROM). The fixed media device is used with an associated fixed media drive 180 such as a CD-ROM drive or DVD-ROM drive, respectively. Alternatively, the image manipulation system 125 may be downloaded from across a network such as the global Internet 199 accessible through network interface 191.
The first task of the image blemish removal system of the present invention is to acquire digital images to process. Referring to FIG. 1, the acquired images are normally stored onto a mass storage device such as hard disk drive (HDD) 169. In one embodiment of the present invention, the image manipulation system 125 may acquire digital images from a number of different sources.
To acquire images, the system of the present invention may include driver software necessary to automatically identify and interoperate with several different types of digital cameras that may be coupled to the host computer system. For example, FIG. 1 illustrates a digital camera system 147 coupled to serial bus 145 controlled by serial bus interface 140 of the computer system 100. Serial bus 145 may be an IEEE 1394 Firewire serial bus, a Universal Serial Bus (USB), or any other type of serial bus. In a preferred embodiment, the system is able to automatically recognize when digital camera system 147 is coupled to computer system 100 such that the digital images in digital camera system 147 can be acquired.
The system can also acquire images in a number of other manners. For example, film photographs may be placed onto a floppy disc (not shown) or onto a fixed media device 181 by a film developer at the request of the film photographer. Such photos may be acquire by image manipulation system 125 by having the user place the floppy disc (not shown) or the fixed media device 181 into the appropriate drive such as floppy disc drive (not show) or the fixed media drive 180.
Digital images may also be obtained through a computer network interface 191 of computer system 100. For example, a user may obtain digital images from a World Wide Web (WWW) site on the global Internet 199. Digital images may also be obtained using a digital scanner (not shown) coupled to computer system 100 as is well known in the art.
To repair a blemish in a digital image, some embodiments provide a method illustrated in FIG. 2 d. This process (1) identifies (at 250) a circle that surrounds the blemish, (2) interpolates (at 252) a smooth function defined by a first set of pixels that define a border of the circle, (3) fills (at 254) the circle with a second set of pixels created by the function, and (4) reduces (at 256) the smoothness of the set of pixels by causing pixelization that is a function of a brightness variance around the circle.
Image Blemishes
Captured digital images do not always appear the way we would like the digital images to appear. For example, acne appearing at an inopportune time can mar the digital image captured of a teen-ager on his Senior Prom. An airplane in the corner sky may disturb any otherwise perfect digital image of a beautiful sunset in a nature setting. A particle of dust on the Charge Coupled Device (CCD) sensor of a digital camera, or on its lens, may result in a black spot on every image. Such small undesirable flaws in a digital image are referred to as ‘blemishes.’
Film photographs tend to be even more susceptible to having image blemishes. The source film may become scratched causing a white line or white dot to appear on a photograph. Hair or dust may be present on a film lens or during the scan of a traditional film-based photograph used to bring the photograph into the digital domain. These blemishes distract the viewer from completely enjoying the image.
It would therefore be desirable to provide users with a blemish removal tool for repairing such image blemishes. Ideally, the blemish removal solution should be simple and intuitive such that any computer user could easily use the blemish removal tool.
Blemish Removal Overview
The present invention introduces a simple and intuitive blemish removal tool that may be used to remove unsightly blemishes from digital images. For example, FIG. 2 a illustrates one embodiment of an example graphical user interface for the blemish removal system of the present invention. In FIG. 2 a, an image window 210 contains an image of a house to use for a real estate listing web site. Unfortunately, a child left his Frisbee 230 in the yard thus disturbing the perfect image of the house. In order to remove the blemish (the Frisbee), a user may select the blemish removal tool by clicking on button 220.
FIG. 2 b illustrates the graphical user interface embodiment of FIG. 2 a after the user has clicked on the blemish removal button 220. The system then displays a small circle 240 on the image that may be moved using a cursor control device such as a mouse. In one embodiment, the size of circle 240 is determined as a function of the size of the overall image. One specific embodiment sets the radius of small circle as:
radius = width * height 125 ( 1 )
In another embodiment, the user is allowed to determine the size of circle 240. However, the blemish removal tool will work best with small circles.
Referring again to FIG. 2 b, the user places circle 240 such that it surrounds the blemish. After the user has placed circle 240 on the blemish, the user simply requests the user to repair the blemish. The request may be indicated by activating a switch on a cursor control device such as a mouse button. FIG. 2 c illustrates the graphical user interface embodiment of FIG. 2 b after the blemish has been repaired. In one embodiment, the user is given the option of creating completely new pixels for the blemish area or creating a blend of the existing pixels and new pixels.
Blemish Removal Embodiment Details
A number of different methods of performing the blemish removal may be used to implement the invention. In one embodiment, one of Dirichiet's Theorems in the imaginary number domain is used to perform blemish removal. The use of the Dirichlet Theorem allows the blemish repair tool to calculate the only unwrinkled surface that fits exactly into the existing photo at the edges of a circle around the blemish point.
The Math
The Dirichlet Theorem in the imaginary number domain used by the present invention is expressed as follows:
U ( Z ) = 1 2 π 0 2 π f ( θ ) 1 - Z 2 Z - θ θ ( 2 )
where Z is an imaginary number that can be expressed in the form of:
Z=x+yi  (3)
Imaginary numbers are commonly plotted on a Cartesian coordinate system with the real number portion on the x-axis and the imaginary portion on the y-axis as illustrated in FIG. 3 a. The absolute value of an imaginary number is the length of the imaginary vector x+yi. Specifically:
|Z|=|x+yi|=√{square root over (x 2 +y 2)}  (4)
The Dirichlet Theorem of equation (2) generates a smooth two-dimensional function for all the area enclosed within a unit circle on the real/imaginary plane since:
e =cos θ+i(sin θ)  (5)
Equation (2) operates by computing a value for internal pixel (Z) as a function of all the vales of pixels around the unit circle (f (e)). Substituting equations (3), (4), and (5) into equation (2) yields:
U ( x + y ) = 1 2 π 0 2 π f ( cos θ + ( sin θ ) ) 1 - ( x 2 + y 2 ) x + y - cos θ - ( sin θ ) 2 θ ( 6 )
The denominator contains the vector subtraction x+yi−cos θ−(sin θ)i which is subtracting the vector of the x+yi position (x, y) being analyzed from the current vector on the unit circle (x0, y0). This vector subtraction is illustrated in FIG. 3 b. However, as seen in equation (6), the absolute value of the difference vector is desired. The absolute value of the difference vector can be determined with the Pythagorean Theorem as illustrated in FIG. 3 c. Thus, equation (6) can be simplified to:
U ( x + y ) = 1 2 π 0 2 π f ( x 0 , y 0 ) 1 - ( x 2 + y 2 ) ( x - x 0 ) 2 + ( y - y 0 ) 2 θ ( 7 )
Equation (6) serves as the basis for one embodiment of the blemish removal tool of the present invention.
An Implementation
Every pixel in the circle surrounding the blemish may be fixed by using equation (7) on every pixel within the blemish circle. This may be performed by iterating across a square and fixing every point within the circle in the square as illustrated in FIG. 4. Specifically, the following code can fix every pixel within radius r2 of the center of the blemish defined by point xcenter, ycenter:
For (i = −r2; i <= r2; i++) // Iterate across the rows (x)
{
for (j = −r2; j <= r2; j++) // Iterate down the column (y)
{
if (i*i + j*j >= r2*r2) continue; // If not in circle, then move
to next pixel in square.
fixpixel(imageBuffer, xcenter+i, ycenter+j);
}
}
Equation (7) can be implemented with a discrete integral that integrates around a circle of pixels that surround a blemish as conceptually illustrated by FIG. 5. The information from all the pixels around the circle edge affects the value of each pixel within the circle. The following code implements a method of determining the red, green, and blue values for a replacement pixel within the blemish area.
// x1 = Current x value inside normalized unit circle
// y1 = Current y value inside normalized unit circle
rsum = gsum = bsum = 0.0; // Start with zero
angleIncrement = 1.183 /((float) r1); // Determine angle increment as
function of radius
//Determine # of points around circle as 2*Pi/angle + 1
nCirclePoints = floor(1+ (2*3.14159265358979)/angleIncrement);
k = 0;   // Create sin & cos look-up table for efficiency
for (theta = 0; theta < 2* 3.14159265358979; theta += angleIncrement)
{
cosTheta[k  ] = cos(theta);
sinTheta[k++] = sin(theta);
}
// Determine Final increment to complete the circle
finalIncrement = 2*3.14159265358979 = theta + angleIncrement;
zmodsq = x1*x1 + y1*y1; // Squared(|Z|) = x1*x1 + y1*y1
npts = 0;
for (k = 0; k < nCirclePoints; k++) //Iterate around the blemish border.
{
npts++;
x2 = cosTheta[k]; // x value of current point on normalized unit
circle
y2 = sinTheta[k]; // Y value of current point on normalized unit
circle
xdelta = floor(r1*x2 + 0.5); // x value of pixel on circle outside
blemish
ydelta = floor(r1*y2 + 0.5); // y value of pixel on circle outside
blemish
// Calculate address of the pixel on the blemish border
borderpix = imageBuffer + (ycenter + ydelta)*rowBytes +
(xcenter+xdelta)*bytesPerPixel;
r = (float) borderpix[1]; // Grab red value of border pixel
g = (float) borderpix[2]; // Grab Green value of border pixel
b = (float) borderpix[3]; // Grab blue value of border pixel
if (k < nCirclePoints − 1) // Last circle border pixel?
{
rsum += r* (1 − zmodsq) / ((x1−x2)*(x1−x2) +
(y1−y2)*(y1−y2));
gsum += g* (1 − zmodsq) / ((x1−x2)*(x1−x2) +
(y1−y2)*(y1−y2));
bsum += b* (1 − zmodsq) / ((x1−x2)*(x1−x2) +
(y1−y2)*(y1−y2));
}
else
{
rsum += finalIncrement/angleIncrement* r* (1−zmodsq) /
((x1−x2)*(x1−x2)+(y1−y2)*(y1−y2));
gsum += finalIncrement/angleIncrement* g* (1−zmodsq) /
((x1−x2)*(x1−x2)+(y1−y2)*(y1−y2));
bsum += finalIncrement/angleIncrement* b* (1−zmodsq) /
((x1−x2)*(x1−x2)+(y1−y2)*(y1−y2));
}
}
rfinal = rsum/(npts−1.0 + finalIncrement/angleIncrement); // Div by # of
pnts−the 1/2*Pi div
qfinal = gsum/(npts−1.0 + finalIncrement/angleIncrement); // Div by # of
pnts−the 1/2*Pi div
bfinal = bsum/(npts−1.0 + finalIncrement/angleIncrement); // Div by # of
pnts−the 1/2*Pi div

where image buffer contains the image, r1 is the radius of the pixel circle around the blemish, xcenter is the x value of center of the blemish, y is the y value of center of the blemish, and x1 & y1 define the point in the unit circle being fixed.
Blemish Repair Tweaks
The blemish removal system described in the previous section will provide a perfectly smooth replacement for the area within the blemish circle. However, a perfectly smooth replacement may appear too perfect. Thus, some pixelization may be created to prevent the repair from appearing unnaturally smooth. The pixelization may be added equally to the red, blue, and green components in order to prevent undesirable color changes.
One embodiment operates by first dividing the blemish border into octants as illustrated by FIG. 6. Next, a brightness average and variance for each octant is calculated. The brightness variance and average of the octants are sorted by the amount of variance and the picks the second highest amount of variance is selected to replicate in the blemish repair area.
To further blend the blemish repair into the existing image, the blemish repair tool may generate Gaussian noise, the statistical type found in general in digital photos, to modulate the smoothed surface with the computed Gaussian noise. The modulation with Gaussian noise achieves a pixelization that blends with the existing photo.
The following code segments the circle around the blemish into eight octants, calculates the mean and variance in brightness in each of those segments, sorts them, chooses an appropriate variance level, and generates a matching Gaussian noise:
brightnessaccum = brightnessSquaredAccum = 0.0;  //clear accumulators
octant = previousoctant = 0;
for (k = 0; k < nCirclePoints; k++ ) { //Iterate around the blemish border.
x2 = cosTheta[k]; // x value of current point on normalized unit
circle
y2 = sinTheta[k]; // y value of current point on normalized unit
circle
xdelta = floor (r1*x2 + 0.5); // x value of pixel on circle outside
blemish
ydelta = floor (r1*y2 + 0.5); // y value of pixel on circle outside
blemish
// Calculate address of the pixel on the blemish border & grab it
pix = imageBuffer + (ycenter+ydelta)*BytesPerRow +
(xcenter+xdelta)*bytesPerPixel;
r = (float) pix[1]; // Grab red value of border pixel
g = (float) pix[2]; // Grab Green value of border pixel
b = (float) pix[3]; // Grab blue value of border pixel
brightness r+g+b; // Determine brightness of
borderpixel
octant = k/(nCirclePoints/8); // Determine what octant we are in
if (octant > previousOctant) {
//calculate average brightness
avg[previousOctant] = (float) brightnessaccum/
(nCirclepPoints/8);
// Calculate variance for previous octant
var[previousOctant] = brightnessSquaredAccum/
(nCirclePoints/8) −
avg[previousOctant]*avg[previousOctant];
previousOctant = octant;
brightnessaccum = 0.0;    //Clear accumulators
brightnessSquaredAccum = 0.0;
}
brightnessaccum += (double) brightness;
brightnessSquaredAccum += (double) brightness*brightness;
}
//Sort array of mean and variance values for octants around reference
// circle from high variance to low variance.
for (m = 0; m < 7; m++) {
for (n = 0; n < 7−m; n++) {
if (var[n+1] < var[n]) { //swap values if lower
var1 = var[n]; var[n] = var[n+1]; var[n+1] = var1;
avg1 = avg[n]; avg[n] = avg[n+1]; avg[n+1] = avg1;
}
}
}
mean = (avg[5] + avg[6] + avg[7])/3.0; //Array now sorted by size
of variance.
variance = var[1]; // Choose second highest
variance
rNoise = sqrt(−2.0 * variance * log(1.0 − (float) rand( )/((double)
RAND_MAX + 0.1)));
theta2 = (float) 2 * 3.14159265 * rand( )/((double) RAND_MAX + 0.1)
−3.14159265;
pointgain = (mean + rNoise*cos(theta2))/mean;
// Generate gain, which has Gaussian
//distribution, for specific pixel
//Some adjustments
if (pointgain < (mean − 1.8*sqrt(variance))/mean)
pointgain=(mean − 1.8*sqrt(variance))/mean;
if (pointgain > (mean + 2.4*sqrt(variance))/mean)
pointgain=(mean + 2.4*sqrt(variance))/mean;
// Adjust the brightness of the pixel color components
rfinal = pointgain * rpixel; gfinal = pointgain * gpixel; bfinal =
pointgain * bpixel;

The above code to cause the pixelization may be combined with the code to perform the pixel repair for a more efficient solution. Specifically, the brightness calculation only needs to be performed once.
The foregoing has described a method and apparatus for the digital manipulation of images to removal blemishes. It is contemplated that changes and modifications may be made by one of ordinary skill in the art, to the materials and arrangements of elements of the present invention without departing from the scope of the invention.

Claims (27)

1. A method of repairing a blemish in a digital image, said method comprising:
identifying a circle about said blemish;
interpolating a smooth function defined by a first set of pixels that define a border of said circle around said blemish;
filling said circle with a second set of pixels created by said function;
reducing a smoothness of the second set of pixels within said circle; and
causing pixelization in said circle to reduce said smoothness, wherein said pixelization is a function of a brightness variance around said circle.
2. The method as claimed in claim 1 wherein interpolating a smooth function is performed using Dirichlet's Theorem.
3. The method as claimed in claim 1 wherein said pixelization includes Gaussian noise.
4. The method as claimed in claim 1 wherein interpolating a smooth function is performed by performing a discrete integral around said circle.
5. A method of repairing a blemish in a digital image, said method comprising:
identifying a circle about said blemish;
interpolating a smooth function defined by a first set of pixels that define a border of said circle around said blemish;
filling said circle with a second set of pixels created by said function; and
reducing a smoothness of the second set of pixels within said circle, wherein a radius of said circle is a function of a size of said digital image.
6. A method of repairing a blemish in a digital image, the method comprising:
activating a blemish removal control to display a blemish removal tool;
moving said blemish removal tool to identify a circle with the blemish;
performing a cursor control operation with said blemish removal tool to automatically remove a blemish; and
causing pixelization in said circle to reduce a smoothness, wherein said pixelization is a function of a brightness variance around said circle.
7. The method of claim 6, wherein the blemish removal tool comprises a visible boundary of a circle.
8. The method of claim 7, wherein identifying said circle comprises surrounding the blemish with said visible boundary of the circle.
9. The method of claim 6, wherein identifying the circle and automatically removing the blemish comprises a cursor control operation of the blemish removal tool.
10. The method of claim 6, wherein a cursor control device operates on the blemish removal tool.
11. The method of claim 6, wherein the cursor control operation is a click.
12. A computer readable medium comprising a computer program for repairing a blemish in a digital image said computer program comprising sets of instructions for:
identifying a circle about said blemish, wherein a radius of said circle is a function of a size of said digital image;
interpolating a smooth function defined by a first set of pixels that define a border of said circle around said blemish;
filling said circle with a second set of pixels created by said function; and
reducing a smoothness of the second set of pixels within said circle.
13. The computer readable medium as claimed in claim 12 wherein said instructions for interpolating a smooth function comprise instructions for using Dirichlet's Theorem.
14. The computer readable medium claimed in claim 12, said computer program further comprising a set of instructions for:
causing pixelization in said circle to reduce said smoothness.
15. The computer readable medium as claimed in claim 14 wherein said pixelization comprises Gaussian noise.
16. The computer readable medium as claimed in claim 12 wherein said set of instructions for interpolating a smooth function comprises instructions for performing a discrete integral around said circle.
17. The computer readable medium of claim 12, wherein said set of instructions for reducing a smoothness comprises instructions for adding Gaussian noise.
18. A computer readable medium comprising a computer program for repairing a blemish in a digital image, said computer program comprising sets of instructions for:
identifying a circle about said blemish;
interpolating a smooth function defined by a first set of pixels that define a border of said circle around said blemish;
filling said circle with a second set of pixels created by said function;
reducing a smoothness of the second set of pixels within said circle; and
causing pixelization in said circle to reduce said smoothness, wherein said pixelization is a function of a brightness variance around said circle.
19. A graphical user interface (“GUI”) for an application, the GUI comprising:
a display area for displaying an image;
a blemish removal tool; and
a blemish removal control for causing the GUI to display the blemish removal tool, wherein the blemish removal tool is for identifying a circle with a blemish in the image and for activating to cause the automated removal of a blemish from the image, wherein a radius of said circle is a function of a size of said image.
20. The GUI of claim 19, wherein the blemish removal tool is movable using a cursor control device.
21. The GUI of claim 19, wherein the blemish removal tool activates through an operation of the cursor control device.
22. The GUI of claim 19, wherein the automated removal of the blemish is achieved by filling in a selected circle with pixels based on the pixels around the selected circle.
23. The GUI of claim 22, wherein the automated removal of the blemish is further achieved by adding noise to the selected circle.
24. The GUI of claim 23, wherein the noise is Gaussian noise.
25. The GUI of claim 19, wherein the blemish removal tool for identifying the circle is movable to surround the circle.
26. The GUI of claim 25, wherein the blemish removal tool is activated by a switch on the cursor control device.
27. The GUI of claim 19, wherein the blemish removal tool acts as a cursor, moves via a cursor control device movement action, and activates by a cursor control device click.
US10/338,786 2003-01-06 2003-01-06 Method and apparatus for digital image manipulation to remove image blemishes Expired - Fee Related US7277595B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US10/338,786 US7277595B1 (en) 2003-01-06 2003-01-06 Method and apparatus for digital image manipulation to remove image blemishes
US11/852,270 US7545994B2 (en) 2003-01-06 2007-09-07 Method and apparatus for digital image manipulation to remove image blemishes

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US10/338,786 US7277595B1 (en) 2003-01-06 2003-01-06 Method and apparatus for digital image manipulation to remove image blemishes

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US11/852,270 Continuation US7545994B2 (en) 2003-01-06 2007-09-07 Method and apparatus for digital image manipulation to remove image blemishes

Publications (1)

Publication Number Publication Date
US7277595B1 true US7277595B1 (en) 2007-10-02

Family

ID=38535860

Family Applications (2)

Application Number Title Priority Date Filing Date
US10/338,786 Expired - Fee Related US7277595B1 (en) 2003-01-06 2003-01-06 Method and apparatus for digital image manipulation to remove image blemishes
US11/852,270 Expired - Lifetime US7545994B2 (en) 2003-01-06 2007-09-07 Method and apparatus for digital image manipulation to remove image blemishes

Family Applications After (1)

Application Number Title Priority Date Filing Date
US11/852,270 Expired - Lifetime US7545994B2 (en) 2003-01-06 2007-09-07 Method and apparatus for digital image manipulation to remove image blemishes

Country Status (1)

Country Link
US (2) US7277595B1 (en)

Cited By (43)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070003157A1 (en) * 2005-06-29 2007-01-04 Xerox Corporation Artifact removal and quality assurance system and method for scanned images
US20070297691A1 (en) * 2003-01-06 2007-12-27 Russell Reid Method and apparatus for digital image manipulation to remove image blemishes
US20080100726A1 (en) * 2006-10-26 2008-05-01 Cazier Robert P Blemish Repair Tool For Digital Photographs In A Camera
US20090202170A1 (en) * 2008-02-11 2009-08-13 Ben Weiss Blemish Removal
US7804897B1 (en) 2002-12-16 2010-09-28 Apple Inc. Method for implementing an improved quantizer in a multimedia compression and encoding system
US20100279418A1 (en) * 2009-05-04 2010-11-04 Loren Robert Larson Glucose meter adaptable for use with handheld devices, and associated communication network
US20110090380A1 (en) * 2009-10-20 2011-04-21 Apple Inc. Image signal processor front-end image data processing system and method
US20110090370A1 (en) * 2009-10-20 2011-04-21 Apple Inc. System and method for sharpening image data
US20110091101A1 (en) * 2009-10-20 2011-04-21 Apple Inc. System and method for applying lens shading correction during image processing
US20110090371A1 (en) * 2009-10-20 2011-04-21 Apple Inc. System and method for detecting and correcting defective pixels in an image sensor
US20110090351A1 (en) * 2009-10-20 2011-04-21 Apple Inc. Temporal filtering techniques for image signal processing
US8428127B2 (en) 2002-07-15 2013-04-23 Apple Inc. Method of performing rate control for a compression system
US8471932B2 (en) 2010-09-30 2013-06-25 Apple Inc. Spatial filtering for image signal processing
US8477843B2 (en) 2002-12-16 2013-07-02 Apple Inc. Method of implementing improved rate control for a multimedia compression and encoding system
US8488055B2 (en) 2010-09-30 2013-07-16 Apple Inc. Flash synchronization using image sensor interface timing signal
US8493482B2 (en) 2010-08-18 2013-07-23 Apple Inc. Dual image sensor image processing system and method
US8508621B2 (en) 2010-09-30 2013-08-13 Apple Inc. Image sensor data formats and memory addressing techniques for image signal processing
US8508612B2 (en) 2010-09-30 2013-08-13 Apple Inc. Image signal processor line buffer configuration for processing ram image data
US8525895B2 (en) 2010-07-29 2013-09-03 Apple Inc. Binning compensation filtering techniques for image signal processing
US8531542B2 (en) 2010-09-01 2013-09-10 Apple Inc. Techniques for acquiring and processing statistics data in an image signal processor
US8605167B2 (en) 2010-09-01 2013-12-10 Apple Inc. Flexible color space selection for auto-white balance processing
US8629913B2 (en) 2010-09-30 2014-01-14 Apple Inc. Overflow control techniques for image signal processing
US8736700B2 (en) 2010-09-30 2014-05-27 Apple Inc. Techniques for synchronizing audio and video data in an image signal processing system
US8786625B2 (en) 2010-09-30 2014-07-22 Apple Inc. System and method for processing image data using an image signal processor having back-end processing logic
US8817120B2 (en) 2012-05-31 2014-08-26 Apple Inc. Systems and methods for collecting fixed pattern noise statistics of image data
US8872946B2 (en) 2012-05-31 2014-10-28 Apple Inc. Systems and methods for raw image processing
US8917336B2 (en) 2012-05-31 2014-12-23 Apple Inc. Image signal processing involving geometric distortion correction
US8922704B2 (en) 2010-09-01 2014-12-30 Apple Inc. Techniques for collection of auto-focus statistics
US8953882B2 (en) 2012-05-31 2015-02-10 Apple Inc. Systems and methods for determining noise statistics of image data
US9014504B2 (en) 2012-05-31 2015-04-21 Apple Inc. Systems and methods for highlight recovery in an image signal processor
US9025867B2 (en) 2012-05-31 2015-05-05 Apple Inc. Systems and methods for YCC image processing
US9031319B2 (en) 2012-05-31 2015-05-12 Apple Inc. Systems and methods for luma sharpening
US9077943B2 (en) 2012-05-31 2015-07-07 Apple Inc. Local image statistics collection
US9105078B2 (en) 2012-05-31 2015-08-11 Apple Inc. Systems and methods for local tone mapping
US9131196B2 (en) 2012-05-31 2015-09-08 Apple Inc. Systems and methods for defective pixel correction with neighboring pixels
US9142012B2 (en) 2012-05-31 2015-09-22 Apple Inc. Systems and methods for chroma noise reduction
CN104952049A (en) * 2015-06-23 2015-09-30 浙江工商大学 Image inpainting method based on domain divergence interpolation
US9332239B2 (en) 2012-05-31 2016-05-03 Apple Inc. Systems and methods for RGB image processing
US9398205B2 (en) 2010-09-01 2016-07-19 Apple Inc. Auto-focus control using image statistics data with coarse and fine auto-focus scores
US11089247B2 (en) 2012-05-31 2021-08-10 Apple Inc. Systems and method for reducing fixed pattern noise in image data
WO2022115298A1 (en) * 2020-11-30 2022-06-02 James R. Glidewell Dental Ceramics, Inc. Out of view ct scan reconstruction
US11544846B2 (en) 2020-08-27 2023-01-03 James R. Glidewell Dental Ceramics, Inc. Out-of-view CT scan detection
US11721017B2 (en) 2021-03-31 2023-08-08 James R. Glidewell Dental Ceramics, Inc. CT reconstruction quality control

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8184176B2 (en) * 2009-12-09 2012-05-22 International Business Machines Corporation Digital camera blending and clashing color warning system
US8934044B2 (en) 2012-07-20 2015-01-13 Adobe Systems Incorporated Systems and methods for live view photo layer in digital imaging applications
US10118696B1 (en) 2016-03-31 2018-11-06 Steven M. Hoffberg Steerable rotating projectile
US11712637B1 (en) 2018-03-23 2023-08-01 Steven M. Hoffberg Steerable disk or ball
CN110415237B (en) * 2019-07-31 2022-02-08 Oppo广东移动通信有限公司 Skin flaw detection method, skin flaw detection device, terminal device and readable storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5229864A (en) * 1990-04-16 1993-07-20 Fuji Photo Film Co., Ltd. Device for regenerating a picture signal by decoding
US6160923A (en) * 1997-11-05 2000-12-12 Microsoft Corporation User directed dust and compact anomaly remover from digital images
US6313840B1 (en) * 1997-04-18 2001-11-06 Adobe Systems Incorporated Smooth shading of objects on display devices
US20030012452A1 (en) * 2001-07-06 2003-01-16 Jasc Software, Inc. Assisted scratch removal
US6587592B2 (en) * 2001-11-16 2003-07-01 Adobe Systems Incorporated Generating replacement data values for an image region
US6870954B1 (en) * 2001-04-04 2005-03-22 Adobe Systems Incorporated Methods apparatus for generating shaped gradient fills
US7002594B2 (en) * 2000-03-24 2006-02-21 Sharp Kabushiki Kaisha Image processing apparatus and image display apparatus using same

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003534743A (en) * 2000-05-19 2003-11-18 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Methods, systems and devices
US7010163B1 (en) 2001-04-20 2006-03-07 Shell & Slate Software Method and apparatus for processing image data
US7277595B1 (en) 2003-01-06 2007-10-02 Apple Inc. Method and apparatus for digital image manipulation to remove image blemishes

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5229864A (en) * 1990-04-16 1993-07-20 Fuji Photo Film Co., Ltd. Device for regenerating a picture signal by decoding
US6313840B1 (en) * 1997-04-18 2001-11-06 Adobe Systems Incorporated Smooth shading of objects on display devices
US6160923A (en) * 1997-11-05 2000-12-12 Microsoft Corporation User directed dust and compact anomaly remover from digital images
US7002594B2 (en) * 2000-03-24 2006-02-21 Sharp Kabushiki Kaisha Image processing apparatus and image display apparatus using same
US6870954B1 (en) * 2001-04-04 2005-03-22 Adobe Systems Incorporated Methods apparatus for generating shaped gradient fills
US20030012452A1 (en) * 2001-07-06 2003-01-16 Jasc Software, Inc. Assisted scratch removal
US6587592B2 (en) * 2001-11-16 2003-07-01 Adobe Systems Incorporated Generating replacement data values for an image region

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
Ballester et al. ("Filling-In by Joint Interpolation of Vector Fields and Gray Levels", IEEE Transaction on Image Processing, vol. 10, No. 8, Aug. 2001, pp. 1200-1211). *
Bertalmio et al., "Image Inpainting", Proc. 27th Conf. on Computer Graphics and Interactive Techniques, Jul. 2000, pp. 417-424. *
Carrier, G. F. et al., Functions of a Complex Variable: Theory and Technique, McGraw-Hill Book Company, NY, 1966, pp. 46-47.
From the IDS: Carrier et al. (Functions of a Complex Variable: Theory and Technique, McGraw-Hill Book Co., NY, 1966, p. 47). *
K.E. Atkinson, An Introduction to Numerical Analysis, John Wiley & Sons, 1978, pp. 482-485. *
Zarantonello et al. ("Domain Decomposition, Boundary Integrals, and Wavelets", SPIE vol. 2569, 1995, pp. 866-875). *

Cited By (68)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8428127B2 (en) 2002-07-15 2013-04-23 Apple Inc. Method of performing rate control for a compression system
US8477843B2 (en) 2002-12-16 2013-07-02 Apple Inc. Method of implementing improved rate control for a multimedia compression and encoding system
US7804897B1 (en) 2002-12-16 2010-09-28 Apple Inc. Method for implementing an improved quantizer in a multimedia compression and encoding system
US20070297691A1 (en) * 2003-01-06 2007-12-27 Russell Reid Method and apparatus for digital image manipulation to remove image blemishes
US7545994B2 (en) 2003-01-06 2009-06-09 Apple Inc. Method and apparatus for digital image manipulation to remove image blemishes
US20070003157A1 (en) * 2005-06-29 2007-01-04 Xerox Corporation Artifact removal and quality assurance system and method for scanned images
US7466873B2 (en) * 2005-06-29 2008-12-16 Xerox Corporation Artifact removal and quality assurance system and method for scanned images
US7889242B2 (en) * 2006-10-26 2011-02-15 Hewlett-Packard Development Company, L.P. Blemish repair tool for digital photographs in a camera
US20080100726A1 (en) * 2006-10-26 2008-05-01 Cazier Robert P Blemish Repair Tool For Digital Photographs In A Camera
US8761542B2 (en) 2008-02-11 2014-06-24 Apple Inc. Blemish removal
US20090202170A1 (en) * 2008-02-11 2009-08-13 Ben Weiss Blemish Removal
US8385681B2 (en) 2008-02-11 2013-02-26 Apple Inc. Blemish removal
US20100279418A1 (en) * 2009-05-04 2010-11-04 Loren Robert Larson Glucose meter adaptable for use with handheld devices, and associated communication network
US20110090381A1 (en) * 2009-10-20 2011-04-21 Apple Inc. System and method for processing image data using an image processing pipeline of an image signal processor
US20110091101A1 (en) * 2009-10-20 2011-04-21 Apple Inc. System and method for applying lens shading correction during image processing
US20110090351A1 (en) * 2009-10-20 2011-04-21 Apple Inc. Temporal filtering techniques for image signal processing
US8259198B2 (en) 2009-10-20 2012-09-04 Apple Inc. System and method for detecting and correcting defective pixels in an image sensor
US8294781B2 (en) 2009-10-20 2012-10-23 Apple Inc. System and method for sharpening image data
US8330772B2 (en) 2009-10-20 2012-12-11 Apple Inc. Image signal processor front-end image data processing system and method
US8358319B2 (en) 2009-10-20 2013-01-22 Apple Inc. System and method for processing image data using an image processing pipeline of an image signal processor
US8638342B2 (en) 2009-10-20 2014-01-28 Apple Inc. System and method for demosaicing image data using weighted gradients
US20110090370A1 (en) * 2009-10-20 2011-04-21 Apple Inc. System and method for sharpening image data
US8593483B2 (en) 2009-10-20 2013-11-26 Apple Inc. Temporal filtering techniques for image signal processing
US8472712B2 (en) 2009-10-20 2013-06-25 Apple Inc. System and method for applying lens shading correction during image processing
US20110090242A1 (en) * 2009-10-20 2011-04-21 Apple Inc. System and method for demosaicing image data using weighted gradients
US20110090380A1 (en) * 2009-10-20 2011-04-21 Apple Inc. Image signal processor front-end image data processing system and method
US20110090371A1 (en) * 2009-10-20 2011-04-21 Apple Inc. System and method for detecting and correcting defective pixels in an image sensor
US8525895B2 (en) 2010-07-29 2013-09-03 Apple Inc. Binning compensation filtering techniques for image signal processing
US8493482B2 (en) 2010-08-18 2013-07-23 Apple Inc. Dual image sensor image processing system and method
US8922704B2 (en) 2010-09-01 2014-12-30 Apple Inc. Techniques for collection of auto-focus statistics
US8531542B2 (en) 2010-09-01 2013-09-10 Apple Inc. Techniques for acquiring and processing statistics data in an image signal processor
US9398205B2 (en) 2010-09-01 2016-07-19 Apple Inc. Auto-focus control using image statistics data with coarse and fine auto-focus scores
US8605167B2 (en) 2010-09-01 2013-12-10 Apple Inc. Flexible color space selection for auto-white balance processing
US8629913B2 (en) 2010-09-30 2014-01-14 Apple Inc. Overflow control techniques for image signal processing
US8488055B2 (en) 2010-09-30 2013-07-16 Apple Inc. Flash synchronization using image sensor interface timing signal
US8643770B2 (en) 2010-09-30 2014-02-04 Apple Inc. Flash synchronization using image sensor interface timing signal
US8736700B2 (en) 2010-09-30 2014-05-27 Apple Inc. Techniques for synchronizing audio and video data in an image signal processing system
US8508621B2 (en) 2010-09-30 2013-08-13 Apple Inc. Image sensor data formats and memory addressing techniques for image signal processing
US8786625B2 (en) 2010-09-30 2014-07-22 Apple Inc. System and method for processing image data using an image signal processor having back-end processing logic
US8471932B2 (en) 2010-09-30 2013-06-25 Apple Inc. Spatial filtering for image signal processing
US9344613B2 (en) 2010-09-30 2016-05-17 Apple Inc. Flash synchronization using image sensor interface timing signal
US8508612B2 (en) 2010-09-30 2013-08-13 Apple Inc. Image signal processor line buffer configuration for processing ram image data
US9142012B2 (en) 2012-05-31 2015-09-22 Apple Inc. Systems and methods for chroma noise reduction
US9342858B2 (en) 2012-05-31 2016-05-17 Apple Inc. Systems and methods for statistics collection using clipped pixel tracking
US9014504B2 (en) 2012-05-31 2015-04-21 Apple Inc. Systems and methods for highlight recovery in an image signal processor
US9025867B2 (en) 2012-05-31 2015-05-05 Apple Inc. Systems and methods for YCC image processing
US9031319B2 (en) 2012-05-31 2015-05-12 Apple Inc. Systems and methods for luma sharpening
US9077943B2 (en) 2012-05-31 2015-07-07 Apple Inc. Local image statistics collection
US9105078B2 (en) 2012-05-31 2015-08-11 Apple Inc. Systems and methods for local tone mapping
US9131196B2 (en) 2012-05-31 2015-09-08 Apple Inc. Systems and methods for defective pixel correction with neighboring pixels
US8917336B2 (en) 2012-05-31 2014-12-23 Apple Inc. Image signal processing involving geometric distortion correction
US11689826B2 (en) 2012-05-31 2023-06-27 Apple Inc. Systems and method for reducing fixed pattern noise in image data
US9317930B2 (en) 2012-05-31 2016-04-19 Apple Inc. Systems and methods for statistics collection using pixel mask
US9332239B2 (en) 2012-05-31 2016-05-03 Apple Inc. Systems and methods for RGB image processing
US8872946B2 (en) 2012-05-31 2014-10-28 Apple Inc. Systems and methods for raw image processing
US8953882B2 (en) 2012-05-31 2015-02-10 Apple Inc. Systems and methods for determining noise statistics of image data
US8817120B2 (en) 2012-05-31 2014-08-26 Apple Inc. Systems and methods for collecting fixed pattern noise statistics of image data
US9710896B2 (en) 2012-05-31 2017-07-18 Apple Inc. Systems and methods for chroma noise reduction
US9741099B2 (en) 2012-05-31 2017-08-22 Apple Inc. Systems and methods for local tone mapping
US9743057B2 (en) 2012-05-31 2017-08-22 Apple Inc. Systems and methods for lens shading correction
US11089247B2 (en) 2012-05-31 2021-08-10 Apple Inc. Systems and method for reducing fixed pattern noise in image data
CN104952049B (en) * 2015-06-23 2018-02-23 浙江工商大学 Based on the image repair method for facing domain diverging interpolation
CN104952049A (en) * 2015-06-23 2015-09-30 浙江工商大学 Image inpainting method based on domain divergence interpolation
US11544846B2 (en) 2020-08-27 2023-01-03 James R. Glidewell Dental Ceramics, Inc. Out-of-view CT scan detection
US11928818B2 (en) 2020-08-27 2024-03-12 James R. Glidewell Dental Ceramics, Inc. Out-of-view CT scan detection
WO2022115298A1 (en) * 2020-11-30 2022-06-02 James R. Glidewell Dental Ceramics, Inc. Out of view ct scan reconstruction
US11847722B2 (en) 2020-11-30 2023-12-19 James R. Glidewell Dental Ceramics, Inc. Out of view CT scan reconstruction
US11721017B2 (en) 2021-03-31 2023-08-08 James R. Glidewell Dental Ceramics, Inc. CT reconstruction quality control

Also Published As

Publication number Publication date
US7545994B2 (en) 2009-06-09
US20070297691A1 (en) 2007-12-27

Similar Documents

Publication Publication Date Title
US7277595B1 (en) Method and apparatus for digital image manipulation to remove image blemishes
US6434265B1 (en) Aligning rectilinear images in 3D through projective registration and calibration
US6359617B1 (en) Blending arbitrary overlaying images into panoramas
US7471849B2 (en) Methods and apparatus for shifting perspective in a composite image
US9998685B2 (en) Spatial and temporal alignment of video sequences
US7492375B2 (en) High dynamic range image viewing on low dynamic range displays
US7519907B2 (en) System and method for image editing using an image stack
US8194097B2 (en) Virtual masking using rigid parametric modeling
US6529206B1 (en) Image processing apparatus and method, and medium therefor
EP1372109B1 (en) Method and system for enhancing portrait images
US7528846B2 (en) Systems and methods to adjust a source image aspect ratio to match a different target display aspect ratio
US7342572B2 (en) System and method for transforming an ordinary computer monitor into a touch screen
US20050147322A1 (en) Digital composition of a mosaic image
US6556195B1 (en) Image processing device and image processing method
US20030012414A1 (en) Automatic digital image enhancement
JP3690391B2 (en) Image editing apparatus, image trimming method, and program
US7916963B2 (en) Method and apparatus for an intuitive digital image processing system that enhances digital images
US6548800B2 (en) Image blur detection methods and arrangements
US8436852B2 (en) Image editing consistent with scene geometry
US7196707B2 (en) Tangent balancing
US6801654B2 (en) Picture processing apparatus, method and recording medium for a natural expansion drawing
CA2285227A1 (en) Computer system process and user interface for providing intelligent scissors for image composition
EP2038848B1 (en) Facilitating image-editing operations across multiple perspective planes
US7129961B1 (en) System and method for dynamic autocropping of images
Nielsen et al. ClickRemoval: Interactive pinpoint image object removal

Legal Events

Date Code Title Description
AS Assignment

Owner name: APPLE COMPUTER, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:REID, RUSSELL;REEL/FRAME:015658/0497

Effective date: 20040510

STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: APPLE INC., CALIFORNIA

Free format text: CHANGE OF NAME;ASSIGNOR:APPLE COMPUTER, INC.;REEL/FRAME:021640/0347

Effective date: 20070109

AS Assignment

Owner name: APPLE INC., CALIFORNIA

Free format text: CHANGE OF NAME;ASSIGNOR:APPLE COMPUTER, INC.;REEL/FRAME:022228/0434

Effective date: 20070109

FPAY Fee payment

Year of fee payment: 4

FPAY Fee payment

Year of fee payment: 8

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20191002