US20050206751A1 - Digital video system for assembling video sequences - Google Patents
Digital video system for assembling video sequences Download PDFInfo
- Publication number
- US20050206751A1 US20050206751A1 US10/804,946 US80494604A US2005206751A1 US 20050206751 A1 US20050206751 A1 US 20050206751A1 US 80494604 A US80494604 A US 80494604A US 2005206751 A1 US2005206751 A1 US 2005206751A1
- Authority
- US
- United States
- Prior art keywords
- image
- presentation
- programmed instructions
- processing system
- operator
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000003860 storage Methods 0.000 claims abstract description 53
- 238000012545 processing Methods 0.000 claims abstract description 39
- 230000004044 response Effects 0.000 claims abstract description 21
- 238000000034 method Methods 0.000 claims description 29
- 230000003287 optical effect Effects 0.000 claims description 16
- 230000007246 mechanism Effects 0.000 claims description 8
- 238000004091 panning Methods 0.000 claims description 3
- 239000007787 solid Substances 0.000 claims 1
- 238000003384 imaging method Methods 0.000 description 35
- 230000000694 effects Effects 0.000 description 23
- 230000008901 benefit Effects 0.000 description 11
- 230000007704 transition Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 7
- 238000013515 script Methods 0.000 description 7
- 238000004519 manufacturing process Methods 0.000 description 6
- 230000009471 action Effects 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000000977 initiatory effect Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- 238000004590 computer program Methods 0.000 description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 239000002131 composite material Substances 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000002156 mixing Methods 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000002360 preparation method Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000000699 topical effect Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 241000251468 Actinopterygii Species 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000007664 blowing Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 238000007598 dipping method Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 235000013305 food Nutrition 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 230000009182 swimming Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/34—Indicating arrangements
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B27/034—Electronic editing of digitised analogue information signals, e.g. audio or video signals on discs
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/02—Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
- G11B27/031—Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B27/036—Insert-editing
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
- G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B27/102—Programmed access in sequence to addressed parts of tracks of operating record carriers
- G11B27/105—Programmed access in sequence to addressed parts of tracks of operating record carriers of operating discs
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B2220/00—Record carriers by type
- G11B2220/20—Disc-shaped record carriers
- G11B2220/25—Disc-shaped record carriers characterised in that the disc is based on a specific recording technology
- G11B2220/2537—Optical discs
- G11B2220/2545—CDs
-
- G—PHYSICS
- G11—INFORMATION STORAGE
- G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
- G11B2220/00—Record carriers by type
- G11B2220/20—Disc-shaped record carriers
- G11B2220/25—Disc-shaped record carriers characterised in that the disc is based on a specific recording technology
- G11B2220/2537—Optical discs
- G11B2220/2562—DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
Definitions
- the present invention relates to the capture and display of motion images and more particularly relates to an apparatus and method for producing more compelling video programs by a consumer.
- 6,292,219 provides a removable motion image-recording medium with programs that automatically, or as instructed by the camera operator, act upon a captured video stream to provide an edited appearance, instructing the camera to affect imaging characteristics such as focus, highlighting effects, segues, fade-outs, and other effects.
- scenes are shot according to a schedule that is best suited for production and are later assembled at an editing facility, according to a scripted storyboard. It can be appreciated that there would be advantages to providing some measure of storyboard capability to a consumer for event-based imaging with an edited appearance. The consumer would benefit from guidance in shooting appropriate scene content and would benefit from an ability to customize an image sequence, suited to the subject, audience, and event.
- the present invention provides a method for forming a presentation comprising a set of image sequences captured electronically, the method comprising:
- FIG. 1 is a schematic block diagram of an image capture and processing apparatus according to the present invention
- FIG. 2 is a perspective view of a camera used for image capture according to the method of the present invention
- FIG. 3 is a perspective view of the camera of FIG. 2 in a supporting imaging dock assembly
- FIG. 4 is a plan view showing the appearance of the display of the camera for accepting preparatory information from the operator;
- FIGS. 5 a - 5 d are plan views of the camera display showing a first example sequence of prompts for setting up a customized event-based image capture sequence
- FIGS. 6 a - 6 p are plan views showing a sequence of displays and prompts in one embodiment
- FIG. 7 is a perspective view of a camera providing a display and prompts, according to the present invention.
- FIG. 8 is a flow chart showing a first example sequence of steps for obtaining images and assembling an electronic storyboard according to the present invention
- FIG. 9 a is a block diagram showing an empty electronic storyboard, assembled according to the present invention.
- FIG. 9 b is a block diagram showing a completed electronic storyboard, populated using images obtained according to the present invention.
- FIGS. 10 a - 10 g are plan views showing images obtained and processed according to the present invention.
- FIG. 11 is a flow chart showing an example sequence of steps for capturing a music video based scripted program according to the present invention.
- FIGS. 12 a - 12 e are plan views of the camera display showing a second example sequence of prompts for capturing a music video based scripted program
- FIG. 13 is a chart showing the relative timing of various components of audio and video components that are combined using the method of the present invention.
- FIGS. 14 a - 14 c are plan views of the camera display showing an example sequence for a humorous scripted presentation according to the present invention.
- FIGS. 15 a - 15 d are plan views of the camera display showing an example sequence for an alternate humorous scripted presentation according to the present invention.
- Imaging apparatus 10 used for obtaining and processing still or motion picture images according to one embodiment of the present invention.
- Imaging apparatus 10 has two major components, a camera 12 and an imaging dock assembly 30 .
- Camera 12 consists of an image capture and recording apparatus 14 that includes imaging optics and electronic sensor circuitry, such as a conventional CCD, for example, for obtaining a two-dimensional electronic image.
- Image capture and recording apparatus 14 is supported by a RAM 16 and a display 18 .
- a control logic processor 20 provides operational logic for operation of camera 12 and cooperates with an interface circuit 22 for communication with imaging dock assembly 30 and with external input devices.
- Imaging dock assembly 30 has an input interface section 24 and an output interface section 26 .
- Input interface section 24 accepts input software and data from any of a number of storage devices or input communication channels.
- Input data and software can be provided, for example, by magnetic or optical storage devices, such as floppy diskettes, memory cards, removable magnetic hard drives or optical media such as CDs and DVDs.
- Input can also be provided over wired or wireless ports, using any of a variety of networked or standalone communication mechanisms, such as Ethernet, IR, RF, or Bluetooth devices.
- Output interface section 26 may support audio input or playback circuitry 28 , a printer 32 , and a host of storage media.
- Image and/or audio output can be provided in the form of hard-copy output or in file form, using any number of wired or wireless communication and storage mechanisms.
- imaging apparatus 10 as shown in FIG. 1 can have any of a number of alternative arrangements of components for camera 12 and its supporting imaging dock assembly 30 .
- the basic components shown in FIG. 1 are needed, in some form, for operation of the method of the present invention.
- imaging apparatus 10 is capable of accepting programmed instructions and of capturing, processing, and playing back image and audio content according to those instructions, in a manner particularly well suited for event-based or theme-based imaging.
- FIG. 2 there is shown, in perspective view, one embodiment of camera 12 with a removable storage device 34 in the form of a memory card.
- An operator interface panel 36 is provided for control of camera 12 operation.
- Display 18 may be in the form of a touchscreen, enabling entry of additional operator instructions to camera 12 .
- a stand 38 enables camera 12 to be seated on a surface for various image capture and playback functions.
- a CD/DVD port 40 is provided as an input/output mechanism for receiving pre-programmed instructions as well as for receiving pre-stored audio and/or imaging content from an optical storage disk 42 , such as a CD or DVD.
- CD/DVD port 40 has read/write capability, allowing a captured image sequence, with optional audio content, to be written to a storage device for playback or archiving.
- other ports could be used for input and/or output to memory card or other storage devices.
- imaging apparatus 10 When operating according to the present invention, imaging apparatus 10 functions as a digital director, providing operator guidance and automated image processing utilities that enable even an unskilled operator to capture and construct a presentation of image sequences that can have the appearance of a professionally prepared production. As can be readily appreciated, imaging apparatus 10 would be especially well-suited for developing an image presentation that is event-based or theme-based.
- imaging apparatus 10 In order to function as a digital director, imaging apparatus 10 must first obtain a set of programmed instructions that are suited to the particular theme or event of interest. Additionally, other images and special effects can also be provided for digital director operation. In one embodiment, programmed instructions and images are available on a DVD or similar optically encoded medium. Various versions of digital director media could be made available to consumers, differentiated by theme or event. Typical activities for which digital director media could be particularly tailored include sports, hobbies, scout troops, camping and outdoor activities, art, music, and travel, for example. Themes for digital director media could include humor, romance, family gatherings, baby showers, babies and children, and the like.
- the consumer To operate the digital director, the consumer first loads digital director software into imaging apparatus 10 .
- the operator feeds optical storage disk 42 into imaging dock assembly 30 as a preparatory step to image capture.
- Digital director software on optical storage disk 42 then prompts the consumer to make some preliminary selections that more precisely identify the event or theme for an image capture session.
- digital director prompts suited to a birthday theme may prompt the operator to identify age, sex, name, and other information about the person whose birthday is being celebrated. Prompts may appear on touchscreen display 18 of camera 12 , as is shown in the example of FIG. 4 .
- FIGS. 5 a - 5 d show an example sequence with typical screen prompts that help to customize digital director operation, as is described subsequently.
- Touchscreen buttons 44 accept operator responses that help to tailor the sequence of operations that follow.
- An on-screen keyboard 46 can be used to enable text entry. Alternatively, if the camera is enabled with voice recognition capabilities, spoken words can be converted into text. Captured audio can also be used instead of text or to augment text.
- an electronic storyboard 70 which is generated as a result of these initial preparation steps.
- electronic storyboard 70 specifies an ordered arrangement of image sequences. This can include graphics 54 and title text 68 as well as planned segments 76 , which are allotted for captured image sequences 64 when these are obtained. Transitions 66 are also provided between captured image sequences 64 . These transitions may be fades, graphics, segues, or other scene transitions, using any of the techniques described in U.S. Pat. No. 6,292,219, listed above or other familiar transitional mechanisms between captured image sequences 64 .
- Electronic storyboard 70 as shown in FIG. 9 a is highly conceptual. In practice, any number of methods could be used for arranging captured image sequences 64 beforehand. However, FIG. 9 a shows the basic principles: a plan is generated for incorporating captured image sequences with pre-stored images and special effects loaded as part of the digital director data. In a simple embodiment, electronic storyboard 70 may be provided automatically, without the option to customize beforehand. However, in the more elaborate embodiment shown with reference to FIGS. 5 a - 5 d , electronic storyboard 70 is customized, based on subject profile data entered by the operator.
- Image sequences could be either motion or still images.
- still images if produced as part of a motion picture presentation, are simply repeated in multiple frames.
- still images may be presented in a pseudo motion manner such as that popularized by director Ken Burns in presentations such as “The Civil War,” where the still image is slowly zoomed and panned to enhance its visual interest
- FIGS. 6 a through 6 o show an example sequence by which the digital director software guides the operator through individual image capture operations for obtaining image sequences that complete electronic storyboard 70 .
- display 18 provides instructions 48 and optional hints 50 to the operator. These may include requests for different types of captured images as well as tips for use of the camera and for lighting and other environmental conditions.
- digital director may also provide messages 52 that give the operator various options for the image capture sequence.
- added graphics 54 may also be suggested and provided for operator approval.
- a still image 56 may be inserted as one type of image sequence if accepted by the operator, as is shown in FIG. 6 f .
- Graphic editing options 58 may also be provided, as is shown in the example of FIG. 6 g.
- FIGS. 6 h , 6 i , and 6 j show a “video karaoke” sequence that can be inserted.
- an operator follows instructions 48 , which may provide branching options, as shown in the example of FIG. 6 h .
- a sequence prompt 60 allows the operator to indicate when to begin capture of the video karaoke sequence.
- the operator may wish to set camera 12 on stand 38 , as is shown in FIG. 7 , and join in on the fun.
- Display 18 may then show the actual scene as a song or other sequence is performed or may simply display graphics 54 , as shown in FIG. 6 j .
- an animated prompt 62 may display to guide participants through the text in an enjoyable manner.
- the output of the camera may be connected to a television so that the text is more legible.
- the captured video may or may not be displayed during this sequence.
- FIGS. 6 l , 6 m , and 6 n show various text graphics 54 added to scene content as it is captured and displayed on display 18 . This example sequence is then completed with graphics 54 added to final scene content, as is shown in FIG. 6 p .
- the text graphics 54 move and/or change appearance as it is displayed, instead of being static. For example, in FIG. 6 p , the text “The End” can slowly fade in, the candles on the cake can flicker, and the balloons can slowly spin, and then “pop” at the end of the scene.
- FIG. 9 b shows the results of image capture and processing, in concept, according to instructions as shown in the example of FIGS. 6 a - 6 p .
- electronic storyboard 70 is effectively transformed into a presentation 72 with captured image sequences 64 arranged in the order specified by electronic storyboard 70 .
- operator responses during image capture may change electronic storyboard 70 dynamically; however, electronic storyboard 70 as initially created provides at least some structure for the prompt sequence and operation of imaging apparatus 10 .
- image sequences captured by the user were not necessarily captured in the order presented on electronic storyboard 70 .
- an initiation step 100 the operator loads programmed instructions and, optionally, image content, and begins.
- an optional prompt step 110 the operator is prompted for profile information on the subject to be filmed, as was shown in the example of FIG. 5 a .
- imaging apparatus 10 prepares electronic storyboard 70 or its equivalent.
- image capture step 130 the operator is prompted to film an image sequence, which may be a still or motion image, as is described above.
- a test step 140 determines if all needed image sequences for completing electronic storyboard 70 have been obtained and causes image capture step 130 to be repeated as necessary. Once this is true, an assembly step 150 processes the obtained image sequences and arranges them in the order assigned by electronic storyboard 70 , forming presentation 72 thereby.
- transitions between image sequences can use segues supplied as part of the digital director software, as described commonly assigned U.S. Pat. No. 6,292,219 (Fredlund et al.), the disclosure of which is incorporated herein by reference. Appropriate segues can be automatically chosen by the digital director to match the theme and to match the image sequences which begin and end at this particular transition.
- a partial completion of storyboard 70 may be used to for presentation 72 , or a final editing step (not shown) can be provided for the user to approve and/or remove segments.
- a further embodiment allows the user to assign previously captured image sequences to storyboard 70 , so that the user is not limited to use of image sequences captured under the guidance of digital director to create final presentation 72 .
- imaging apparatus 10 writes presentation 72 onto a recording medium.
- this is a DVD. This may be an unused, blank DVD or, optionally, the same DVD originally used to load programmed instructions in initiation step 100 .
- a control logic processor for executing the pre-programmed instructions and for arranging the playback of the presentation from the plurality of the captured image sequences stored in said random-access electronic memory according to said operator responses may not be in the camera. This function may be executed downstream of imaging apparatus 10 in a writer or computer (not shown).
- Numerous special effects can be employed for further customization of presentation 72 . These include, for example, slow motion, accelerated motion, auto-rotoscope, programmable time-lapse, time-reversal, image mirroring, and geometric distortion. Special effects can be applied at the time of image sequence capture or during final processing in preparing presentation 72 . These effects are provided as part of the digital director software and can be automatically used for particular sequences. For example, an sequence of the “birthday boy” blowing out the candles on his birthday cake can be shown in the slow-motion “forward” (e.g. normal) direction, and then shown in the slow-motion “backward” direction, so that it appears that the candles are magically re-lighted. Similarly, by using “time-reversal,” gifts can be re-wrapped and cakes can be re-assembled from cut pieces.
- Graphic templates may be provided for one or more image sequences, for both still and motion pictures. Templates may include decorative or thematic borders and could be populated with still or motion images using “drag and drop” or other suitable user interface techniques. A template could also allow display of multiple images at one time. Templates may also be used to create multiple still image formats for generating printed output, such as photographic quality prints, album pages, calendar pages, index prints, photo books, cards, invitations and the like, or for digital slide shows. Other graphic content, such as avatars and overlays, could also be used.
- auxiliary sound clips could also be added, including sound clips provided with digital director media.
- auxiliary sound clips for example, laugh tracks, music, sound effects, and other audio content could be mixed in with, or replace, audio content obtained during filming.
- the digital director takes advantage of the capabilities afforded by digital image capture and random-access storage of image sequences and allows effective use of imaging tools disclosed in both U.S. Pat. Nos. 6,292,219 and 5,477,264, while adding prompt/response sequences and automatic assembly of image sequences that both customize and simplify the image capture process.
- Digital director capability can be effectively used to guide a relatively unskilled operator through the sequence of steps necessary to produce a pleasing and effective theme- or event-based video presentation.
- the digital director frees the operator from timing and sequencing concerns when obtaining image sequences, allowing the operator to concentrate on one sequence at a time.
- the digital director helps to provide an amateur camera user with an output that has the advantages and appearance of being professionally edited.
- the operator can immediately review the sequence that has just been captured along with any special effects, text graphics, etc. that will be used in the final presentation. If the final sequence is not visually pleasing, the operator can immediately recapture a new sequence while the subjects being photographed are still available.
- a music video is used as the script to enable the user to produce a compelling program.
- the digital video camera is provided with digital director programmed instructions in the form of an audio soundtrack (a popular song, for example) a video track (such as music video scenes, for example) and a control track that provides instructions for video frame compositing and pseudo-camera movements.
- the audio soundtrack is played back as new video sequences are captured, so that the filmed subjects can act in time to the music.
- the captured live video is then combined with the video track to produce a new video sequence that includes both live images and the music video scenes provided by the digital director.
- the live images are captured with the subjects moving in front of a stationary background, allowing an alternate background to be added once the live images have been obtained.
- image processing one or more initial images of background content only are first obtained and stored. This allows the background to be subsequently masked and replaced by a different moving background sequence provided by the video track, as described in commonly-assigned U.S. Pat. No. 5,914,748 (Parulski et al.), the disclosure of which is incorporated herein by reference.
- FIGS. 10 a - 10 g show how compositing of image frames 80 can be performed.
- a background image 82 is first obtained from the camera 12 position as shown in FIG. 10 a .
- a stored image 84 loaded for the digital director, provides the background desired for the edited music video sequence, as shown in FIG. 10 b .
- one or more subjects 86 are then captured, against the same background of FIG. 10 a .
- image capture and recording apparatus 14 FIG. 1
- replaces background image 82 with stored image 84 with masking techniques applied to subjects 86 , thereby forming a composited image 88 , as is shown in FIG.
- FIGS. 10 e , 10 f , and 10 g show a zoom sequence executed on composited image 88 by image capture and recording apparatus 14 .
- the control track provided by the digital director defines the way in which the foreground live image is composited into the background video track.
- the control track can include camera 12 control information that defines the camera 12 motion (such as zooming, panning, and rotation) of the background video track. This enables the same camera 12 motion to be digitally simulated in the live camera 12 image. For example, during a particular sequence, the background video can slowly pan to the left while zooming in. Using this technique, camera 12 itself can be stationary during the capture of the music video sequence, unchanged with respect to actual zoom position and other settings.
- the control track enables the digital video sequence captured by camera 12 to be digitally manipulated within image capture and processing apparatus 14 itself, thereby producing an apparent pan and zoom sequence. Thus, without requiring sophisticated camera 12 controls or capture techniques, the digital director enables a visually interesting and realistic sequence to be produced.
- a transfer step 200 initiated by a user selection for example, loads music audio to image capture and recording apparatus 14 .
- a selection step 202 specifies a particular tune for playback during image capture.
- the camera 12 operator is provided with the procedures for setup in a setup instruction step 204 .
- Camera 12 then captures background image 82 in a background capture step 206 .
- a content capture step 208 is executed, obtaining images of subjects 86 singing to the music.
- image capture and recording apparatus 14 performs a number of processing steps.
- Each subject 86 is extracted from the actual background image during a background extraction step 210 . Panning, zoom, and other camera 12 effects are simulated in a simulation step 212 .
- Each subject 86 is then combined with stored image 84 in a compositing step 214 . Audio is added in a mixing step 216 .
- the compositing step can include interspersing various background segments, provided as part of the digital director software, with the composite image segments. For the example shown in FIG. 10 a - 10 g , these background segments can show close-ups of the oars dipping in the water, the sun reflecting off the water, fish swimming in the water, etc.
- the final video production can then be recorded onto a suitable video storage medium, including a DVD provided with digital director software and content, for example.
- FIG. 12 a shows a selection prompt 90 for operator specification of song and overall theme, with a song selection button 92 a for song selection and a theme selection button 92 b for theme selection.
- FIG. 12 b shows a song selection menu 94 a that displays when the operator presses song selection button 92 a in FIG. 12 a .
- FIG. 12 c shows a theme selection menu 94 b that displays when the operator presses theme selection button 92 b in FIG. 12 a .
- FIG. 12 d shows an initiation prompt 96 for beginning the music video sequence with setup instruction step 204 and following steps of FIG. 11 .
- FIG. 12 e shows a lyrics prompt 98 that appears on display 18 during capture of the music video.
- Composited image 88 might be displayed dynamically, as is shown in FIG. 12 e .
- display 18 may simply show subjects 86 as they are being captured, before image frame composition or display 18 may be limited to display of lyrics alone
- the timing chart of FIG. 13 shows the temporal relationship of stored audio, shown as an audio file 170 in MP3 format, for example, with a stored background video sequence 172 and user video sequences 174 .
- the digital director Based on a digital storyboard technique, the digital director generates a composited video sequence 176 that has segments showing background image segments 82 interspersed with composited image segments 88 .
- the result is a “personalized” music video, produced by the user and featuring a selected song, with one or more subjects 86 suitably featured as the “stars.”
- the live scene is captured as the actors perform in accordance with the song, and the composited video images are stored in synchronization with the song.
- a random playback humorous scripted program or other topical sequence is produced.
- the digital director provides instructions to the user to capture images of a number of participants. The participants are asked to answer a group of questions or to perform a group of actions in response to visual or audio prompts from camera 12 or, more generally, from image capture and recording apparatus 14 . The image of the participant is then captured while answering the questions and performing the actions as instructed. Each participant response segment is then available for playback by the digital director, controlled by a script.
- FIGS. 14 a - 14 c shows a simple example of a humorous sequence.
- one or more suggestion prompts 180 may be posted to display 18 , suggesting a scripted sequence.
- the operator can accept the displayed sequence or select a different sequence.
- a series of prompts and instructions follows, as has been described with reference to the embodiments described above.
- subject 86 is posed a question or directed to execute an action.
- the response of subject 86 is recorded for playback, under control of the script that is selected.
- FIG. 14 c video playback appears on display 18 of camera 12 itself.
- the context of the original question may be changed as shown, for example, for humorous effect.
- Example questions can elicit, for example, names of favorite foods, drinks, sports teams, activities, historical facts, and the like, for example.
- Prompted actions may include facial distortions, for example.
- Captured segments can be played back with interspersed narration.
- Special effects could be used, such as replay in the form of a news program, old movie, news reel, and the like, for example.
- FIGS. 15 a - 15 d show another example using a humorous scripted sequence.
- images of various subjects 86 are captured and graphics added during image compositing, such as thought bubbles 182 shown in FIGS. 15 b , 15 c , and 15 d , for example.
- These thought bubbles can be static text, or can be transparent graphic overlays that slowly become more visible.
- the digital director software can be provided as a computer program product.
- a computer program product can include one or more storage medium, for example; magnetic storage media such as magnetic disk (such as a floppy disk) or magnetic tape; optical storage media such as optical disk, optical tape, or machine readable bar code; solid-state electronic storage devices such as random access memory (RAM), or read-only memory (ROM), or obtained via a communications link such as connection to a website via the Internet; or any other physical device or media employed to store a computer program having instructions for practicing a method according to the present invention.
- magnetic storage media such as magnetic disk (such as a floppy disk) or magnetic tape
- optical storage media such as optical disk, optical tape, or machine readable bar code
- solid-state electronic storage devices such as random access memory (RAM), or read-only memory (ROM), or obtained via a communications link such as connection to a website via the Internet; or any other physical device or media employed to store a computer program having instructions for practicing a method according to the present invention.
- imaging apparatus 10 may have any number of different arrangements of the basic components necessary for implementing digital director functions. Read and write capability could be executed using a number of different media types and wired or wireless communication mechanisms.
Abstract
Description
- The present invention relates to the capture and display of motion images and more particularly relates to an apparatus and method for producing more compelling video programs by a consumer.
- In the consumer market, steadily improving image quality, increased affordability, and ease of use have contributed to the rapid acceptance and use of video camcorders. Digital video capture and display devices have further benefited from advances in storage technologies using magnetic, optical, and electronic storage media such as digital videotape, memory cards, optical disks, and high-capacity random access memory (RAM) components. However, while camcorders have grown in popularity, many consumers express some level of disappointment with the image output achieved. For many reasons, the output from consumer camcorders can be disappointing or uninteresting to the viewer when compared with output captured and edited by imaging professionals. The problem is not typically caused by failure to use the equipment properly, but by a basic lack of a script, or a story, which is compelling to the viewer. Post-production modification and editing of video sequences is possible, but is inconvenient, and difficult to use for most consumers.
- Efforts have been made to address this problem by providers of digital image capture equipment and accessories. For example, commonly-assigned U.S. Pat. No. 5,477,264 (Sarbadhikari et al.) discloses, for an electronic camera, a removable storage device preloaded with enhancement files for effecting camera operation, with additional graphical overlays and borders, and with built-in optimization, compression, and image enhancement algorithms. Commonly-assigned U.S. Pat. No. 6,292,219 (Fredlund et al.) discloses an electronic motion picture camera system that provides special-effect video output. The device of U.S. Pat. No. 6,292,219 provides a removable motion image-recording medium with programs that automatically, or as instructed by the camera operator, act upon a captured video stream to provide an edited appearance, instructing the camera to affect imaging characteristics such as focus, highlighting effects, segues, fade-outs, and other effects.
- One technological advance of particular importance for imaging enhancement, and capitalized upon by the methods and apparatus of U.S. Pat. No. 6,292,219 relates to the use of high-capacity random access storage device, such as an optical disc. Use of a random access storage device, as contrasted with accessing a continuous video stream stored on a cassette tape, provides the ability to insert pre-stored image sequences and effects at suitable points in the captured image sequence in order to display a pleasing motion picture presentation without abrupt scene changes. In place of an optical disc, Flash EPROM memory can be used instead. An example of an image capture apparatus incorporating a high-capacity Flash EPROM is the SD Multi A/V Recorder, a camcorder from Panasonic, with U.S. headquarters in Secaucus, N.J.
- While the solutions disclosed in both the U.S. Pat. Nos. 5,477,264 and 6,292,219 provide, for example, the ability to improve the transitions between scenes, there remains significant room for improvement in making the captured scenes themselves interesting. For example, while these methods allow the use of pre-programmed transitions, borders, and special effects, the consumer must make appropriate decisions in order to use these enhancements effectively. Further, solutions offered in these patents are not intended to provide the benefits afforded by professional editing techniques, such as scripting and composition used by cinematographers, who seldom shoot a motion picture sequence in the studio according to the timeline sequence of a motion picture. In motion picture studios, as is well known, scenes are shot according to a schedule that is best suited for production and are later assembled at an editing facility, according to a scripted storyboard. It can be appreciated that there would be advantages to providing some measure of storyboard capability to a consumer for event-based imaging with an edited appearance. The consumer would benefit from guidance in shooting appropriate scene content and would benefit from an ability to customize an image sequence, suited to the subject, audience, and event.
- Certainly, a substantial amount of motion video capture by consumers is event-based. Birthdays, holidays, family gatherings, parties, graduations, wedding anniversaries, and the like are among the salient types of events that consumers wish to capture, retain, and display in a pleasing format. While the capabilities offered in U.S. Pat. Nos. 5,477,246 and 6,292,219 provide useful utilities and effects that can make electronic images from such events more pleasing, there remains a need for ways to help the consumer to capture images and image sequences that will display in a pleasing manner.
- Furthermore, there is an interest in using motion video capture devices to portray family and friends in a compelling or humorous way as part of a video program. For example, incorporating motion images of family or friends within a music video or parody show would provide an amusing program.
- Thus, there is a need to provide an electronic imaging system which includes instructions to enable a digital video system to obtain appropriate video sequences, and to automatically assemble the video sequences into a video program.
- Briefly, according to one aspect of the present invention an image processing system for obtaining a plurality of image sequences and assembling a presentation from the plurality of image sequences comprises:
-
- (a) a camera for capturing the plurality of image sequences;
- (b) a random-access electronic memory for temporary storage of the plurality of image sequences;
- (c) an input reader for accepting pre-programmed instructions from a first storage medium;
- (d) a control panel comprising:
- (i) a display for viewing the image sequences and presenting pre-programmed instructions to the operator; and
- (ii) an operator interface for accepting operator responses to the pre-programmed instructions;
- (e) a control logic processor for executing the pre-programmed instructions and for arranging the presentation from the plurality of captured image sequences stored in the random-access electronic memory according to the operator responses; and
- (f) an output writer for recording the presentation onto a second storage medium.
- According to another embodiment, the present invention provides a method for forming a presentation comprising a set of image sequences captured electronically, the method comprising:
-
- (a) obtaining programmed instructions for capturing members of the set of image sequences;
- (b) assembling an electronic storyboard, according to the programmed instructions, comprising a plan for the arrangement of the members of the set of image sequences;
- (c) prompting the camera operator to obtain each member of the set of image sequences by displaying operator instructions to the camera operator;
- (d) storing the set of image sequences in a memory;
- (e) assembling the presentation using the set of image sequences, according to the electronic storyboard; and
- (f) recording the presentation onto a storage medium.
- It is an object of the present invention to meet the needs for consumer capture of more pleasing images and image sequences electronically.
- It is an advantage of the present invention that it provides guidance to the camera operator by means of visual and/or audible instructions for capturing image content appropriate to a theme or event.
- It is a further advantage of the present invention that it frees the camera operator from editing or imaging sequence concerns, enabling the operator to have a more enjoyable experience in obtaining images electronically.
- It is a further advantage of the present invention that it enables a novice camera operator to produce images and image sequences that have a more professional, edited appearance.
- It is yet a further advantage of the present invention that it allows an operator to customize an event- or theme-based image sequence to be suited to a particular subject or audience.
- These and other objects, features, and advantages of the present invention will become apparent to those skilled in the art upon a reading of the following detailed description when taken in conjunction with the drawings wherein there is shown and described an illustrative embodiment of the invention.
- While the specification concludes with claims particularly pointing out and distinctly claiming the subject matter of the present invention, it is believed that the invention will be better understood from the following description when taken in conjunction with the accompanying drawings, wherein:
-
FIG. 1 is a schematic block diagram of an image capture and processing apparatus according to the present invention; -
FIG. 2 is a perspective view of a camera used for image capture according to the method of the present invention; -
FIG. 3 is a perspective view of the camera ofFIG. 2 in a supporting imaging dock assembly; -
FIG. 4 is a plan view showing the appearance of the display of the camera for accepting preparatory information from the operator; -
FIGS. 5 a-5 d are plan views of the camera display showing a first example sequence of prompts for setting up a customized event-based image capture sequence; -
FIGS. 6 a-6 p are plan views showing a sequence of displays and prompts in one embodiment; -
FIG. 7 is a perspective view of a camera providing a display and prompts, according to the present invention; -
FIG. 8 is a flow chart showing a first example sequence of steps for obtaining images and assembling an electronic storyboard according to the present invention; -
FIG. 9 a is a block diagram showing an empty electronic storyboard, assembled according to the present invention; -
FIG. 9 b is a block diagram showing a completed electronic storyboard, populated using images obtained according to the present invention. -
FIGS. 10 a-10 g are plan views showing images obtained and processed according to the present invention; -
FIG. 11 is a flow chart showing an example sequence of steps for capturing a music video based scripted program according to the present invention; -
FIGS. 12 a-12 e are plan views of the camera display showing a second example sequence of prompts for capturing a music video based scripted program; -
FIG. 13 is a chart showing the relative timing of various components of audio and video components that are combined using the method of the present invention; -
FIGS. 14 a-14 c are plan views of the camera display showing an example sequence for a humorous scripted presentation according to the present invention; and -
FIGS. 15 a-15 d are plan views of the camera display showing an example sequence for an alternate humorous scripted presentation according to the present invention. - The present description is directed in particular to elements forming part of, or cooperating more directly with, apparatus in accordance with the invention. It is to be understood that elements not specifically shown or described may take various forms well known to those skilled in the art.
- Referring to
FIG. 1 , there is shown a schematic block diagram of animaging apparatus 10 used for obtaining and processing still or motion picture images according to one embodiment of the present invention.Imaging apparatus 10 has two major components, acamera 12 and animaging dock assembly 30.Camera 12 consists of an image capture andrecording apparatus 14 that includes imaging optics and electronic sensor circuitry, such as a conventional CCD, for example, for obtaining a two-dimensional electronic image. Image capture andrecording apparatus 14 is supported by aRAM 16 and adisplay 18. Acontrol logic processor 20 provides operational logic for operation ofcamera 12 and cooperates with aninterface circuit 22 for communication withimaging dock assembly 30 and with external input devices. -
Imaging dock assembly 30 has aninput interface section 24 and anoutput interface section 26.Input interface section 24 accepts input software and data from any of a number of storage devices or input communication channels. Input data and software can be provided, for example, by magnetic or optical storage devices, such as floppy diskettes, memory cards, removable magnetic hard drives or optical media such as CDs and DVDs. Input can also be provided over wired or wireless ports, using any of a variety of networked or standalone communication mechanisms, such as Ethernet, IR, RF, or Bluetooth devices.Output interface section 26 may support audio input orplayback circuitry 28, aprinter 32, and a host of storage media. Image and/or audio output can be provided in the form of hard-copy output or in file form, using any number of wired or wireless communication and storage mechanisms. - It must be emphasized that
imaging apparatus 10 as shown inFIG. 1 can have any of a number of alternative arrangements of components forcamera 12 and its supportingimaging dock assembly 30. The basic components shown inFIG. 1 are needed, in some form, for operation of the method of the present invention. Using the overall arrangement shown inFIG. 1 ,imaging apparatus 10 is capable of accepting programmed instructions and of capturing, processing, and playing back image and audio content according to those instructions, in a manner particularly well suited for event-based or theme-based imaging. - Referring to
FIG. 2 , there is shown, in perspective view, one embodiment ofcamera 12 with aremovable storage device 34 in the form of a memory card. Anoperator interface panel 36 is provided for control ofcamera 12 operation.Display 18 may be in the form of a touchscreen, enabling entry of additional operator instructions tocamera 12. Astand 38 enablescamera 12 to be seated on a surface for various image capture and playback functions. - Referring to
FIG. 3 ,camera 12 is shown in position withimaging dock assembly 30. A CD/DVD port 40 is provided as an input/output mechanism for receiving pre-programmed instructions as well as for receiving pre-stored audio and/or imaging content from anoptical storage disk 42, such as a CD or DVD. In one embodiment, CD/DVD port 40 has read/write capability, allowing a captured image sequence, with optional audio content, to be written to a storage device for playback or archiving. Alternatively, other ports (not shown inFIG. 3 ) could be used for input and/or output to memory card or other storage devices. - When operating according to the present invention,
imaging apparatus 10 functions as a digital director, providing operator guidance and automated image processing utilities that enable even an unskilled operator to capture and construct a presentation of image sequences that can have the appearance of a professionally prepared production. As can be readily appreciated,imaging apparatus 10 would be especially well-suited for developing an image presentation that is event-based or theme-based. - Obtaining Programmed Instructions and Images
- In order to function as a digital director,
imaging apparatus 10 must first obtain a set of programmed instructions that are suited to the particular theme or event of interest. Additionally, other images and special effects can also be provided for digital director operation. In one embodiment, programmed instructions and images are available on a DVD or similar optically encoded medium. Various versions of digital director media could be made available to consumers, differentiated by theme or event. Typical activities for which digital director media could be particularly tailored include sports, hobbies, scout troops, camping and outdoor activities, art, music, and travel, for example. Themes for digital director media could include humor, romance, family gatherings, baby showers, babies and children, and the like. - Other types of media could be used for providing or supporting digital director operation. Alternatives include magnetic media of various types, such as diskettes. Flash memory cards or other types of memory cards could also be employed as digital director media. Still other alternatives include communication of digital director instructions and images over wired or wireless connections, including the internet.
- In effect, loading or downloading digital director instructions and images gives a “personality” to
imaging apparatus 10 operation and editing, customizing how image and audio content is assembled to produce a presentation that is particularly suited to an event or theme. The example that follows shows one basic event of interest, a birthday party. Using the digital director, with the procedures described subsequently, an operator can film and produce a video presentation that is appropriately adapted for the age, sex, and interests of the subject. - Operating Sequence—Preparation to Customize the Digital Director
- To operate the digital director, the consumer first loads digital director software into
imaging apparatus 10. In the embodiment ofFIG. 3 , electronic storyboard software and supporting images and effects load from a DVD that acts asoptical storage disk 42. As is shown inFIG. 3 , the operator feedsoptical storage disk 42 intoimaging dock assembly 30 as a preparatory step to image capture. Digital director software onoptical storage disk 42 then prompts the consumer to make some preliminary selections that more precisely identify the event or theme for an image capture session. By way of example, and not of limitation, digital director prompts suited to a birthday theme may prompt the operator to identify age, sex, name, and other information about the person whose birthday is being celebrated. Prompts may appear ontouchscreen display 18 ofcamera 12, as is shown in the example ofFIG. 4 . -
FIGS. 5 a-5 d show an example sequence with typical screen prompts that help to customize digital director operation, as is described subsequently.Touchscreen buttons 44 accept operator responses that help to tailor the sequence of operations that follow. An on-screen keyboard 46, as shown inFIG. 5 d, can be used to enable text entry. Alternatively, if the camera is enabled with voice recognition capabilities, spoken words can be converted into text. Captured audio can also be used instead of text or to augment text. - Referring to
FIG. 9 a, there is shown, in conceptual form, anelectronic storyboard 70, which is generated as a result of these initial preparation steps. Like its well-known equivalent in movie production,electronic storyboard 70 specifies an ordered arrangement of image sequences. This can includegraphics 54 andtitle text 68 as well as plannedsegments 76, which are allotted for capturedimage sequences 64 when these are obtained.Transitions 66 are also provided between capturedimage sequences 64. These transitions may be fades, graphics, segues, or other scene transitions, using any of the techniques described in U.S. Pat. No. 6,292,219, listed above or other familiar transitional mechanisms between capturedimage sequences 64. -
Electronic storyboard 70 as shown inFIG. 9 a is highly conceptual. In practice, any number of methods could be used for arranging capturedimage sequences 64 beforehand. However,FIG. 9 a shows the basic principles: a plan is generated for incorporating captured image sequences with pre-stored images and special effects loaded as part of the digital director data. In a simple embodiment,electronic storyboard 70 may be provided automatically, without the option to customize beforehand. However, in the more elaborate embodiment shown with reference toFIGS. 5 a-5 d,electronic storyboard 70 is customized, based on subject profile data entered by the operator. - Image sequences, as the term is used in the present application, could be either motion or still images. As is well known, still images, if produced as part of a motion picture presentation, are simply repeated in multiple frames. Additionally, still images may be presented in a pseudo motion manner such as that popularized by director Ken Burns in presentations such as “The Civil War,” where the still image is slowly zoomed and panned to enhance its visual interest
- Operating Sequence—Operator Instructions from the Digital Director
-
FIGS. 6 a through 6 o show an example sequence by which the digital director software guides the operator through individual image capture operations for obtaining image sequences that completeelectronic storyboard 70. - Referring to
FIG. 6 a,display 18 providesinstructions 48 andoptional hints 50 to the operator. These may include requests for different types of captured images as well as tips for use of the camera and for lighting and other environmental conditions. As shown inFIG. 6 b, digital director may also providemessages 52 that give the operator various options for the image capture sequence. As shown inFIG. 6 c, addedgraphics 54 may also be suggested and provided for operator approval. A stillimage 56 may be inserted as one type of image sequence if accepted by the operator, as is shown inFIG. 6 f.Graphic editing options 58 may also be provided, as is shown in the example ofFIG. 6 g. - The display examples shown in
FIGS. 6 h, 6 i, and 6 j show a “video karaoke” sequence that can be inserted. Using this feature, an operator followsinstructions 48, which may provide branching options, as shown in the example ofFIG. 6 h. As shown inFIG. 6 i, asequence prompt 60 allows the operator to indicate when to begin capture of the video karaoke sequence. At this point, for example, the operator may wish to setcamera 12 onstand 38, as is shown inFIG. 7 , and join in on the fun.Display 18 may then show the actual scene as a song or other sequence is performed or may simply displaygraphics 54, as shown inFIG. 6 j. During the karaoke sequence, ananimated prompt 62 may display to guide participants through the text in an enjoyable manner. The output of the camera may be connected to a television so that the text is more legible. The captured video may or may not be displayed during this sequence. -
FIGS. 6 l, 6 m, and 6 n showvarious text graphics 54 added to scene content as it is captured and displayed ondisplay 18. This example sequence is then completed withgraphics 54 added to final scene content, as is shown inFIG. 6 p. To enhance the visual interest, it is preferred that thetext graphics 54 move and/or change appearance as it is displayed, instead of being static. For example, inFIG. 6 p, the text “The End” can slowly fade in, the candles on the cake can flicker, and the balloons can slowly spin, and then “pop” at the end of the scene. - Completing
Electronic Storyboard 70 -
FIG. 9 b shows the results of image capture and processing, in concept, according to instructions as shown in the example ofFIGS. 6 a-6 p. With reference back toFIG. 9 a,electronic storyboard 70 is effectively transformed into apresentation 72 with capturedimage sequences 64 arranged in the order specified byelectronic storyboard 70. Of course, operator responses during image capture may changeelectronic storyboard 70 dynamically; however,electronic storyboard 70 as initially created provides at least some structure for the prompt sequence and operation ofimaging apparatus 10. Note also that image sequences captured by the user were not necessarily captured in the order presented onelectronic storyboard 70. - Referring to
FIG. 8 , a summary of steps carried out by control logic processing circuitry inimaging apparatus 10 for completingelectronic storyboard 70 is shown. In aninitiation step 100, the operator loads programmed instructions and, optionally, image content, and begins. In an optionalprompt step 110, the operator is prompted for profile information on the subject to be filmed, as was shown in the example ofFIG. 5 a. In an assemblestoryboard step 120,imaging apparatus 10 prepareselectronic storyboard 70 or its equivalent. In animage capture step 130, the operator is prompted to film an image sequence, which may be a still or motion image, as is described above. Atest step 140 determines if all needed image sequences for completingelectronic storyboard 70 have been obtained and causesimage capture step 130 to be repeated as necessary. Once this is true, anassembly step 150 processes the obtained image sequences and arranges them in the order assigned byelectronic storyboard 70, formingpresentation 72 thereby. - During the
assembly step 150, transitions between image sequences can use segues supplied as part of the digital director software, as described commonly assigned U.S. Pat. No. 6,292,219 (Fredlund et al.), the disclosure of which is incorporated herein by reference. Appropriate segues can be automatically chosen by the digital director to match the theme and to match the image sequences which begin and end at this particular transition. - Alternatively, a partial completion of
storyboard 70 may be used to forpresentation 72, or a final editing step (not shown) can be provided for the user to approve and/or remove segments. A further embodiment allows the user to assign previously captured image sequences tostoryboard 70, so that the user is not limited to use of image sequences captured under the guidance of digital director to createfinal presentation 72. Finally, in arecording step 160,imaging apparatus 10 writespresentation 72 onto a recording medium. In one embodiment, this is a DVD. This may be an unused, blank DVD or, optionally, the same DVD originally used to load programmed instructions ininitiation step 100. - Note that in an alternative embodiment, a control logic processor for executing the pre-programmed instructions and for arranging the playback of the presentation from the plurality of the captured image sequences stored in said random-access electronic memory according to said operator responses may not be in the camera. This function may be executed downstream of
imaging apparatus 10 in a writer or computer (not shown). - Special Effects
- Numerous special effects can be employed for further customization of
presentation 72. These include, for example, slow motion, accelerated motion, auto-rotoscope, programmable time-lapse, time-reversal, image mirroring, and geometric distortion. Special effects can be applied at the time of image sequence capture or during final processing in preparingpresentation 72. These effects are provided as part of the digital director software and can be automatically used for particular sequences. For example, an sequence of the “birthday boy” blowing out the candles on his birthday cake can be shown in the slow-motion “forward” (e.g. normal) direction, and then shown in the slow-motion “backward” direction, so that it appears that the candles are magically re-lighted. Similarly, by using “time-reversal,” gifts can be re-wrapped and cakes can be re-assembled from cut pieces. - Graphic templates may be provided for one or more image sequences, for both still and motion pictures. Templates may include decorative or thematic borders and could be populated with still or motion images using “drag and drop” or other suitable user interface techniques. A template could also allow display of multiple images at one time. Templates may also be used to create multiple still image formats for generating printed output, such as photographic quality prints, album pages, calendar pages, index prints, photo books, cards, invitations and the like, or for digital slide shows. Other graphic content, such as avatars and overlays, could also be used.
- Auxiliary sound clips could also be added, including sound clips provided with digital director media. Using auxiliary sound clips, for example, laugh tracks, music, sound effects, and other audio content could be mixed in with, or replace, audio content obtained during filming.
- As can be seen from the above description, the digital director takes advantage of the capabilities afforded by digital image capture and random-access storage of image sequences and allows effective use of imaging tools disclosed in both U.S. Pat. Nos. 6,292,219 and 5,477,264, while adding prompt/response sequences and automatic assembly of image sequences that both customize and simplify the image capture process. Digital director capability can be effectively used to guide a relatively unskilled operator through the sequence of steps necessary to produce a pleasing and effective theme- or event-based video presentation. The digital director frees the operator from timing and sequencing concerns when obtaining image sequences, allowing the operator to concentrate on one sequence at a time. By arranging the image content according to an electronic storyboard and adding transitional and special effects graphics element, the digital director helps to provide an amateur camera user with an output that has the advantages and appearance of being professionally edited. In some embodiments, the operator can immediately review the sequence that has just been captured along with any special effects, text graphics, etc. that will be used in the final presentation. If the final sequence is not visually pleasing, the operator can immediately recapture a new sequence while the subjects being photographed are still available.
- In a second embodiment, a music video is used as the script to enable the user to produce a compelling program. The digital video camera is provided with digital director programmed instructions in the form of an audio soundtrack (a popular song, for example) a video track (such as music video scenes, for example) and a control track that provides instructions for video frame compositing and pseudo-camera movements. The audio soundtrack is played back as new video sequences are captured, so that the filmed subjects can act in time to the music. The captured live video is then combined with the video track to produce a new video sequence that includes both live images and the music video scenes provided by the digital director.
- In a preferred embodiment, the live images are captured with the subjects moving in front of a stationary background, allowing an alternate background to be added once the live images have been obtained. For this type of image processing, one or more initial images of background content only are first obtained and stored. This allows the background to be subsequently masked and replaced by a different moving background sequence provided by the video track, as described in commonly-assigned U.S. Pat. No. 5,914,748 (Parulski et al.), the disclosure of which is incorporated herein by reference.
- The basic example series of image frames shown in
FIGS. 10 a-10 g show how compositing of image frames 80 can be performed. Abackground image 82 is first obtained from thecamera 12 position as shown inFIG. 10 a. A storedimage 84, loaded for the digital director, provides the background desired for the edited music video sequence, as shown inFIG. 10 b. As is shown inFIG. 10 c, one ormore subjects 86 are then captured, against the same background ofFIG. 10 a. Following capture ofsubjects 86, image capture and recording apparatus 14 (FIG. 1 ) then replacesbackground image 82 with storedimage 84, with masking techniques applied tosubjects 86, thereby forming a compositedimage 88, as is shown inFIG. 10 d. Alternatively, a user captured image sequence can be used to replacebackground image 82. However, the finalcomposite image 88 may not be as convincingFIGS. 10 e, 10 f, and 10 g show a zoom sequence executed on compositedimage 88 by image capture andrecording apparatus 14. - In a preferred embodiment, the control track provided by the digital director defines the way in which the foreground live image is composited into the background video track. The control track can include
camera 12 control information that defines thecamera 12 motion (such as zooming, panning, and rotation) of the background video track. This enables thesame camera 12 motion to be digitally simulated in thelive camera 12 image. For example, during a particular sequence, the background video can slowly pan to the left while zooming in. Using this technique,camera 12 itself can be stationary during the capture of the music video sequence, unchanged with respect to actual zoom position and other settings. The control track enables the digital video sequence captured bycamera 12 to be digitally manipulated within image capture andprocessing apparatus 14 itself, thereby producing an apparent pan and zoom sequence. Thus, without requiringsophisticated camera 12 controls or capture techniques, the digital director enables a visually interesting and realistic sequence to be produced. - Referring to the flow chart of
FIG. 11 and to the series of display prompts inFIGS. 12 a-12 e, there is shown an example sequence of steps for capturing a music video based scripted program according to this embodiment of the present invention. A transfer step 200, initiated by a user selection for example, loads music audio to image capture andrecording apparatus 14. A selection step 202, described in more detail subsequently, specifies a particular tune for playback during image capture. Thecamera 12 operator is provided with the procedures for setup in a setup instruction step 204.Camera 12 then capturesbackground image 82 in abackground capture step 206. Withsubjects 86 in place, a content capture step 208 is executed, obtaining images ofsubjects 86 singing to the music. Following capture of this image content, image capture andrecording apparatus 14 performs a number of processing steps. Each subject 86 is extracted from the actual background image during abackground extraction step 210. Panning, zoom, andother camera 12 effects are simulated in a simulation step 212. Each subject 86 is then combined with storedimage 84 in acompositing step 214. Audio is added in a mixingstep 216. The compositing step can include interspersing various background segments, provided as part of the digital director software, with the composite image segments. For the example shown inFIG. 10 a-10 g, these background segments can show close-ups of the oars dipping in the water, the sun reflecting off the water, fish swimming in the water, etc. The final video production can then be recorded onto a suitable video storage medium, including a DVD provided with digital director software and content, for example. -
Display 18 ofcamera 12 provides prompts and allows operator selections to support the image capture portions of the sequence described with reference toFIG. 11 .FIG. 12 a shows aselection prompt 90 for operator specification of song and overall theme, with asong selection button 92 a for song selection and atheme selection button 92 b for theme selection.FIG. 12 b shows asong selection menu 94 a that displays when the operator pressessong selection button 92 a inFIG. 12 a.FIG. 12 c shows atheme selection menu 94 b that displays when the operator pressestheme selection button 92 b inFIG. 12 a.FIG. 12 d shows an initiation prompt 96 for beginning the music video sequence with setup instruction step 204 and following steps ofFIG. 11 .FIG. 12 e shows a lyrics prompt 98 that appears ondisplay 18 during capture of the music video.Composited image 88 might be displayed dynamically, as is shown inFIG. 12 e. Alternately,display 18 may simply showsubjects 86 as they are being captured, before image frame composition ordisplay 18 may be limited to display of lyrics alone - The timing chart of
FIG. 13 shows the temporal relationship of stored audio, shown as anaudio file 170 in MP3 format, for example, with a storedbackground video sequence 172 anduser video sequences 174. Based on a digital storyboard technique, the digital director generates a compositedvideo sequence 176 that has segments showingbackground image segments 82 interspersed with compositedimage segments 88. - The result is a “personalized” music video, produced by the user and featuring a selected song, with one or
more subjects 86 suitably featured as the “stars.” The live scene is captured as the actors perform in accordance with the song, and the composited video images are stored in synchronization with the song. - In a third embodiment, a random playback humorous scripted program or other topical sequence is produced. In this embodiment, the digital director provides instructions to the user to capture images of a number of participants. The participants are asked to answer a group of questions or to perform a group of actions in response to visual or audio prompts from
camera 12 or, more generally, from image capture andrecording apparatus 14. The image of the participant is then captured while answering the questions and performing the actions as instructed. Each participant response segment is then available for playback by the digital director, controlled by a script. - The display sequence of
FIGS. 14 a-14 c shows a simple example of a humorous sequence. As shown inFIG. 14 a, one or more suggestion prompts 180 may be posted to display 18, suggesting a scripted sequence. The operator can accept the displayed sequence or select a different sequence. A series of prompts and instructions follows, as has been described with reference to the embodiments described above. InFIG. 14 b, subject 86 is posed a question or directed to execute an action. The response of subject 86 is recorded for playback, under control of the script that is selected. For example, inFIG. 14 c, video playback appears ondisplay 18 ofcamera 12 itself. The context of the original question may be changed as shown, for example, for humorous effect. Example questions can elicit, for example, names of favorite foods, drinks, sports teams, activities, historical facts, and the like, for example. Prompted actions may include facial distortions, for example. Captured segments can be played back with interspersed narration. Special effects could be used, such as replay in the form of a news program, old movie, news reel, and the like, for example. -
FIGS. 15 a-15 d show another example using a humorous scripted sequence. After an affirmative response to suggestion prompts 180 inFIG. 15 a, images ofvarious subjects 86 are captured and graphics added during image compositing, such as thought bubbles 182 shown inFIGS. 15 b, 15 c, and 15 d, for example. These thought bubbles can be static text, or can be transparent graphic overlays that slowly become more visible. - Numerous other approaches could be used for execution of a script using the digital director. Editing, randomization, repetition, action reversal or “rewinding,” and similar effects can be utilized to generate a suitable, compelling story line. More complex scripts could be employed, involving
several subjects 86 in different scenes of a story, such as a fairy tale, for example. The digital director utility could be used, for example, to generate a scripted tribute or greeting card, for example. Video sequences obtained using the digital director could be stored on various types of storage media, printed, and transferred to other display apparatus, for example. - The digital director software can be provided as a computer program product. A computer program product can include one or more storage medium, for example; magnetic storage media such as magnetic disk (such as a floppy disk) or magnetic tape; optical storage media such as optical disk, optical tape, or machine readable bar code; solid-state electronic storage devices such as random access memory (RAM), or read-only memory (ROM), or obtained via a communications link such as connection to a website via the Internet; or any other physical device or media employed to store a computer program having instructions for practicing a method according to the present invention.
- The invention has been described in detail with particular reference to certain preferred embodiments thereof, but it will be understood that variations and modifications can be effected within the scope of the invention as described above, and as noted in the appended claims, by a person of ordinary skill in the art without departing from the scope of the invention. For example,
imaging apparatus 10 may have any number of different arrangements of the basic components necessary for implementing digital director functions. Read and write capability could be executed using a number of different media types and wired or wireless communication mechanisms. - Thus, what is provided is an apparatus and method for enhancing image output from an electronic imaging device, using real-time instructions and scripted special effects.
-
- 10 imaging apparatus
- 12 camera
- 14 image capture and recording apparatus
- 16 RAM
- 18 display
- 20 control logic processor
- 22 interface circuit
- 24 input interface section
- 26 output interface section
- 28 audio input or playback circuitry
- 30 imaging dock assembly
- 32 printer
- 34 removable storage device
- 36 operator interface panel
- 38 stand
- 40 CD/DVD port
- 42 optical storage disk
- 44 touchscreen buttons
- 46 keyboard
- 48 instructions
- 50 hints
- 52 messages
- 54 graphics
- 56 still image
- 58 graphic editing options
- 60 sequence prompt
- 62 animated prompt
- 64 captured image sequence
- 66 transition
- 68 title text
- 70 electronic storyboard
- 72 presentation
- 76 planned segment
- 80 image frame
- 82 background image
- 84 stored image
- 86 subject
- 88 composited image
- 90 selection prompt
- 92 a song selection button
- 92 b theme selection button
- 94 a song selection menu
- 94 b theme selection menu
- 96 initiation prompt
- 98 lyrics prompt
- 100 initiation step
- 110 prompt step
- 120 assemble storyboard step
- 130 image capture step
- 140 test step
- 150 assembly step
- 160 recording step
- 170 audio file
- 172 stored background video sequence
- 174 user video sequence
- 176 composited video sequence
- 180 suggestion prompt
- 182 thought bubble
- 200 transfer step
- 202 selection step
- 204 setup instruction step
- 206 background capture step
- 208 content capture step
- 210 background extraction step
- 212 simulation step
- 214 compositing step
- 216 mixing step
Claims (40)
Priority Applications (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/804,946 US20050206751A1 (en) | 2004-03-19 | 2004-03-19 | Digital video system for assembling video sequences |
EP05724419A EP1726158A2 (en) | 2004-03-19 | 2005-03-04 | Digital video system for assembling video sequences |
PCT/US2005/006874 WO2005096621A2 (en) | 2004-03-19 | 2005-03-04 | Digital video system for assembling video sequences |
JP2007503932A JP2007529951A (en) | 2004-03-19 | 2005-03-04 | Digital video system for video sequence synthesis |
US12/908,926 US8139126B2 (en) | 2004-03-19 | 2010-10-21 | Digital video system for assembling video sequences |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/804,946 US20050206751A1 (en) | 2004-03-19 | 2004-03-19 | Digital video system for assembling video sequences |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/908,926 Continuation US8139126B2 (en) | 2004-03-19 | 2010-10-21 | Digital video system for assembling video sequences |
Publications (1)
Publication Number | Publication Date |
---|---|
US20050206751A1 true US20050206751A1 (en) | 2005-09-22 |
Family
ID=34961790
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/804,946 Abandoned US20050206751A1 (en) | 2004-03-19 | 2004-03-19 | Digital video system for assembling video sequences |
US12/908,926 Expired - Fee Related US8139126B2 (en) | 2004-03-19 | 2010-10-21 | Digital video system for assembling video sequences |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/908,926 Expired - Fee Related US8139126B2 (en) | 2004-03-19 | 2010-10-21 | Digital video system for assembling video sequences |
Country Status (4)
Country | Link |
---|---|
US (2) | US20050206751A1 (en) |
EP (1) | EP1726158A2 (en) |
JP (1) | JP2007529951A (en) |
WO (1) | WO2005096621A2 (en) |
Cited By (36)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030184674A1 (en) * | 2002-03-26 | 2003-10-02 | Eastman Kodak Company | System for capturing and archiving motion video segments |
US20060268121A1 (en) * | 2005-02-20 | 2006-11-30 | Nucore Technology Inc. | In-camera cinema director |
US20060285034A1 (en) * | 2005-06-15 | 2006-12-21 | Canon Kabushiki Kaisha | Image Display Method and Image Display Apparatus |
US20070162855A1 (en) * | 2006-01-06 | 2007-07-12 | Kelly Hawk | Movie authoring |
WO2007081599A2 (en) * | 2005-10-22 | 2007-07-19 | Gain Enterprises, Llc | Accelerated visual text to screen translation method |
US20070182861A1 (en) * | 2006-02-03 | 2007-08-09 | Jiebo Luo | Analyzing camera captured video for key frames |
US20070183497A1 (en) * | 2006-02-03 | 2007-08-09 | Jiebo Luo | Extracting key frame candidates from video clip |
US20070202480A1 (en) * | 2006-02-28 | 2007-08-30 | Steele Brenda S | Digital karaoke jukebox |
US20070226254A1 (en) * | 2006-03-21 | 2007-09-27 | Microsoft Corporation | Recorder User Interface |
US20080077844A1 (en) * | 2006-09-26 | 2008-03-27 | Samsung Electronics Co., Ltd. | Apparatus and method for managing multimedia content in mobile terminal |
US20080276792A1 (en) * | 2007-05-07 | 2008-11-13 | Bennetts Christopher L | Lyrics superimposed on video feed |
US20080304808A1 (en) * | 2007-06-05 | 2008-12-11 | Newell Catherine D | Automatic story creation using semantic classifiers for digital assets and associated metadata |
US20090003794A1 (en) * | 2007-06-28 | 2009-01-01 | Disney Enterprises, Inc. | Method and system for facilitating creation of content |
US20090062944A1 (en) * | 2007-09-04 | 2009-03-05 | Apple Inc. | Modifying media files |
US20100020181A1 (en) * | 2006-12-26 | 2010-01-28 | Kabushiki Kaisha Toshiba | Information processing apparatus and information processing method |
US20100220197A1 (en) * | 2009-03-02 | 2010-09-02 | John Nicholas Dukellis | Assisted Video Creation Utilizing a Camera |
US20110035700A1 (en) * | 2009-08-05 | 2011-02-10 | Brian Meaney | Multi-Operation User Interface Tool |
WO2011049799A1 (en) * | 2009-10-20 | 2011-04-28 | Qwiki, Inc. | Method and system for assembling animated media based on keyword and string input |
US20110234613A1 (en) * | 2010-03-25 | 2011-09-29 | Apple Inc. | Generating digital media presentation layouts dynamically based on image features |
US20110235858A1 (en) * | 2010-03-25 | 2011-09-29 | Apple Inc. | Grouping Digital Media Items Based on Shared Features |
US20120086834A1 (en) * | 2010-10-12 | 2012-04-12 | Sony Ericsson Mobile Communications Japan, Inc. | Terminal apparatus and method for adding an effect to an image |
US20130272673A1 (en) * | 2012-03-13 | 2013-10-17 | Lee Eugene Swearingen | System and method for guided video creation |
CN103384896A (en) * | 2011-10-21 | 2013-11-06 | 富士胶片株式会社 | Digital comic editing device and method therefor |
US8584015B2 (en) | 2010-10-19 | 2013-11-12 | Apple Inc. | Presenting media content items using geographical data |
WO2012037419A3 (en) * | 2010-09-16 | 2014-03-20 | Omnyx, LLC | Digital pathology image manipulation |
US20140369667A1 (en) * | 2007-01-09 | 2014-12-18 | Sony Corporation | Information processing apparatus, information processing method, and program |
US20150155008A1 (en) * | 2013-12-02 | 2015-06-04 | Magix Ag | System and method for theme based video creation with real-time effects |
US9142253B2 (en) | 2006-12-22 | 2015-09-22 | Apple Inc. | Associating keywords to media |
US20160042764A1 (en) * | 2005-11-07 | 2016-02-11 | Koninklijke Philips N.V. | Method and apparatus for editing a program |
US20160088279A1 (en) * | 2014-09-19 | 2016-03-24 | Foundation Partners Group, Llc | Multi-sensory environment room |
US20180241937A1 (en) * | 2017-02-17 | 2018-08-23 | Microsoft Technology Licensing, Llc | Directed content capture and content analysis |
US20190104249A1 (en) * | 2017-09-29 | 2019-04-04 | Dwango Co., Ltd. | Server apparatus, distribution system, distribution method, and program |
CN110866963A (en) * | 2018-08-28 | 2020-03-06 | 日本聚逸株式会社 | Moving image distribution system, moving image distribution method, and recording medium |
CN112506412A (en) * | 2020-12-07 | 2021-03-16 | 北京达佳互联信息技术有限公司 | Video editing method and device and electronic equipment |
CN113840088A (en) * | 2021-09-23 | 2021-12-24 | 展讯通信(天津)有限公司 | Image recording method and device, electronic equipment and computer readable storage medium |
US20220350840A1 (en) * | 2020-11-18 | 2022-11-03 | Beijing Zitiao Network Technology Co., Ltd. | Method and apparatus for lyric video display, electronic device, and computer-readable medium |
Families Citing this family (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070008321A1 (en) | 2005-07-11 | 2007-01-11 | Eastman Kodak Company | Identifying collection images with special events |
JP5023932B2 (en) * | 2007-09-28 | 2012-09-12 | カシオ計算機株式会社 | Imaging apparatus, image capturing method by scenario, and program |
KR101382501B1 (en) | 2007-12-04 | 2014-04-10 | 삼성전자주식회사 | Apparatus for photographing moving image and method thereof |
EP2091046A1 (en) * | 2008-02-15 | 2009-08-19 | Thomson Licensing | Presentation system and method for controlling the same |
KR101443637B1 (en) | 2008-05-20 | 2014-09-23 | 엘지전자 주식회사 | Mobile terminal and method of generating contents therein |
JP5233491B2 (en) * | 2008-08-06 | 2013-07-10 | 株式会社ニコン | Electronic camera, data distribution method and server |
EP2395770A3 (en) * | 2008-09-30 | 2013-09-25 | Panasonic Corporation | Recording medium, playback device, integrated circuit, playback method |
US20100238323A1 (en) * | 2009-03-23 | 2010-09-23 | Sony Ericsson Mobile Communications Ab | Voice-controlled image editing |
KR101764372B1 (en) * | 2011-04-19 | 2017-08-03 | 삼성전자주식회사 | Apparatus and method for compositing image in a portable terminal |
US9524651B2 (en) * | 2011-07-25 | 2016-12-20 | Raymond Fix | System and method for electronic communication using a voiceover in combination with user interaction events on a selected background |
JP2013097773A (en) * | 2011-11-07 | 2013-05-20 | Sony Corp | Information processing apparatus, information processing method, and program |
JP5889028B2 (en) * | 2012-02-13 | 2016-03-22 | キヤノン株式会社 | Moving picture recording apparatus, control method therefor, computer program, and storage medium |
US9177394B2 (en) * | 2012-03-23 | 2015-11-03 | Konica Minolta Laboratory U.S.A., Inc. | Image processing device |
KR102042265B1 (en) * | 2012-03-30 | 2019-11-08 | 엘지전자 주식회사 | Mobile terminal |
US10645345B2 (en) | 2012-07-03 | 2020-05-05 | Verint Americas Inc. | System and method of video capture and search optimization |
US9020325B2 (en) * | 2012-11-14 | 2015-04-28 | Storyvine, LLC | Storyboard-directed video production from shared and individualized assets |
US9501507B1 (en) | 2012-12-27 | 2016-11-22 | Palantir Technologies Inc. | Geo-temporal indexing and searching |
US10432877B2 (en) * | 2014-06-30 | 2019-10-01 | Nec Corporation | Image processing system, image processing method and program storage medium for protecting privacy |
KR101490507B1 (en) | 2014-07-08 | 2015-02-10 | 주식회사 테라클 | Method and apparatus for editing moving picture contents |
KR101490506B1 (en) | 2014-07-08 | 2015-02-10 | 주식회사 테라클 | Method and apparatus for editing moving picture contents |
US10579239B1 (en) | 2017-03-23 | 2020-03-03 | Palantir Technologies Inc. | Systems and methods for production and display of dynamically linked slide presentations |
CN108337451B (en) * | 2017-12-15 | 2020-04-24 | 北京纳米维景科技有限公司 | Image sensor simulation system and simulation method thereof |
US11585672B1 (en) | 2018-04-11 | 2023-02-21 | Palantir Technologies Inc. | Three-dimensional representations of routes |
US10942635B1 (en) | 2020-02-21 | 2021-03-09 | International Business Machines Corporation | Displaying arranged photos in sequence based on a locus of a moving object in photos |
WO2022138469A1 (en) * | 2020-12-21 | 2022-06-30 | 昭一 ▲高▼橋 | Last letter and last letter creation system |
Citations (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4688105A (en) * | 1985-05-10 | 1987-08-18 | Bloch Arthur R | Video recording system |
US5477264A (en) * | 1994-03-29 | 1995-12-19 | Eastman Kodak Company | Electronic imaging system using a removable software-enhanced storage device |
US5914748A (en) * | 1996-08-30 | 1999-06-22 | Eastman Kodak Company | Method and apparatus for generating a composite image using the difference of two images |
US6005972A (en) * | 1996-11-19 | 1999-12-21 | Eastman Kodak Company | Method for adding personalized text and/or graphics to composite digital image products |
US6072933A (en) * | 1995-03-06 | 2000-06-06 | Green; David | System for producing personalized video recordings |
US6085195A (en) * | 1998-06-02 | 2000-07-04 | Xstasis, Llc | Internet photo booth |
US6292219B1 (en) * | 1997-03-18 | 2001-09-18 | Eastman Kodak Company | Motion processing system using an effects-enhanced motion storage medium |
US20010033732A1 (en) * | 1996-09-20 | 2001-10-25 | Masahiro Juen | Image reproduction apparatus for reproducing multiple image files |
US20020007718A1 (en) * | 2000-06-20 | 2002-01-24 | Isabelle Corset | Karaoke system |
US20020030744A1 (en) * | 2000-04-07 | 2002-03-14 | Youichi Sawachi | Portable multi-function apparatus and controller |
US6369908B1 (en) * | 1999-03-31 | 2002-04-09 | Paul J. Frey | Photo kiosk for electronically creating, storing and distributing images, audio, and textual messages |
US20020110354A1 (en) * | 1997-01-09 | 2002-08-15 | Osamu Ikeda | Image recording and editing apparatus, and method for capturing and editing an image |
US20020136294A1 (en) * | 2001-03-21 | 2002-09-26 | Apple Computer, Inc. | Track for improved video compression |
US6556243B1 (en) * | 1997-06-13 | 2003-04-29 | Sanyo Electric, Co., Ltd. | Digital camera |
US20030169350A1 (en) * | 2002-03-07 | 2003-09-11 | Avi Wiezel | Camera assisted method and apparatus for improving composition of photography |
US20040004665A1 (en) * | 2002-06-25 | 2004-01-08 | Kotaro Kashiwa | System for creating content using content project data |
US20040095474A1 (en) * | 2002-11-11 | 2004-05-20 | Isao Matsufune | Imaging apparatus using imaging template |
US7024054B2 (en) * | 2002-09-27 | 2006-04-04 | Eastman Kodak Company | Method and system for generating a foreground mask for a composite image |
US7375769B2 (en) * | 2000-02-29 | 2008-05-20 | Canon Kabushiki Kaisha | Image processing apparatus |
US7391444B1 (en) * | 1999-07-23 | 2008-06-24 | Sharp Kabushiki Kaisha | Image pickup apparatus capable of selecting output according to time measured by timer |
US7519236B2 (en) * | 2003-04-09 | 2009-04-14 | Arcsoft, Inc. | Image retrieval |
US7580068B2 (en) * | 1997-07-15 | 2009-08-25 | Silverbrook Research Pty Ltd | Image data manipulation system |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3988266B2 (en) * | 1998-07-31 | 2007-10-10 | ソニー株式会社 | Recording / reproducing apparatus, recording / reproducing method, and recording medium |
US6362850B1 (en) * | 1998-08-04 | 2002-03-26 | Flashpoint Technology, Inc. | Interactive movie creation from one or more still images in a digital imaging device |
JP4446140B2 (en) * | 2000-01-26 | 2010-04-07 | ソニー株式会社 | Information processing apparatus and method, and program storage medium |
JP3708854B2 (en) * | 2001-09-04 | 2005-10-19 | 株式会社東芝 | Media production support device and program |
-
2004
- 2004-03-19 US US10/804,946 patent/US20050206751A1/en not_active Abandoned
-
2005
- 2005-03-04 EP EP05724419A patent/EP1726158A2/en not_active Withdrawn
- 2005-03-04 JP JP2007503932A patent/JP2007529951A/en active Pending
- 2005-03-04 WO PCT/US2005/006874 patent/WO2005096621A2/en not_active Application Discontinuation
-
2010
- 2010-10-21 US US12/908,926 patent/US8139126B2/en not_active Expired - Fee Related
Patent Citations (23)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4688105A (en) * | 1985-05-10 | 1987-08-18 | Bloch Arthur R | Video recording system |
US4688105B1 (en) * | 1985-05-10 | 1992-07-14 | Short Takes Inc | |
US5477264A (en) * | 1994-03-29 | 1995-12-19 | Eastman Kodak Company | Electronic imaging system using a removable software-enhanced storage device |
US6072933A (en) * | 1995-03-06 | 2000-06-06 | Green; David | System for producing personalized video recordings |
US5914748A (en) * | 1996-08-30 | 1999-06-22 | Eastman Kodak Company | Method and apparatus for generating a composite image using the difference of two images |
US20010033732A1 (en) * | 1996-09-20 | 2001-10-25 | Masahiro Juen | Image reproduction apparatus for reproducing multiple image files |
US6005972A (en) * | 1996-11-19 | 1999-12-21 | Eastman Kodak Company | Method for adding personalized text and/or graphics to composite digital image products |
US20020110354A1 (en) * | 1997-01-09 | 2002-08-15 | Osamu Ikeda | Image recording and editing apparatus, and method for capturing and editing an image |
US6292219B1 (en) * | 1997-03-18 | 2001-09-18 | Eastman Kodak Company | Motion processing system using an effects-enhanced motion storage medium |
US6556243B1 (en) * | 1997-06-13 | 2003-04-29 | Sanyo Electric, Co., Ltd. | Digital camera |
US7580068B2 (en) * | 1997-07-15 | 2009-08-25 | Silverbrook Research Pty Ltd | Image data manipulation system |
US6085195A (en) * | 1998-06-02 | 2000-07-04 | Xstasis, Llc | Internet photo booth |
US6369908B1 (en) * | 1999-03-31 | 2002-04-09 | Paul J. Frey | Photo kiosk for electronically creating, storing and distributing images, audio, and textual messages |
US7391444B1 (en) * | 1999-07-23 | 2008-06-24 | Sharp Kabushiki Kaisha | Image pickup apparatus capable of selecting output according to time measured by timer |
US7375769B2 (en) * | 2000-02-29 | 2008-05-20 | Canon Kabushiki Kaisha | Image processing apparatus |
US20020030744A1 (en) * | 2000-04-07 | 2002-03-14 | Youichi Sawachi | Portable multi-function apparatus and controller |
US20020007718A1 (en) * | 2000-06-20 | 2002-01-24 | Isabelle Corset | Karaoke system |
US20020136294A1 (en) * | 2001-03-21 | 2002-09-26 | Apple Computer, Inc. | Track for improved video compression |
US20030169350A1 (en) * | 2002-03-07 | 2003-09-11 | Avi Wiezel | Camera assisted method and apparatus for improving composition of photography |
US20040004665A1 (en) * | 2002-06-25 | 2004-01-08 | Kotaro Kashiwa | System for creating content using content project data |
US7024054B2 (en) * | 2002-09-27 | 2006-04-04 | Eastman Kodak Company | Method and system for generating a foreground mask for a composite image |
US20040095474A1 (en) * | 2002-11-11 | 2004-05-20 | Isao Matsufune | Imaging apparatus using imaging template |
US7519236B2 (en) * | 2003-04-09 | 2009-04-14 | Arcsoft, Inc. | Image retrieval |
Cited By (58)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20030184674A1 (en) * | 2002-03-26 | 2003-10-02 | Eastman Kodak Company | System for capturing and archiving motion video segments |
US7295244B2 (en) * | 2002-03-26 | 2007-11-13 | Eastman Kodak Company | System for capturing and archiving motion video segments |
US20060268121A1 (en) * | 2005-02-20 | 2006-11-30 | Nucore Technology Inc. | In-camera cinema director |
US20060285034A1 (en) * | 2005-06-15 | 2006-12-21 | Canon Kabushiki Kaisha | Image Display Method and Image Display Apparatus |
US7808555B2 (en) * | 2005-06-15 | 2010-10-05 | Canon Kabushiki Kaisha | Image display method and image display apparatus with zoom-in to face area of still image |
WO2007081599A3 (en) * | 2005-10-22 | 2008-04-24 | Gain Entpr Llc | Accelerated visual text to screen translation method |
WO2007081599A2 (en) * | 2005-10-22 | 2007-07-19 | Gain Enterprises, Llc | Accelerated visual text to screen translation method |
US9424883B2 (en) * | 2005-11-07 | 2016-08-23 | Koninklijke Philips N.V. | Method and apparatus for editing a video and/or audio program |
US20160042764A1 (en) * | 2005-11-07 | 2016-02-11 | Koninklijke Philips N.V. | Method and apparatus for editing a program |
US20070162855A1 (en) * | 2006-01-06 | 2007-07-12 | Kelly Hawk | Movie authoring |
US20070183497A1 (en) * | 2006-02-03 | 2007-08-09 | Jiebo Luo | Extracting key frame candidates from video clip |
US20070182861A1 (en) * | 2006-02-03 | 2007-08-09 | Jiebo Luo | Analyzing camera captured video for key frames |
US8031775B2 (en) * | 2006-02-03 | 2011-10-04 | Eastman Kodak Company | Analyzing camera captured video for key frames |
US7889794B2 (en) * | 2006-02-03 | 2011-02-15 | Eastman Kodak Company | Extracting key frame candidates from video clip |
US20070202480A1 (en) * | 2006-02-28 | 2007-08-30 | Steele Brenda S | Digital karaoke jukebox |
US8020097B2 (en) * | 2006-03-21 | 2011-09-13 | Microsoft Corporation | Recorder user interface |
US20070226254A1 (en) * | 2006-03-21 | 2007-09-27 | Microsoft Corporation | Recorder User Interface |
US20080077844A1 (en) * | 2006-09-26 | 2008-03-27 | Samsung Electronics Co., Ltd. | Apparatus and method for managing multimedia content in mobile terminal |
EP1906405A3 (en) * | 2006-09-26 | 2008-09-03 | Samsung Electronics Co., Ltd. | Apparatus and method for managing multimedia content in mobile terminal |
EP1906405A2 (en) | 2006-09-26 | 2008-04-02 | Samsung Electronics Co., Ltd. | Apparatus and method for managing multimedia content in mobile terminal |
US9142253B2 (en) | 2006-12-22 | 2015-09-22 | Apple Inc. | Associating keywords to media |
US20100020181A1 (en) * | 2006-12-26 | 2010-01-28 | Kabushiki Kaisha Toshiba | Information processing apparatus and information processing method |
US20140369667A1 (en) * | 2007-01-09 | 2014-12-18 | Sony Corporation | Information processing apparatus, information processing method, and program |
US20080276792A1 (en) * | 2007-05-07 | 2008-11-13 | Bennetts Christopher L | Lyrics superimposed on video feed |
US8934717B2 (en) * | 2007-06-05 | 2015-01-13 | Intellectual Ventures Fund 83 Llc | Automatic story creation using semantic classifiers for digital assets and associated metadata |
US20080304808A1 (en) * | 2007-06-05 | 2008-12-11 | Newell Catherine D | Automatic story creation using semantic classifiers for digital assets and associated metadata |
US20090003794A1 (en) * | 2007-06-28 | 2009-01-01 | Disney Enterprises, Inc. | Method and system for facilitating creation of content |
EP2160892B1 (en) * | 2007-06-28 | 2019-05-01 | Disney Enterprises, Inc. | Method and system for facilitating creation of content |
US7929028B2 (en) * | 2007-06-28 | 2011-04-19 | Disney Enterprises, Inc. | Method and system for facilitating creation of content |
US20090062944A1 (en) * | 2007-09-04 | 2009-03-05 | Apple Inc. | Modifying media files |
US20100220197A1 (en) * | 2009-03-02 | 2010-09-02 | John Nicholas Dukellis | Assisted Video Creation Utilizing a Camera |
US8860865B2 (en) | 2009-03-02 | 2014-10-14 | Burning Moon, Llc | Assisted video creation utilizing a camera |
US20110035700A1 (en) * | 2009-08-05 | 2011-02-10 | Brian Meaney | Multi-Operation User Interface Tool |
US9177407B2 (en) | 2009-10-20 | 2015-11-03 | Yahoo! Inc. | Method and system for assembling animated media based on keyword and string input |
WO2011049799A1 (en) * | 2009-10-20 | 2011-04-28 | Qwiki, Inc. | Method and system for assembling animated media based on keyword and string input |
US20110115799A1 (en) * | 2009-10-20 | 2011-05-19 | Qwiki, Inc. | Method and system for assembling animated media based on keyword and string input |
US8988456B2 (en) | 2010-03-25 | 2015-03-24 | Apple Inc. | Generating digital media presentation layouts dynamically based on image features |
US20110235858A1 (en) * | 2010-03-25 | 2011-09-29 | Apple Inc. | Grouping Digital Media Items Based on Shared Features |
US20110234613A1 (en) * | 2010-03-25 | 2011-09-29 | Apple Inc. | Generating digital media presentation layouts dynamically based on image features |
US8611678B2 (en) * | 2010-03-25 | 2013-12-17 | Apple Inc. | Grouping digital media items based on shared features |
WO2012037419A3 (en) * | 2010-09-16 | 2014-03-20 | Omnyx, LLC | Digital pathology image manipulation |
US8902334B2 (en) * | 2010-10-12 | 2014-12-02 | Sony Corporation | Terminal apparatus and method for adding an effect to an image |
US20120086834A1 (en) * | 2010-10-12 | 2012-04-12 | Sony Ericsson Mobile Communications Japan, Inc. | Terminal apparatus and method for adding an effect to an image |
US8584015B2 (en) | 2010-10-19 | 2013-11-12 | Apple Inc. | Presenting media content items using geographical data |
CN103384896A (en) * | 2011-10-21 | 2013-11-06 | 富士胶片株式会社 | Digital comic editing device and method therefor |
US9998722B2 (en) * | 2012-03-13 | 2018-06-12 | Tapshot, Inc. | System and method for guided video creation |
US20130272673A1 (en) * | 2012-03-13 | 2013-10-17 | Lee Eugene Swearingen | System and method for guided video creation |
US20150155008A1 (en) * | 2013-12-02 | 2015-06-04 | Magix Ag | System and method for theme based video creation with real-time effects |
US10026449B2 (en) * | 2013-12-02 | 2018-07-17 | Bellevue Investments Gmbh & Co. Kgaa | System and method for theme based video creation with real-time effects |
US10075757B2 (en) * | 2014-09-19 | 2018-09-11 | Foundation Partners Group, Llc | Multi-sensory environment room |
US20160088279A1 (en) * | 2014-09-19 | 2016-03-24 | Foundation Partners Group, Llc | Multi-sensory environment room |
US20180241937A1 (en) * | 2017-02-17 | 2018-08-23 | Microsoft Technology Licensing, Llc | Directed content capture and content analysis |
US20190104249A1 (en) * | 2017-09-29 | 2019-04-04 | Dwango Co., Ltd. | Server apparatus, distribution system, distribution method, and program |
US10645274B2 (en) * | 2017-09-29 | 2020-05-05 | Dwango Co., Ltd. | Server apparatus, distribution system, distribution method, and program with a distributor of live content and a viewer terminal for the live content including a photographed image of a viewer taking a designated body pose |
CN110866963A (en) * | 2018-08-28 | 2020-03-06 | 日本聚逸株式会社 | Moving image distribution system, moving image distribution method, and recording medium |
US20220350840A1 (en) * | 2020-11-18 | 2022-11-03 | Beijing Zitiao Network Technology Co., Ltd. | Method and apparatus for lyric video display, electronic device, and computer-readable medium |
CN112506412A (en) * | 2020-12-07 | 2021-03-16 | 北京达佳互联信息技术有限公司 | Video editing method and device and electronic equipment |
CN113840088A (en) * | 2021-09-23 | 2021-12-24 | 展讯通信(天津)有限公司 | Image recording method and device, electronic equipment and computer readable storage medium |
Also Published As
Publication number | Publication date |
---|---|
WO2005096621A2 (en) | 2005-10-13 |
JP2007529951A (en) | 2007-10-25 |
US20110032388A1 (en) | 2011-02-10 |
US8139126B2 (en) | 2012-03-20 |
WO2005096621A3 (en) | 2005-12-22 |
EP1726158A2 (en) | 2006-11-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8139126B2 (en) | Digital video system for assembling video sequences | |
US6061532A (en) | Animated image presentations with personalized digitized images | |
JP4261644B2 (en) | Multimedia editing method and apparatus | |
US7352952B2 (en) | System and method for improved video editing | |
US8244104B2 (en) | System for creating content using content project data | |
US9117483B2 (en) | Method and apparatus for dynamically recording, editing and combining multiple live video clips and still photographs into a finished composition | |
US20100220197A1 (en) | Assisted Video Creation Utilizing a Camera | |
US20130047082A1 (en) | Methods and systems for creating and editing video content on mobile devices | |
US20060268121A1 (en) | In-camera cinema director | |
US8406609B2 (en) | Method for generating a sound series of photographs and apparatus for the generating and reproducing such sound series | |
JP3942471B2 (en) | Data editing method, data editing device, data recording device, and recording medium | |
AU719329B2 (en) | Multi-media editing method and apparatus | |
Jago | Adobe Premiere Pro Classroom in a Book (2022 Release) | |
Wood | Sony Vegas Pro 11 Beginner's Guide | |
Eagle | Vegas Pro 9 Editing Workshop | |
Boykin | iMovie for iPhone and iPad | |
Kelly | Getting StartED with Windows Live Movie Maker | |
Scoppettuolo | Apple Pro Training Series: IMovie | |
Sachdeva et al. | Digital Video Editing | |
Besley et al. | Flash MX Video | |
Grothaus et al. | Using IMovie'11 | |
Costello | Time-Based Editing | |
JP5607332B2 (en) | Movie playback apparatus and computer program | |
Tiwary | SCREENING CONJUGALITY: THE AFFECTIVE INFRASTRUCTURE OF THE MARRIAGE VIDEO | |
Smith | Video in Photoshop for Photographers and Designers |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: EASTMAN KODAK COMPANY, NEW YORK Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MANICO, JOSEPH A.;FREDLUND, JOHN R.;PARULSKI, KENNETH A.;AND OTHERS;REEL/FRAME:015124/0307;SIGNING DATES FROM 20040310 TO 20040319 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: KODAK AVIATION LEASING LLC, NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK AMERICAS, LTD., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK PORTUGUESA LIMITED, NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK (NEAR EAST), INC., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: PAKON, INC., INDIANA Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: LASER-PACIFIC MEDIA CORPORATION, NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: FPC INC., CALIFORNIA Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK IMAGING NETWORK, INC., CALIFORNIA Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: QUALEX INC., NORTH CAROLINA Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK PHILIPPINES, LTD., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: EASTMAN KODAK INTERNATIONAL CAPITAL COMPANY, INC., Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: FAR EAST DEVELOPMENT LTD., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: CREO MANUFACTURING AMERICA LLC, WYOMING Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: EASTMAN KODAK COMPANY, NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: NPEC INC., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 Owner name: KODAK REALTY, INC., NEW YORK Free format text: PATENT RELEASE;ASSIGNORS:CITICORP NORTH AMERICA, INC.;WILMINGTON TRUST, NATIONAL ASSOCIATION;REEL/FRAME:029913/0001 Effective date: 20130201 |