US20140331137A1 - Method and apparatus for annotating video content with metadata generated using speech recognition technology - Google Patents

Method and apparatus for annotating video content with metadata generated using speech recognition technology Download PDF

Info

Publication number
US20140331137A1
US20140331137A1 US14/336,063 US201414336063A US2014331137A1 US 20140331137 A1 US20140331137 A1 US 20140331137A1 US 201414336063 A US201414336063 A US 201414336063A US 2014331137 A1 US2014331137 A1 US 2014331137A1
Authority
US
United States
Prior art keywords
video
particular portion
header
utterance
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US14/336,063
Inventor
Kevin M. McKoen
Michael A. Grossman
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Google Technology Holdings LLC
Original Assignee
Google Technology Holdings LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Google Technology Holdings LLC filed Critical Google Technology Holdings LLC
Priority to US14/336,063 priority Critical patent/US20140331137A1/en
Assigned to GENERAL INSTRUMENT CORPORATION reassignment GENERAL INSTRUMENT CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GROSSMAN, MICHAEL A., MCKOEN, KEVIN M.
Assigned to GENERAL INSTRUMENT HOLDINGS, INC. reassignment GENERAL INSTRUMENT HOLDINGS, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GENERAL INSTRUMENT CORPORATION
Assigned to MOTOROLA MOBILITY LLC reassignment MOTOROLA MOBILITY LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GENERAL INSTRUMENT HOLDINGS, INC.
Publication of US20140331137A1 publication Critical patent/US20140331137A1/en
Assigned to Google Technology Holdings LLC reassignment Google Technology Holdings LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MOTOROLA MOBILITY LLC
Priority to US15/472,033 priority patent/US10482168B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/91Television signal processing therefor
    • H04N5/92Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • G06F17/241
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/10Text processing
    • G06F40/166Editing, e.g. inserting or deleting
    • G06F40/169Annotation, e.g. comment data or footnotes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7844Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using original textual content or text extracted from visual content or transcript of audio data
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/26Speech to text systems
    • G10L15/265
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/02Editing, e.g. varying the order of information signals recorded on, or reproduced from, record carriers
    • G11B27/031Electronic editing of digitised analogue information signals, e.g. audio or video signals
    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10Indexing; Addressing; Timing or synchronising; Measuring tape travel

Definitions

  • the present invention relates generally to a method and apparatus for annotating video content, and more particularly to a method and apparatus for annotating video content with metadata that is converted to text from speech.
  • DVRs digital video recorders
  • third party service providers such as broadcasters may add metadata along with the associated content.
  • metadata source is an electronic program guide that is made available by some cable and satellite TV providers.
  • EPG metadata includes a title, time of broadcast, and a brief description of the broadcasts.
  • home movies Unlike commercially produced video, home movies generally contain little if any, metadata concerning their content which would allow them to be cataloged, searched and retrieved. Moreover, unlike the commercial context in which well defined metadata such as that available in an EPG can be used, consumers may often desire to index and retrieve movies in more idiosyncratic ways which do not lend themselves to the use of a limited number of descriptors. Even if consumers were to author metadata themselves, the process can be inconvenient and time-consuming.
  • a method and apparatus for annotating video content with metadata generated using speech recognition technology.
  • the method begins by rendering video content on a display device.
  • a segment of speech is received from a user such that the speech segment annotates a portion of the video content currently being rendered.
  • the speech segment is converted to a text-segment and the text-segment is associated with the rendered portion of the video content.
  • the text segment is stored in a selectively retrievable manner so that it is associated with the rendered portion of the video content.
  • a signal may be received from the user selecting an operational state before receiving the segment of speech.
  • the operational state may be selected from the group consisting of an annotate state, a narrate state, a commentary state, an analyze state and a review/edit state.
  • the video content may be rendered by a set top box.
  • the video content may be rendered by a DVR.
  • the set top box may receive the video content from a video camera.
  • the DVR may receive the video content from a video camera.
  • the user may be presented with a plurality of different user-selectable operational states defining a mode in which the speech request is to be received.
  • the user-selectable operational states may be presented as a GUI on the display device.
  • the GUI may be superimposed over the video content being rendered.
  • FIG. 1 shows an arrangement for transferring video information from a digital video camera to a video storage device such as a digital video recorder (DVR).
  • DVR digital video recorder
  • FIG. 2 shows a screen shot on a display device in which a graphical user interface (GUI) representing a series of menu choices is superimposed over the video.
  • GUI graphical user interface
  • FIG. 3 shows an illustrative set-top box equipped with a DVR subsystem.
  • FIG. 4 shows one example of an architecture that may be employed by the metadata generation module depicted in FIG. 3 .
  • FIG. 5 shows an illustrative MPEG digital video transport stream or sequence in which, in some examples, the metadata may be inserted.
  • FIG. 6 is an illustrative state diagram showing one particular example of the various operational states of the metadata generation module from among which the user may select.
  • Described herein is a method and apparatus for archiving and annotating a large collection of home movies, turning the browsing and assembling of video into a simple and even enjoyable task.
  • video information from a video camera or the like is transferred to a video storage device that includes a voice-based metadata generation module.
  • the end-user uses a microphone or other voice input device to annotate the video.
  • the speech undergoes speech-to-text conversion to provide a source of metadata that can be used to catalogue, search and retrieve the locally stored video.
  • Some of the metadata may include in-depth descriptions and details concerning the content.
  • a user may wish to provide searchable access to a video of a recent family vacation.
  • the user may wish to identify segments of the video by their location and the family members or other individuals appearing in the video segments. If, for instance, a first segment of video was recorded on a ski slope and a second portion was recorded in a ski lodge, the user may verbally describe the ski slope, pointing out say, that Tim and Beth are skiing, while the first segment is being played or otherwise rendered. Likewise, when the second video segment is being played, the user may describe the ski lodge in whatever detail is desired and identify those individuals who are present. In this way the user can continue to narrate the entire video.
  • FIG. 1 shows an arrangement for transferring video information from a digital video camera 110 to a video storage device such as a digital video recorder (DVR) 115 .
  • the DVR 115 and the digital video camera 110 include interfaces conforming to an appropriate standard (e.g., IEEE 1394) and are connected by a cable 120 conforming to the standard.
  • an appropriate standard e.g., IEEE 1394
  • the recorder 115 and the video camera 110 can recognize each other and communicate with each other.
  • a display device 125 such as a computer monitor or television is connected to the digital video recorder 115 for displaying the video content stored on the recorder 115 .
  • FIG. 1 Also shown in FIG. 1 are a variety of voice input devices that are connected to the digital video recorder 115 , any of which may be employed by the user to annotate the video.
  • the voice input devices that are depicted include a microphone 130 , cellular phone 135 , and a remote control unit 135 equipped with a built-in microphone.
  • the video recorder 115 is depicted as being in communication with the digital video camera 110 , microphone 130 , cellular phone 135 , and remote control unit 135 over wired links, they may alternatively be in communication with the video recorder 115 over a wireless link.
  • the digital video recorder 115 includes a voice-based metadata generation module, described in more detail below, which receives the speech from one or more of the voice input devices.
  • the metadata generation module includes a speech recognition component for converting speech to text. The resulting text is used by the metadata generation module to generate metadata for the video.
  • FIG. 2 shows a screen shot on the display device.
  • the video stored on the DVR 115 is displayed along with a graphical user interface (GUI) 145 that is superimposed on top of the video.
  • GUI graphical user interface
  • the GUI 145 includes a series of menu choices from among which the user can select using an appropriate user input device such as a remote control unit. The various menu choices will be described in more detail below in connection with the operational states of the metadata generation module.
  • the video storage device on which the video is transferred from the video camera is not limited to a digital video recorder 115 as depicted in FIGS. 1 and 2 . More generally, the video storage device may be implemented in a wide variety of different devices or combination of devices capable of receiving, recording and playing back a video signal.
  • the video storage device may be a set-top box, television, personal computer, PDA and the like.
  • the video storage device in which the metadata generation module is incorporated will be depicted in FIG. 3 as a set-top box equipped with a DVR subsystem.
  • set top box 10 includes external network connection/communication interfaces 59 , which support devices such as modems, streaming media players and other network connection support devices and/or software, coupled through local or wide area networks (e.g., broadband network 11 ) to program providers and providers of other content, such as advertising content.
  • Communication interfaces 59 also include the appropriate connections for connecting the video camera 110 and various input devices depicted in FIG. 1 .
  • Set top box 10 further includes an in-band tuner 43 , which tunes to a channel signal 16 selected by a viewer via user interface 55 .
  • user interface 55 may be any type of known or future device or technology allowing the consumer to select channels or programs the consumer wishes to receive, such as a remote control, mouse, microphone, keyboard, or touch-screen display associated with a personal video recorder.
  • Channel signal 16 includes video and/or audio components.
  • Demodulators 40 and 42 are responsive to in-band tuner 43 .
  • Demodulator 40 which may be a 64/256 quadrature amplitude modulation demodulator, for example, is responsive to receive a digital version of channel signal 16 .
  • Demodulator 40 identifies digital data packets from one or more digital sources, such as a Moving Pictures Experts' Group (MPEG) transport stream, a high-definition television stream, or a media stream from an external network connection 59 , such as a cable modem, using well-known methods and techniques.
  • Demodulator 42 which may be an NTSC demodulator, for example, is responsive to receive an analog version of channel signal 16 and to decode signals and markers according to well-known methods and techniques. Demodulators 40 and 42 are operative to output video information 20 .
  • MPEG Moving Pictures Experts' Group
  • Video information 20 includes raw video or audio data, arranged for formatting in accordance with a predetermined media format.
  • Video information 20 is preferably arranged in accordance with an MPEG media format, such as the MPEG-2 media format, but may be arranged in accordance with other media formats, including but not limited to other MPEG formats, Hypertext Markup Language (HTML), Virtual Reality Modeling Language (VRML), extensible markup language (XML), H.261, or H.263 formats.
  • MPEG media format such as the MPEG-2 media format
  • Storage medium 64 is responsive to receive, among other things, encoded video signal 20 for storage.
  • Storage medium 64 may be any local or remote device, now known or later developed, capable of recording data, including but not limited to a hard disk drive, a videocassette recorder tape, all types of optical storage media such as compact disks and digital videodisks, a magnetic tape, a home router, or a server.
  • the video transferred from the video camera 115 in FIG. 1 is received by connection/communication interfaces 59 and stored in storage medium 64 .
  • Decoder 49 is responsive to receive recorded encoded video signal 20 from storage medium 64 , and to play back recorded encoded video signal 20 via display device 25 , in response to instructions from user interface 55 . Decoder 49 is also responsive to receive and pass through video programming directly from tuner 43 .
  • Internal arrangements of decoder 49 are well known—decoder 49 may include analog-to-digital converters, one or more storage media and/or buffers, and general or special-purpose processors or application-specific integrated circuits, along with demultiplexors for demultiplexing and/or synchronizing at least two transport streams, for example, video and audio.
  • Video and audio decoders and/or analog and digital decoders may be separate, with communication between separate decoders allowing for synchronization, error correction and control.
  • Metadata generation module 30 allows a user to create, review and revise metadata that is obtained from a spoken word stream while the video is being played.
  • the metadata generation module 30 may represent the metadata in any appropriate format.
  • the TV-Anytime forum has published metadata specifications which define the data structures that allow consumers to find, navigate and manage content.
  • the “TV-Anytime” standard is an emerging standard which is defined by the TV-Anytime forum.
  • the metadata is represented in XML.
  • an MPEG standard for the effective and efficient access to multimedia content (ISO/IEC 15938: Information Technology—Multimedia content description interface), MPEG-7, offers metadata elements and their structure and relationships, which are defined in the form of Descriptors and Description Schemes to create descriptions of the multimedia content.
  • MPEG-7 specifies a standard set of description tools which can be used to describe various types of multimedia information. MPEG-7 does not replace earlier MPEG standards, as its objective is to provide additional functionality to other MPEG standards. Earlier MPEG standards make the content available, while MPEG-7 allows finding the content the user needs.
  • the metadata may be physically located with the associated video, either in the same data stream or on the same local storage medium, but the metadata descriptions could also be located somewhere else, such as on a network. If the content and its metadata descriptions are not co-located, a mechanism that links these two is employed, such as a content identifier, for example.
  • Display device 25 which includes speakers for outputting audio signals, displaying video programming received from tuner 43 or storage medium 64 .
  • Display device 25 is responsive to receive analog or digital signals, including but not limited to S-video, composite audio-video, SPDIF, and DVI.
  • Processor 39 and software 22 are illustrated functionally, and are responsive to various elements of set top box 10 , including demodulators 40 and 42 , storage medium 64 , decoder 49 and metadata generation module 30 .
  • software 22 When loaded into a processor, such as processor 39 , software 22 is operative to control encoding, recording and playback of encoded video signal 20 .
  • aspects of the set top box are not limited to any specific embodiments of computer software or signal processing methods.
  • one or more processors packaged together or with other elements of set top box 10 may implement functions of processor 39 in a variety of ways.
  • software 22 may be any stored instructions, in one or more parts (stored, for example, on storage medium 64 , or another internal or external storage medium such as a read-only-memory or a random-access memory) electronically controlling functions provided by DVR system 10 , including firmware, and may be used or implemented by one or more elements, including one or more processors, of set top box 10 .
  • FIG. 4 shows one example of an architecture that may be employed by the metadata generation module 30 .
  • the metadata generation module 30 includes a user interface component 210 , a voice recognition component 220 , an annotation metadata engine 230 , and playback control logic 240 .
  • the user interface component 210 displays GUI screen shots, such as shown in FIG. 2 for example, plus the other interactive functionalities needed to enable user control over the annotation.
  • the voice-recognition component 220 also referred to as the speech recognition component, performs the voice recognition.
  • speech recognition and “voice recognition” are interchangeably used.
  • the voice recognition component 220 may be either a speaker dependent speech recognition unit or a speaker independent speech recognition unit.
  • Such conventional voice recognition components are well known in the art and thus do not need to be discussed in detail.
  • a speaker dependent speech recognition configuration a speaker is identified, and only words or phrases which are spoken by the identified speaker are recognized.
  • specific words are recognized, regardless of the person who speaks them.
  • the annotation metadata engine 230 generates the metadata from the text that is provided by the voice-recognition component using any appropriate format such as the aforementioned TV-Anytime and MPEG-7 standards.
  • the annotation metadata engine 230 also inserts the metadata into an appropriate database record or file or directly into the MPEG bitstream.
  • the playback control logic 240 coordinates the playback of the video stream with the incorporation of the metadata that is received from the user.
  • the metadata generation module 30 can reside in a software application, firmware or hardware layer, or in any combination thereof.
  • the metadata generation module may be implemented in hardware using an ASIC along with associated firmware.
  • Metadata generation module 30 can establish the association between the metadata and the video in any number of different ways. For instance, as previously mentioned, the metadata may be stored in a database record. Alternatively, in another example, metadata generation module 30 inserts the metadata into the digital bitstream.
  • FIG. 5 shows the sequence headers in an illustrative MPEG digital video transport stream or sequence. Of course, an MPEG stream is shown for illustrative purposes only. The techniques described herein are more generally applicable to a digitally encoded video stream that conforms to any appropriate standard. Typically, the succession of frames comprising such a video sequence is divided for convenience into groups of frames or groups of pictures (GOP).
  • the MPEG standard defines a sequence layer and a GOP layer. The sequence layer begins with a sequence header and ends with a sequence end.
  • the sequence layer comprises more than one GOP.
  • the GOP layer begins with a GOP header and comprises a plurality of pictures or frames.
  • the first frame is generally an I-picture, followed by a P-picture and a B-picture.
  • MPEG provides flexibility as to the use, size, and make up of the GOP, but a 12-frame GOP is typical for a 25 frames per second system frame rate and a 15-frame GOP is typical for a 30 frames per second system.
  • the annotation metadata engine inserts the metadata and an associated time-stamp as user data bits into the headers associated with the video stream. In this way the metadata will be available to the MPEG decoder resident on set top box so it can be extracted, synchronized with the video stream and perhaps even displayed on the screen as either a caption or subtitle.
  • the metadata generation module 30 may be operable in a number of different states so that the user can easily and conveniently annotate a video.
  • FIG. 6 is an illustrative state diagram showing one particular example of the various operational states of the metadata generation module 30 from among which the user may select. Of course, these states are presented by way of illustration only and are not to be construed as limiting in any way. Each of the various states may be entered using an appropriate user input device such as a remote control unit that is associated with the video storage device. The various states from among which the user selects may be presented on the display device, such as shown in FIG. 1 .
  • the user may begin the annotation process while the video is being transferred from the video camera to the video storage device or after the transfer has already been performed.
  • the user first activates or arms the metadata generation module 30 by entering an armed voice-recognition annotation state, which serves as an entry point to the other states, which include an annotate state, a narrate state, a commentary state, an analyze state and a review/edit state.
  • the user simply responds to predefined questions or categories that are presented on the display device. For instance, the user may be asked to enter a title, topic, date and location of the video.
  • the annotate state is thus used to provide basic information about the video similar in type to the information that may be found, for example, in an EPG for a broadcast program.
  • the narrate state the user can provide a more detailed and comprehensive description of the video as it is being displayed. In this state the user can annotate in whatever detail is desired each and every scene or segment of the video or selected scenes or segments.
  • the voice recognition component will typically remain in an active state throughout the playback of the video so that the user can describe any portion of it as it is displayed on the screen. The user can fast-forward, rewind and pause the video at any point.
  • the commentary state is similar to the narrate state except that the default state of the voice recognition component will typically be inactive throughout the playback of the video. This state is most convenient to use when the user is primarily watching the video but does not anticipate adding a running commentary. On those occasions when the user does wish to add a description, he or she must activate the voice recognition component with the user interface by, for example, actuating a press-to-talk button. Once again, the user can fast-forward, rewind and pause the video at any point.
  • the analyze state serves as a description checker analogous to a spell-checker.
  • the analyze state will generally be entered manually or automatically after completion of the narrate state. This state highlights for the user such items as ambiguous words or phrases, scenes or segments that lack a description, or anything else that might suggest there is an irregularity in the text that has been converted from the user's speech. If the user in fact wishes to modify any of the descriptions, the user enters the review/edit state to do so.
  • the edit/review state may be entered from any number of other states. For instance, it is accessible during the narration state to refine the content. It may also be entered to correct a description while in the commentary state or at the end of the commentary state. In the edit/review state playback of the video is paused and the current description is displayed on the screen along with a substitute line in which the new description is added.

Abstract

A method and apparatus is provided for annotating video content with metadata generated using speech recognition technology. The method begins by rendering video content on a display device. A segment of speech is received from a user such that the speech segment annotates a portion of the video content currently being rendered. The speech segment is converted to a text-segment and the text-segment is associated with the rendered portion of the video content. The text segment is stored in a selectively retrievable manner so that it is associated with the rendered portion of the video content.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • This application is a continuation of U.S. application Ser. No. 13/654,327, filed Oct. 17, 2012, which is a continuation of U.S. application Ser. No.: 11/747,584, filed May 11, 2007, which issued as U.S. Pat. No. 8,316,302 on Nov. 20, 2012, the contents of which is incorporated by reference.
  • FIELD OF THE INVENTION
  • The present invention relates generally to a method and apparatus for annotating video content, and more particularly to a method and apparatus for annotating video content with metadata that is converted to text from speech.
  • BACKGROUND OF THE INVENTION
  • Consumers are more and more often recording family events, travel experiences and the like using digital video recording techniques. Today's advanced home video camera technologies allow huge amounts of such “home-grown” and other video data to be conveniently recorded and stored. For example, future generations of digital video recorders (DVRs) will provide storage capacities measured in terabytes. The more storage available, the more content the user will be likely to store. Services are also commonplace to convert older analog video formats such as film into more modern digital formats, increasing the quantity of digital video material even more. Commercial video editing products also allow for the creation of videos that greatly increase the quality and presentation of home movies.
  • Unfortunately, as easy as it is to record and edit family and other movies, archiving and retrieving them is not quite as simple. Unlike a stack of photos, a collection of home movies is not easily browsed. Finding a particular segment of content depends on identifying the specific item, which means that the item has to be categorized. For commercially produced movies and songs there are a number of services available today which provide detailed metadata (e.g. Internet Movie Database IMDb for movies, CDDB for music). But to be of real use for the management of large amounts of content, this metadata should stored in conjunction with the related content and presented to the user as one coherent item.
  • Recently, a number of standards such as MPEG-7 and TV-AnyTime have been developed which allow features of multimedia content to be described so that users can search for, browse and retrieve that content as efficiently and effectively as today's text-based search engines. These standards generally use a set of audio-visual metadata elements. Such so-called multimedia content management systems are already widely used in commercial contexts by content creators/owners and service providers/broadcasters. For example, in the case of DVD movies, metadata is often provided as part of the content itself so that users can access a specific part of a video through a “chapter selection” interface. Access to specific segments of the recorded program requires segmentation information concerning a program that describes a title, category, start position and duration of each segment, which is generated through a process called “video indexing”. To access a specific segment without the segmentation information of a program, viewers would have to linearly search through the program from the beginning, such as by using the fast forward button, which is a cumbersome and time-consuming process.
  • In another commercial context, third party service providers such as broadcasters may add metadata along with the associated content. For example, one such metadata source is an electronic program guide that is made available by some cable and satellite TV providers. EPG metadata includes a title, time of broadcast, and a brief description of the broadcasts.
  • Unlike commercially produced video, home movies generally contain little if any, metadata concerning their content which would allow them to be cataloged, searched and retrieved. Moreover, unlike the commercial context in which well defined metadata such as that available in an EPG can be used, consumers may often desire to index and retrieve movies in more idiosyncratic ways which do not lend themselves to the use of a limited number of descriptors. Even if consumers were to author metadata themselves, the process can be inconvenient and time-consuming.
  • SUMMARY OF THE INVENTION
  • In accordance with the present invention, a method and apparatus is provided for annotating video content with metadata generated using speech recognition technology. The method begins by rendering video content on a display device. A segment of speech is received from a user such that the speech segment annotates a portion of the video content currently being rendered. The speech segment is converted to a text-segment and the text-segment is associated with the rendered portion of the video content. The text segment is stored in a selectively retrievable manner so that it is associated with the rendered portion of the video content.
  • In accordance with one aspect of the invention, a signal may be received from the user selecting an operational state before receiving the segment of speech.
  • In accordance with another aspect of the invention, the operational state may be selected from the group consisting of an annotate state, a narrate state, a commentary state, an analyze state and a review/edit state.
  • In accordance with another aspect of the invention, the video content may be rendered by a set top box.
  • In accordance with another aspect of the invention, the video content may be rendered by a DVR.
  • In accordance with another aspect of the invention, the set top box may receive the video content from a video camera.
  • In accordance with another aspect of the invention, the DVR may receive the video content from a video camera.
  • In accordance with another aspect of the invention, the user may be presented with a plurality of different user-selectable operational states defining a mode in which the speech request is to be received.
  • In accordance with another aspect of the invention, the user-selectable operational states may be presented as a GUI on the display device.
  • In accordance with another aspect of the invention, the GUI may be superimposed over the video content being rendered.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 shows an arrangement for transferring video information from a digital video camera to a video storage device such as a digital video recorder (DVR).
  • FIG. 2 shows a screen shot on a display device in which a graphical user interface (GUI) representing a series of menu choices is superimposed over the video.
  • FIG. 3 shows an illustrative set-top box equipped with a DVR subsystem.
  • FIG. 4 shows one example of an architecture that may be employed by the metadata generation module depicted in FIG. 3.
  • FIG. 5 shows an illustrative MPEG digital video transport stream or sequence in which, in some examples, the metadata may be inserted.
  • FIG. 6 is an illustrative state diagram showing one particular example of the various operational states of the metadata generation module from among which the user may select.
  • DETAILED DESCRIPTION
  • Described herein is a method and apparatus for archiving and annotating a large collection of home movies, turning the browsing and assembling of video into a simple and even enjoyable task. As described in more detail below, video information from a video camera or the like is transferred to a video storage device that includes a voice-based metadata generation module. As the video on the storage device is being displayed, the end-user uses a microphone or other voice input device to annotate the video. The speech undergoes speech-to-text conversion to provide a source of metadata that can be used to catalogue, search and retrieve the locally stored video. Some of the metadata may include in-depth descriptions and details concerning the content.
  • In one illustrative scenario, a user may wish to provide searchable access to a video of a recent family vacation. The user may wish to identify segments of the video by their location and the family members or other individuals appearing in the video segments. If, for instance, a first segment of video was recorded on a ski slope and a second portion was recorded in a ski lodge, the user may verbally describe the ski slope, pointing out say, that Tim and Beth are skiing, while the first segment is being played or otherwise rendered. Likewise, when the second video segment is being played, the user may describe the ski lodge in whatever detail is desired and identify those individuals who are present. In this way the user can continue to narrate the entire video.
  • FIG. 1 shows an arrangement for transferring video information from a digital video camera 110 to a video storage device such as a digital video recorder (DVR) 115. The DVR 115 and the digital video camera 110 include interfaces conforming to an appropriate standard (e.g., IEEE 1394) and are connected by a cable 120 conforming to the standard. Thus, the recorder 115 and the video camera 110 can recognize each other and communicate with each other. A display device 125 such as a computer monitor or television is connected to the digital video recorder 115 for displaying the video content stored on the recorder 115.
  • Also shown in FIG. 1 are a variety of voice input devices that are connected to the digital video recorder 115, any of which may be employed by the user to annotate the video. In particular, the voice input devices that are depicted include a microphone 130, cellular phone 135, and a remote control unit 135 equipped with a built-in microphone. It should be noted that while the video recorder 115 is depicted as being in communication with the digital video camera 110, microphone 130, cellular phone 135, and remote control unit 135 over wired links, they may alternatively be in communication with the video recorder 115 over a wireless link.
  • The digital video recorder 115 includes a voice-based metadata generation module, described in more detail below, which receives the speech from one or more of the voice input devices. The metadata generation module includes a speech recognition component for converting speech to text. The resulting text is used by the metadata generation module to generate metadata for the video.
  • FIG. 2 shows a screen shot on the display device. The video stored on the DVR 115 is displayed along with a graphical user interface (GUI) 145 that is superimposed on top of the video. The GUI 145 includes a series of menu choices from among which the user can select using an appropriate user input device such as a remote control unit. The various menu choices will be described in more detail below in connection with the operational states of the metadata generation module.
  • It should be noted that the video storage device on which the video is transferred from the video camera is not limited to a digital video recorder 115 as depicted in FIGS. 1 and 2. More generally, the video storage device may be implemented in a wide variety of different devices or combination of devices capable of receiving, recording and playing back a video signal. For example, the video storage device may be a set-top box, television, personal computer, PDA and the like. For purposes of illustration, the video storage device in which the metadata generation module is incorporated will be depicted in FIG. 3 as a set-top box equipped with a DVR subsystem.
  • Referring to FIG. 3, set top box 10 includes external network connection/communication interfaces 59, which support devices such as modems, streaming media players and other network connection support devices and/or software, coupled through local or wide area networks (e.g., broadband network 11) to program providers and providers of other content, such as advertising content. Communication interfaces 59 also include the appropriate connections for connecting the video camera 110 and various input devices depicted in FIG. 1.
  • Set top box 10 further includes an in-band tuner 43, which tunes to a channel signal 16 selected by a viewer via user interface 55. As mentioned in connection with FIG. 1, user interface 55 may be any type of known or future device or technology allowing the consumer to select channels or programs the consumer wishes to receive, such as a remote control, mouse, microphone, keyboard, or touch-screen display associated with a personal video recorder.
  • Channel signal 16 includes video and/or audio components. Demodulators 40 and 42 are responsive to in-band tuner 43. Demodulator 40, which may be a 64/256 quadrature amplitude modulation demodulator, for example, is responsive to receive a digital version of channel signal 16. Demodulator 40 identifies digital data packets from one or more digital sources, such as a Moving Pictures Experts' Group (MPEG) transport stream, a high-definition television stream, or a media stream from an external network connection 59, such as a cable modem, using well-known methods and techniques. Demodulator 42, which may be an NTSC demodulator, for example, is responsive to receive an analog version of channel signal 16 and to decode signals and markers according to well-known methods and techniques. Demodulators 40 and 42 are operative to output video information 20.
  • Video information 20 includes raw video or audio data, arranged for formatting in accordance with a predetermined media format. Video information 20 is preferably arranged in accordance with an MPEG media format, such as the MPEG-2 media format, but may be arranged in accordance with other media formats, including but not limited to other MPEG formats, Hypertext Markup Language (HTML), Virtual Reality Modeling Language (VRML), extensible markup language (XML), H.261, or H.263 formats.
  • Storage medium 64 is responsive to receive, among other things, encoded video signal 20 for storage. Storage medium 64 may be any local or remote device, now known or later developed, capable of recording data, including but not limited to a hard disk drive, a videocassette recorder tape, all types of optical storage media such as compact disks and digital videodisks, a magnetic tape, a home router, or a server. The video transferred from the video camera 115 in FIG. 1 is received by connection/communication interfaces 59 and stored in storage medium 64.
  • Decoder 49 is responsive to receive recorded encoded video signal 20 from storage medium 64, and to play back recorded encoded video signal 20 via display device 25, in response to instructions from user interface 55. Decoder 49 is also responsive to receive and pass through video programming directly from tuner 43. Internal arrangements of decoder 49 are well known—decoder 49 may include analog-to-digital converters, one or more storage media and/or buffers, and general or special-purpose processors or application-specific integrated circuits, along with demultiplexors for demultiplexing and/or synchronizing at least two transport streams, for example, video and audio. Video and audio decoders and/or analog and digital decoders may be separate, with communication between separate decoders allowing for synchronization, error correction and control.
  • Metadata generation module 30 allows a user to create, review and revise metadata that is obtained from a spoken word stream while the video is being played. The metadata generation module 30 may represent the metadata in any appropriate format. For example, the TV-Anytime forum has published metadata specifications which define the data structures that allow consumers to find, navigate and manage content. The “TV-Anytime” standard is an emerging standard which is defined by the TV-Anytime forum. In the TV-Anytime standard, the metadata is represented in XML. Similarly, an MPEG standard for the effective and efficient access to multimedia content (ISO/IEC 15938: Information Technology—Multimedia content description interface), MPEG-7, offers metadata elements and their structure and relationships, which are defined in the form of Descriptors and Description Schemes to create descriptions of the multimedia content. MPEG-7 specifies a standard set of description tools which can be used to describe various types of multimedia information. MPEG-7 does not replace earlier MPEG standards, as its objective is to provide additional functionality to other MPEG standards. Earlier MPEG standards make the content available, while MPEG-7 allows finding the content the user needs.
  • Regardless of format, the metadata may be physically located with the associated video, either in the same data stream or on the same local storage medium, but the metadata descriptions could also be located somewhere else, such as on a network. If the content and its metadata descriptions are not co-located, a mechanism that links these two is employed, such as a content identifier, for example.
  • Also shown in FIG. 3 is a display device 25, which includes speakers for outputting audio signals, displaying video programming received from tuner 43 or storage medium 64. Display device 25 is responsive to receive analog or digital signals, including but not limited to S-video, composite audio-video, SPDIF, and DVI.
  • Processor 39 and software 22 are illustrated functionally, and are responsive to various elements of set top box 10, including demodulators 40 and 42, storage medium 64, decoder 49 and metadata generation module 30. When loaded into a processor, such as processor 39, software 22 is operative to control encoding, recording and playback of encoded video signal 20. It will be appreciated, however, that aspects of the set top box are not limited to any specific embodiments of computer software or signal processing methods. For example, one or more processors packaged together or with other elements of set top box 10 may implement functions of processor 39 in a variety of ways. It will also be appreciated that software 22 may be any stored instructions, in one or more parts (stored, for example, on storage medium 64, or another internal or external storage medium such as a read-only-memory or a random-access memory) electronically controlling functions provided by DVR system 10, including firmware, and may be used or implemented by one or more elements, including one or more processors, of set top box 10.
  • FIG. 4 shows one example of an architecture that may be employed by the metadata generation module 30. The metadata generation module 30 includes a user interface component 210, a voice recognition component 220, an annotation metadata engine 230, and playback control logic 240. The user interface component 210 displays GUI screen shots, such as shown in FIG. 2 for example, plus the other interactive functionalities needed to enable user control over the annotation. The voice-recognition component 220, also referred to as the speech recognition component, performs the voice recognition. Hereinafter, the terms “speech recognition” and “voice recognition” are interchangeably used. The voice recognition component 220 may be either a speaker dependent speech recognition unit or a speaker independent speech recognition unit. Such conventional voice recognition components are well known in the art and thus do not need to be discussed in detail. Generally, in a speaker dependent speech recognition configuration a speaker is identified, and only words or phrases which are spoken by the identified speaker are recognized. In a speaker independent speech recognition configuration specific words are recognized, regardless of the person who speaks them.
  • The annotation metadata engine 230 generates the metadata from the text that is provided by the voice-recognition component using any appropriate format such as the aforementioned TV-Anytime and MPEG-7 standards. The annotation metadata engine 230 also inserts the metadata into an appropriate database record or file or directly into the MPEG bitstream. The playback control logic 240 coordinates the playback of the video stream with the incorporation of the metadata that is received from the user. It should be noted that the metadata generation module 30 can reside in a software application, firmware or hardware layer, or in any combination thereof. For example, in some cases the metadata generation module may be implemented in hardware using an ASIC along with associated firmware.
  • Metadata generation module 30 can establish the association between the metadata and the video in any number of different ways. For instance, as previously mentioned, the metadata may be stored in a database record. Alternatively, in another example, metadata generation module 30 inserts the metadata into the digital bitstream. FIG. 5 shows the sequence headers in an illustrative MPEG digital video transport stream or sequence. Of course, an MPEG stream is shown for illustrative purposes only. The techniques described herein are more generally applicable to a digitally encoded video stream that conforms to any appropriate standard. Typically, the succession of frames comprising such a video sequence is divided for convenience into groups of frames or groups of pictures (GOP). The MPEG standard defines a sequence layer and a GOP layer. The sequence layer begins with a sequence header and ends with a sequence end. The sequence layer comprises more than one GOP. The GOP layer begins with a GOP header and comprises a plurality of pictures or frames. The first frame is generally an I-picture, followed by a P-picture and a B-picture. MPEG provides flexibility as to the use, size, and make up of the GOP, but a 12-frame GOP is typical for a 25 frames per second system frame rate and a 15-frame GOP is typical for a 30 frames per second system. As shown, the annotation metadata engine inserts the metadata and an associated time-stamp as user data bits into the headers associated with the video stream. In this way the metadata will be available to the MPEG decoder resident on set top box so it can be extracted, synchronized with the video stream and perhaps even displayed on the screen as either a caption or subtitle.
  • The metadata generation module 30 may be operable in a number of different states so that the user can easily and conveniently annotate a video. FIG. 6 is an illustrative state diagram showing one particular example of the various operational states of the metadata generation module 30 from among which the user may select. Of course, these states are presented by way of illustration only and are not to be construed as limiting in any way. Each of the various states may be entered using an appropriate user input device such as a remote control unit that is associated with the video storage device. The various states from among which the user selects may be presented on the display device, such as shown in FIG. 1.
  • The user may begin the annotation process while the video is being transferred from the video camera to the video storage device or after the transfer has already been performed. The user first activates or arms the metadata generation module 30 by entering an armed voice-recognition annotation state, which serves as an entry point to the other states, which include an annotate state, a narrate state, a commentary state, an analyze state and a review/edit state.
  • In the annotate state, the user simply responds to predefined questions or categories that are presented on the display device. For instance, the user may be asked to enter a title, topic, date and location of the video. The annotate state is thus used to provide basic information about the video similar in type to the information that may be found, for example, in an EPG for a broadcast program. In the narrate state the user can provide a more detailed and comprehensive description of the video as it is being displayed. In this state the user can annotate in whatever detail is desired each and every scene or segment of the video or selected scenes or segments. In the narrate state the voice recognition component will typically remain in an active state throughout the playback of the video so that the user can describe any portion of it as it is displayed on the screen. The user can fast-forward, rewind and pause the video at any point.
  • The commentary state is similar to the narrate state except that the default state of the voice recognition component will typically be inactive throughout the playback of the video. This state is most convenient to use when the user is primarily watching the video but does not anticipate adding a running commentary. On those occasions when the user does wish to add a description, he or she must activate the voice recognition component with the user interface by, for example, actuating a press-to-talk button. Once again, the user can fast-forward, rewind and pause the video at any point.
  • The analyze state serves as a description checker analogous to a spell-checker. The analyze state will generally be entered manually or automatically after completion of the narrate state. This state highlights for the user such items as ambiguous words or phrases, scenes or segments that lack a description, or anything else that might suggest there is an irregularity in the text that has been converted from the user's speech. If the user in fact wishes to modify any of the descriptions, the user enters the review/edit state to do so.
  • The edit/review state may be entered from any number of other states. For instance, it is accessible during the narration state to refine the content. It may also be entered to correct a description while in the commentary state or at the end of the commentary state. In the edit/review state playback of the video is paused and the current description is displayed on the screen along with a substitute line in which the new description is added.
  • It should be noted that while a convenient method of annotating video with the use of metadata has been presented above, the manner in which a search engine or other application is used to search, access and otherwise make use of the metadata is not restricted. Rather, the metadata may be queried in any way available to those of ordinary skill in the art.

Claims (21)

1. (canceled)
2. A computer-implemented method comprising:
providing a particular portion of a video for output;
receiving an utterance while the particular portion of the video is being provided for output;
obtaining, from an automated speech recognizer, a transcription of the utterance;
generating video metadata based on the transcription; and
updating a header of a frame of data that corresponds to the particular portion of the video, to include the video metadata.
3. The method of claim 1, wherein updating a header of a frame of data that corresponds to the particular portion of the video, to include the video metadata comprises:
updating the header of the frame to include a sequence header, a GOP header, user data, and an I-frame header.
4. The method of claim 1, wherein:
providing a particular portion of a video for output comprises:
providing, by a set-top box, the particular portion of the video for output,
receiving an utterance while the particular portion of the video is being provided for output comprises:
receiving, by the set-top box, the utterance while the particular portion of the video is being provided for output,
obtaining, from an automated speech recognizer, a transcription of the utterance comprises:
obtaining, from the automated speech recognizer on the set-top box, the transcription of the utterance,
generating video metadata based on the transcription comprises:
generating, by the set-top box, the video metadata based on the transcription, and
updating a header of a frame of data that corresponds to the particular portion of the video, to include the video metadata comprises:
updating, by the set-top box, the header of the frame of data that corresponds to the particular portion of the video, to include the video metadata.
5. The method of claim 1, wherein generating video metadata based on the transcription comprises:
generating the video metadata based on a particular video standard.
6. The method of claim 1, wherein updating a header of a frame of data that corresponds to the particular portion of the video, to include the video metadata comprises:
updating the header of the frame of data to include the video metadata and an associated time-stamp as user data bits.
7. The method of claim 1, comprising:
generating a caption or a subtitle based on the video metadata; and
storing the caption or the subtitle for display with the particular portion of the video.
8. The method of claim 1, comprising:
providing a predetermined user prompt,
wherein the utterance is received in response to the predetermined user prompt.
9. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
providing a particular portion of a video for output;
receiving an utterance while the particular portion of the video is being provided for output;
obtaining, from an automated speech recognizer, a transcription of the utterance;
generating video metadata based on the transcription; and
updating a header of a frame of data that corresponds to the particular portion of the video, to include the video metadata.
10. The system of claim 9, wherein updating a header of a frame of data that corresponds to the particular portion of the video, to include the video metadata comprises:
updating the header of the frame to include a sequence header, a GOP header, user data, and an I-frame header.
11. The system of claim 9, wherein:
providing a particular portion of a video for output comprises:
providing, by a set-top box, the particular portion of the video for output,
receiving an utterance while the particular portion of the video is being provided for output comprises:
receiving, by the set-top box, the utterance while the particular portion of the video is being provided for output,
obtaining, from an automated speech recognizer, a transcription of the utterance comprises:
obtaining, from the automated speech recognizer on the set-top box, the transcription of the utterance,
generating video metadata based on the transcription comprises:
generating, by the set-top box, the video metadata based on the transcription, and
updating a header of a frame of data that corresponds to the particular portion of the video, to include the video metadata comprises:
updating, by the set-top box, the header of the frame of data that corresponds to the particular portion of the video, to include the video metadata.
12. The system of claim 9, wherein generating video metadata based on the transcription comprises:
generating the video metadata based on a particular video standard.
13. The system of claim 9, wherein updating a header of a frame of data that corresponds to the particular portion of the video, to include the video metadata comprises:
updating the header of the frame of data to include the video metadata and an associated time-stamp as user data bits.
14. The system of claim 9, wherein the operations further comprise:
generating a caption or a subtitle based on the video metadata; and
storing the caption or the subtitle for display with the particular portion of the video.
15. The system of claim 9, wherein the operations further comprise:
providing a predetermined user prompt,
wherein the utterance is received in response to the predetermined user prompt.
16. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
providing a particular portion of a video for output;
receiving an utterance while the particular portion of the video is being provided for output;
obtaining, from an automated speech recognizer, a transcription of the utterance;
generating video metadata based on the transcription; and
updating a header of a frame of data that corresponds to the particular portion of the video, to include the video metadata.
17. The medium of claim 16, wherein updating a header of a frame of data that corresponds to the particular portion of the video, to include the video metadata comprises:
updating the header of the frame to include a sequence header, a GOP header, user data, and an I-frame header.
18. The medium of claim 16, wherein:
providing a particular portion of a video for output comprises:
providing, by a set-top box, the particular portion of the video for output,
receiving an utterance while the particular portion of the video is being provided for output comprises:
receiving, by the set-top box, the utterance while the particular portion of the video is being provided for output,
obtaining, from an automated speech recognizer, a transcription of the utterance comprises:
obtaining, from the automated speech recognizer on the set-top box, the transcription of the utterance,
generating video metadata based on the transcription comprises:
generating, by the set-top box, the video metadata based on the transcription, and
updating a header of a frame of data that corresponds to the particular portion of the video, to include the video metadata comprises:
updating, by the set-top box, the header of the frame of data that corresponds to the particular portion of the video, to include the video metadata.
19. The medium of claim 16, wherein generating video metadata based on the transcription comprises:
generating the video metadata based on a particular video standard.
20. The medium of claim 16, wherein the operations further comprise:
generating a caption or a subtitle based on the video metadata; and
storing the caption or the subtitle for display with the particular portion of the video.
21. The medium of claim 16, wherein the operations further comprise:
providing a predetermined user prompt,
wherein the utterance is received in response to the predetermined user prompt.
US14/336,063 2007-05-11 2014-07-21 Method and apparatus for annotating video content with metadata generated using speech recognition technology Abandoned US20140331137A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US14/336,063 US20140331137A1 (en) 2007-05-11 2014-07-21 Method and apparatus for annotating video content with metadata generated using speech recognition technology
US15/472,033 US10482168B2 (en) 2007-05-11 2017-03-28 Method and apparatus for annotating video content with metadata generated using speech recognition technology

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US11/747,584 US8316302B2 (en) 2007-05-11 2007-05-11 Method and apparatus for annotating video content with metadata generated using speech recognition technology
US13/654,327 US8793583B2 (en) 2007-05-11 2012-10-17 Method and apparatus for annotating video content with metadata generated using speech recognition technology
US14/336,063 US20140331137A1 (en) 2007-05-11 2014-07-21 Method and apparatus for annotating video content with metadata generated using speech recognition technology

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US13/654,327 Continuation US8793583B2 (en) 2007-05-11 2012-10-17 Method and apparatus for annotating video content with metadata generated using speech recognition technology

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/472,033 Continuation US10482168B2 (en) 2007-05-11 2017-03-28 Method and apparatus for annotating video content with metadata generated using speech recognition technology

Publications (1)

Publication Number Publication Date
US20140331137A1 true US20140331137A1 (en) 2014-11-06

Family

ID=39970326

Family Applications (4)

Application Number Title Priority Date Filing Date
US11/747,584 Active 2030-09-24 US8316302B2 (en) 2007-05-11 2007-05-11 Method and apparatus for annotating video content with metadata generated using speech recognition technology
US13/654,327 Active US8793583B2 (en) 2007-05-11 2012-10-17 Method and apparatus for annotating video content with metadata generated using speech recognition technology
US14/336,063 Abandoned US20140331137A1 (en) 2007-05-11 2014-07-21 Method and apparatus for annotating video content with metadata generated using speech recognition technology
US15/472,033 Active 2028-02-19 US10482168B2 (en) 2007-05-11 2017-03-28 Method and apparatus for annotating video content with metadata generated using speech recognition technology

Family Applications Before (2)

Application Number Title Priority Date Filing Date
US11/747,584 Active 2030-09-24 US8316302B2 (en) 2007-05-11 2007-05-11 Method and apparatus for annotating video content with metadata generated using speech recognition technology
US13/654,327 Active US8793583B2 (en) 2007-05-11 2012-10-17 Method and apparatus for annotating video content with metadata generated using speech recognition technology

Family Applications After (1)

Application Number Title Priority Date Filing Date
US15/472,033 Active 2028-02-19 US10482168B2 (en) 2007-05-11 2017-03-28 Method and apparatus for annotating video content with metadata generated using speech recognition technology

Country Status (4)

Country Link
US (4) US8316302B2 (en)
JP (1) JP5528324B2 (en)
KR (1) KR101115701B1 (en)
WO (1) WO2008140922A1 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10289915B1 (en) * 2018-06-05 2019-05-14 Eight Plus Ventures, LLC Manufacture of image inventories
US10565358B1 (en) 2019-09-16 2020-02-18 Eight Plus Ventures, LLC Image chain of title management
US10606888B2 (en) 2018-06-05 2020-03-31 Eight Plus Ventures, LLC Image inventory production
US10824699B2 (en) 2018-08-23 2020-11-03 Eight Plus Ventures, LLC Manufacture of secure printed image inventories
US10938568B2 (en) 2018-06-05 2021-03-02 Eight Plus Ventures, LLC Image inventory production
US20230095692A1 (en) * 2021-09-30 2023-03-30 Samsung Electronics Co., Ltd. Parallel metadata generation based on a window of overlapped frames

Families Citing this family (55)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9471333B2 (en) * 2006-11-03 2016-10-18 Conceptual Speech, Llc Contextual speech-recognition user-interface driven system and method
TWI423041B (en) * 2007-07-09 2014-01-11 Cyberlink Corp Av playing method capable of improving multimedia interactive mechanism and related apparatus
US10354689B2 (en) 2008-04-06 2019-07-16 Taser International, Inc. Systems and methods for event recorder logging
US8837901B2 (en) * 2008-04-06 2014-09-16 Taser International, Inc. Systems and methods for a recorder user interface
US20090251311A1 (en) * 2008-04-06 2009-10-08 Smith Patrick W Systems And Methods For Cooperative Stimulus Control
US20090307227A1 (en) * 2008-06-06 2009-12-10 Disney Enterprises, Inc. Methods and apparatuses for combining and distributing user enhanced video/audio content
US8892553B2 (en) * 2008-06-18 2014-11-18 Microsoft Corporation Auto-generation of events with annotation and indexing
US10574932B2 (en) * 2008-07-28 2020-02-25 Fox Digital Enterprises, Inc. System and method of generating subtitling for media
US9141859B2 (en) 2008-11-17 2015-09-22 Liveclips Llc Method and system for segmenting and transmitting on-demand live-action video in real-time
US9141860B2 (en) 2008-11-17 2015-09-22 Liveclips Llc Method and system for segmenting and transmitting on-demand live-action video in real-time
US9026668B2 (en) 2012-05-26 2015-05-05 Free Stream Media Corp. Real-time and retargeted advertising on multiple screens of a user watching television
US10419541B2 (en) 2008-11-26 2019-09-17 Free Stream Media Corp. Remotely control devices over a network without authentication or registration
US10977693B2 (en) 2008-11-26 2021-04-13 Free Stream Media Corp. Association of content identifier of audio-visual data with additional data through capture infrastructure
US8180891B1 (en) 2008-11-26 2012-05-15 Free Stream Media Corp. Discovery, access control, and communication with networked services from within a security sandbox
US10880340B2 (en) 2008-11-26 2020-12-29 Free Stream Media Corp. Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device
US10334324B2 (en) 2008-11-26 2019-06-25 Free Stream Media Corp. Relevant advertisement generation based on a user operating a client device communicatively coupled with a networked media device
US10567823B2 (en) 2008-11-26 2020-02-18 Free Stream Media Corp. Relevant advertisement generation based on a user operating a client device communicatively coupled with a networked media device
US9986279B2 (en) 2008-11-26 2018-05-29 Free Stream Media Corp. Discovery, access control, and communication with networked services
US9519772B2 (en) 2008-11-26 2016-12-13 Free Stream Media Corp. Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device
US9386356B2 (en) 2008-11-26 2016-07-05 Free Stream Media Corp. Targeting with television audience data across multiple screens
US9961388B2 (en) 2008-11-26 2018-05-01 David Harrison Exposure of public internet protocol addresses in an advertising exchange server to improve relevancy of advertisements
US9154942B2 (en) 2008-11-26 2015-10-06 Free Stream Media Corp. Zero configuration communication between a browser and a networked media device
US10631068B2 (en) 2008-11-26 2020-04-21 Free Stream Media Corp. Content exposure attribution based on renderings of related content across multiple devices
WO2010132718A2 (en) 2009-05-13 2010-11-18 Coincident.Tv , Inc. Playing and editing linked and annotated audiovisual works
US20100306232A1 (en) * 2009-05-28 2010-12-02 Harris Corporation Multimedia system providing database of shared text comment data indexed to video source data and related methods
US8887190B2 (en) * 2009-05-28 2014-11-11 Harris Corporation Multimedia system generating audio trigger markers synchronized with video source data and related methods
US9152139B2 (en) * 2009-06-16 2015-10-06 Control4 Corporation Automation Control of Electronic Devices
GB2472650A (en) * 2009-08-14 2011-02-16 All In The Technology Ltd Metadata tagging of moving and still image content
US8935204B2 (en) * 2009-08-14 2015-01-13 Aframe Media Services Limited Metadata tagging of moving and still image content
JP4930564B2 (en) * 2009-09-24 2012-05-16 カシオ計算機株式会社 Image display apparatus and method, and program
US8781824B2 (en) * 2010-12-31 2014-07-15 Eldon Technology Limited Offline generation of subtitles
US9342516B2 (en) * 2011-05-18 2016-05-17 Microsoft Technology Licensing, Llc Media presentation playback annotation
US20120308195A1 (en) * 2011-05-31 2012-12-06 Michael Bannan Feedback system and method
US9264471B2 (en) 2011-06-22 2016-02-16 Google Technology Holdings LLC Method and apparatus for segmenting media content
US9348554B2 (en) * 2011-12-20 2016-05-24 Audible, Inc. Managing playback of supplemental information
US20140129221A1 (en) * 2012-03-23 2014-05-08 Dwango Co., Ltd. Sound recognition device, non-transitory computer readable storage medium stored threreof sound recognition program, and sound recognition method
US20130283143A1 (en) 2012-04-24 2013-10-24 Eric David Petajan System for Annotating Media Content for Automatic Content Understanding
US9367745B2 (en) 2012-04-24 2016-06-14 Liveclips Llc System for annotating media content for automatic content understanding
CN103517092B (en) 2012-06-29 2018-01-30 腾讯科技(深圳)有限公司 A kind of method and device of video display
US20140123012A1 (en) * 2012-10-31 2014-05-01 Research In Motion Limited Video-annotation entry and display apparatus
US20140258472A1 (en) * 2013-03-06 2014-09-11 Cbs Interactive Inc. Video Annotation Navigation
CN104239354A (en) * 2013-06-20 2014-12-24 珠海扬智电子科技有限公司 Video and audio content evaluation sharing and playing methods and video and audio sharing system
KR20150087034A (en) 2014-01-21 2015-07-29 한국전자통신연구원 Object recognition apparatus using object-content sub information correlation and method therefor
KR101640317B1 (en) 2014-11-20 2016-07-19 소프트온넷(주) Apparatus and method for storing and searching image including audio and video data
WO2016118537A1 (en) * 2015-01-19 2016-07-28 Srinivas Rao Method and system for creating seamless narrated videos using real time streaming media
KR101742779B1 (en) 2015-05-12 2017-06-01 이석희 System for making dynamic digital image by voice recognition
US10592750B1 (en) * 2015-12-21 2020-03-17 Amazon Technlogies, Inc. Video rule engine
EP3438852B1 (en) 2016-07-21 2023-03-15 Samsung Electronics Co., Ltd. Electronic device and control method thereof
US11043221B2 (en) * 2017-04-24 2021-06-22 Iheartmedia Management Services, Inc. Transmission schedule analysis and display
US11263489B2 (en) * 2017-06-29 2022-03-01 Intel Corporation Techniques for dense video descriptions
US10390097B1 (en) * 2018-05-30 2019-08-20 Rovi Guides, Inc. Systems and methods for creating an asynchronous social watching experience among users
CN108960316B (en) * 2018-06-27 2020-10-30 北京字节跳动网络技术有限公司 Method and apparatus for generating a model
US10630738B1 (en) * 2018-09-28 2020-04-21 Ringcentral, Inc. Method and system for sharing annotated conferencing content among conference participants
WO2021246793A1 (en) * 2020-06-03 2021-12-09 주식회사 피제이팩토리 Method for generating multi-depth image
CN114827716B (en) * 2022-03-08 2023-08-11 深圳软牛科技有限公司 Method, device and related components for creating video player in WPF

Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5517652A (en) * 1990-05-30 1996-05-14 Hitachi, Ltd. Multi-media server for treating multi-media information and communication system empolying the multi-media server
US5880788A (en) * 1996-03-25 1999-03-09 Interval Research Corporation Automated synchronization of video image sequences to new soundtracks
US20010019657A1 (en) * 2000-03-01 2001-09-06 Mcgrath Mark John Audio and/or video generation apparatus and method of generating audio and/or video signals
US20020065074A1 (en) * 2000-10-23 2002-05-30 Sorin Cohn Methods, systems, and devices for wireless delivery, storage, and playback of multimedia content on mobile devices
US20020170062A1 (en) * 2001-05-14 2002-11-14 Chen Edward Y. Method for content-based non-linear control of multimedia playback
US20030093790A1 (en) * 2000-03-28 2003-05-15 Logan James D. Audio and video program recording, editing and playback systems using metadata
US6585521B1 (en) * 2001-12-21 2003-07-01 Hewlett-Packard Development Company, L.P. Video indexing based on viewers' behavior and emotion feedback
US6621979B1 (en) * 1998-06-11 2003-09-16 Koninklijke Philips Electronics N.V. Trick play signal generation for a digital video recorder using retrieved intra-encoded pictures and generated inter-encoded pictures
US20030231198A1 (en) * 2002-06-18 2003-12-18 Koninklijke Philips Electronics N.V. System and method for providing videomarks for a video program
US6961954B1 (en) * 1997-10-27 2005-11-01 The Mitre Corporation Automated segmentation, information extraction, summarization, and presentation of broadcast news
US7050110B1 (en) * 1999-10-29 2006-05-23 Intel Corporation Method and system for generating annotations video
US20060206327A1 (en) * 2005-02-21 2006-09-14 Marcus Hennecke Voice-controlled data system
US20070038999A1 (en) * 2003-07-28 2007-02-15 Rincon Networks, Inc. System and method for synchronizing operations among a plurality of independently clocked digital data processing devices
US20070250901A1 (en) * 2006-03-30 2007-10-25 Mcintire John P Method and apparatus for annotating media streams
US20080240490A1 (en) * 2007-03-30 2008-10-02 Microsoft Corporation Source authentication and usage tracking of video
US7640272B2 (en) * 2006-12-07 2009-12-29 Microsoft Corporation Using automated content analysis for audio/video content consumption

Family Cites Families (36)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5241619A (en) * 1991-06-25 1993-08-31 Bolt Beranek And Newman Inc. Word dependent N-best search method
US5621859A (en) * 1994-01-19 1997-04-15 Bbn Corporation Single tree method for grammar directed, very large vocabulary speech recognizer
US5721827A (en) * 1996-10-02 1998-02-24 James Logan System for electrically distributing personalized information
US6243713B1 (en) * 1998-08-24 2001-06-05 Excalibur Technologies Corp. Multimedia document retrieval by application of multimedia queries to a unified index of multimedia data for a plurality of multimedia data types
US7222163B1 (en) * 2000-04-07 2007-05-22 Virage, Inc. System and method for hosting of video content over a network
US7548565B2 (en) * 2000-07-24 2009-06-16 Vmark, Inc. Method and apparatus for fast metadata generation, delivery and access for live broadcast program
US20060064716A1 (en) * 2000-07-24 2006-03-23 Vivcom, Inc. Techniques for navigating multiple video streams
US7155517B1 (en) 2000-09-28 2006-12-26 Nokia Corporation System and method for communicating reference information via a wireless terminal
US7444660B2 (en) * 2000-11-16 2008-10-28 Meevee, Inc. System and method for generating metadata for video programming events
US7496767B2 (en) * 2001-01-19 2009-02-24 Xerox Corporation Secure content objects
US20020184195A1 (en) * 2001-05-30 2002-12-05 Qian Richard J. Integrating content from media sources
US7206806B2 (en) 2001-05-30 2007-04-17 Pineau Richard A Method and system for remote utilizing a mobile device to share data objects
US7970260B2 (en) * 2001-06-27 2011-06-28 Verizon Business Global Llc Digital media asset management system and method for supporting multiple users
US7133862B2 (en) * 2001-08-13 2006-11-07 Xerox Corporation System with user directed enrichment and import/export control
US20040237032A1 (en) * 2001-09-27 2004-11-25 David Miele Method and system for annotating audio/video data files
EP1313327A1 (en) * 2001-11-14 2003-05-21 Deutsche Thomson-Brandt Gmbh Digital video recorder and methods for digital recording
US7861169B2 (en) * 2001-11-19 2010-12-28 Ricoh Co. Ltd. Multimedia print driver dialog interfaces
EP1349080A1 (en) * 2002-03-26 2003-10-01 Deutsche Thomson-Brandt Gmbh Methods and apparatus for using metadata from different sources
AU2003239385A1 (en) * 2002-05-10 2003-11-11 Richard R. Reisman Method and apparatus for browsing using multiple coordinated device
GB2397406B (en) * 2002-07-23 2005-02-09 Samsung Electronics Co Ltd Index structure of metadata, method for providing indices of metadata, and metadata searching method and apparatus using the indices of metadata
AU2003253158A1 (en) * 2002-09-23 2004-04-08 Koninklijke Philips Electronics N.V. A video recorder unit and method of operation therefor
KR20050061545A (en) * 2002-10-18 2005-06-22 코닌클리케 필립스 일렉트로닉스 엔.브이. Method and system for metadata protection in tv-anytime
JP2004193979A (en) * 2002-12-11 2004-07-08 Canon Inc Video distribution system
KR101009629B1 (en) * 2003-03-13 2011-01-21 한국전자통신연구원 Extended Metadata Structure and Adaptive Program Service Providing System and Method for Providing Digital Broadcast Program Service
GB2399983A (en) * 2003-03-24 2004-09-29 Canon Kk Picture storage and retrieval system for telecommunication system
JP2005182460A (en) * 2003-12-19 2005-07-07 Canon Inc Information processor, annotation processing method, information processing program, and recording medium having information processing program stored therein
US20050154987A1 (en) * 2004-01-14 2005-07-14 Isao Otsuka System and method for recording and reproducing multimedia
KR100831531B1 (en) * 2004-01-14 2008-05-22 미쓰비시덴키 가부시키가이샤 Recording device, recording method, recording media, summarizing reproduction device, summarizing reproduction method, multimedia summarizing system, and multimedia summarizing method
EP1973348A3 (en) * 2004-09-17 2009-10-21 Korea Electronics Technology Institute Method for deleting user metadata managed by a tv-anytime metadata server using a SOAP operation
EP1677536A1 (en) * 2004-12-30 2006-07-05 Korea Electronics Technology Institute Method for delivering non-anonymous user metadata using a soap operation in TV-Anytime metadata service
US20070106685A1 (en) * 2005-11-09 2007-05-10 Podzinger Corp. Method and apparatus for updating speech recognition databases and reindexing audio and video content using the same
US8065710B2 (en) * 2006-03-02 2011-11-22 At& T Intellectual Property I, L.P. Apparatuses and methods for interactive communication concerning multimedia content
US8024762B2 (en) * 2006-06-13 2011-09-20 Time Warner Cable Inc. Methods and apparatus for providing virtual content over a network
WO2008031625A2 (en) * 2006-09-15 2008-03-20 Exbiblio B.V. Capture and display of annotations in paper and electronic documents
US20080276159A1 (en) * 2007-05-01 2008-11-06 International Business Machines Corporation Creating Annotated Recordings and Transcripts of Presentations Using a Mobile Device
US20110145068A1 (en) * 2007-09-17 2011-06-16 King Martin T Associating rendered advertisements with digital content

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5517652A (en) * 1990-05-30 1996-05-14 Hitachi, Ltd. Multi-media server for treating multi-media information and communication system empolying the multi-media server
US5880788A (en) * 1996-03-25 1999-03-09 Interval Research Corporation Automated synchronization of video image sequences to new soundtracks
US6961954B1 (en) * 1997-10-27 2005-11-01 The Mitre Corporation Automated segmentation, information extraction, summarization, and presentation of broadcast news
US6621979B1 (en) * 1998-06-11 2003-09-16 Koninklijke Philips Electronics N.V. Trick play signal generation for a digital video recorder using retrieved intra-encoded pictures and generated inter-encoded pictures
US7050110B1 (en) * 1999-10-29 2006-05-23 Intel Corporation Method and system for generating annotations video
US20010019657A1 (en) * 2000-03-01 2001-09-06 Mcgrath Mark John Audio and/or video generation apparatus and method of generating audio and/or video signals
US20030093790A1 (en) * 2000-03-28 2003-05-15 Logan James D. Audio and video program recording, editing and playback systems using metadata
US20020065074A1 (en) * 2000-10-23 2002-05-30 Sorin Cohn Methods, systems, and devices for wireless delivery, storage, and playback of multimedia content on mobile devices
US20020170062A1 (en) * 2001-05-14 2002-11-14 Chen Edward Y. Method for content-based non-linear control of multimedia playback
US6585521B1 (en) * 2001-12-21 2003-07-01 Hewlett-Packard Development Company, L.P. Video indexing based on viewers' behavior and emotion feedback
US20030231198A1 (en) * 2002-06-18 2003-12-18 Koninklijke Philips Electronics N.V. System and method for providing videomarks for a video program
US20070038999A1 (en) * 2003-07-28 2007-02-15 Rincon Networks, Inc. System and method for synchronizing operations among a plurality of independently clocked digital data processing devices
US20060206327A1 (en) * 2005-02-21 2006-09-14 Marcus Hennecke Voice-controlled data system
US20070250901A1 (en) * 2006-03-30 2007-10-25 Mcintire John P Method and apparatus for annotating media streams
US7640272B2 (en) * 2006-12-07 2009-12-29 Microsoft Corporation Using automated content analysis for audio/video content consumption
US20080240490A1 (en) * 2007-03-30 2008-10-02 Microsoft Corporation Source authentication and usage tracking of video

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11625432B2 (en) 2018-06-05 2023-04-11 Eight Plus Ventures, LLC Derivation of film libraries into NFTs based on image frames
US10938568B2 (en) 2018-06-05 2021-03-02 Eight Plus Ventures, LLC Image inventory production
US11755645B2 (en) 2018-06-05 2023-09-12 Eight Plus Ventures, LLC Converting film libraries into image frame NFTs for lead talent benefit
US10606888B2 (en) 2018-06-05 2020-03-31 Eight Plus Ventures, LLC Image inventory production
US11586671B2 (en) 2018-06-05 2023-02-21 Eight Plus Ventures, LLC Manufacture of NFTs from film libraries
US11755646B2 (en) 2018-06-05 2023-09-12 Eight Plus Ventures, LLC NFT inventory production including metadata about a represented geographic location
WO2019236661A1 (en) * 2018-06-05 2019-12-12 Eight Plus Ventures, LLC Manufacture of image inventories
US11586670B2 (en) 2018-06-05 2023-02-21 Eight Plus Ventures, LLC NFT production from feature films for economic immortality on the blockchain
US10289915B1 (en) * 2018-06-05 2019-05-14 Eight Plus Ventures, LLC Manufacture of image inventories
US11609950B2 (en) 2018-06-05 2023-03-21 Eight Plus Ventures, LLC NFT production from feature films including spoken lines
US11625431B2 (en) 2018-06-05 2023-04-11 Eight Plus Ventures, LLC NFTS of images with provenance and chain of title
US10824699B2 (en) 2018-08-23 2020-11-03 Eight Plus Ventures, LLC Manufacture of secure printed image inventories
US10860695B1 (en) 2019-09-16 2020-12-08 Eight Plus Ventures, LLC Image chain of title management
US10565358B1 (en) 2019-09-16 2020-02-18 Eight Plus Ventures, LLC Image chain of title management
US20230095692A1 (en) * 2021-09-30 2023-03-30 Samsung Electronics Co., Ltd. Parallel metadata generation based on a window of overlapped frames
US11930189B2 (en) * 2021-09-30 2024-03-12 Samsung Electronics Co., Ltd. Parallel metadata generation based on a window of overlapped frames

Also Published As

Publication number Publication date
WO2008140922A1 (en) 2008-11-20
JP5528324B2 (en) 2014-06-25
US20130041664A1 (en) 2013-02-14
KR101115701B1 (en) 2012-03-06
KR20090125292A (en) 2009-12-04
US20170199856A1 (en) 2017-07-13
JP2010525497A (en) 2010-07-22
US8316302B2 (en) 2012-11-20
US10482168B2 (en) 2019-11-19
US8793583B2 (en) 2014-07-29
US20080281592A1 (en) 2008-11-13

Similar Documents

Publication Publication Date Title
US10482168B2 (en) Method and apparatus for annotating video content with metadata generated using speech recognition technology
US10034028B2 (en) Caption and/or metadata synchronization for replay of previously or simultaneously recorded live programs
KR100547317B1 (en) Simultaneous recording and playback apparatus with indexing/searching/browsing functionality
CA2572709C (en) Navigating recorded video using closed captioning
US20070214488A1 (en) Method and system for managing information on a video recording device
JP2009065680A (en) System and method for providing multimedia summary of video program
JP2004516752A (en) System and method for accessing multimedia summaries of video programs
US20070154176A1 (en) Navigating recorded video using captioning, dialogue and sound effects
US20130089300A1 (en) Method and Apparatus for Providing Voice Metadata
JP2007052626A (en) Metadata input device and content processor
US6931201B2 (en) Video indexing using high quality sound
US20230229702A1 (en) Methods and systems for providing searchable media content and for searching within media content
JP2006115052A (en) Content retrieval device and its input device, content retrieval system, content retrieval method, program and recording medium
KR20050041797A (en) Storage medium including meta data for enhanced search and subtitle data and display playback device thereof
KR20080112975A (en) Method, system and recording medium storing a computer program for building moving picture search database and method for searching moving picture using the same
EP1698174A1 (en) Method and circuit for creating a multimedia summary of a stream of audiovisual data
KR101401974B1 (en) Method and apparatus for browsing recorded news programs
US20160127807A1 (en) Dynamically determined audiovisual content guidebook
US8000584B1 (en) Approach for storing digital content onto digital versatile discs (DVDs)
Gibbon et al. Video Data Sources and Applications

Legal Events

Date Code Title Description
AS Assignment

Owner name: GENERAL INSTRUMENT CORPORATION, PENNSYLVANIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MCKOEN, KEVIN M.;GROSSMAN, MICHAEL A.;SIGNING DATES FROM 20070201 TO 20070205;REEL/FRAME:033593/0195

Owner name: MOTOROLA MOBILITY LLC, ILLINOIS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:GENERAL INSTRUMENT HOLDINGS, INC.;REEL/FRAME:033593/0425

Effective date: 20130528

Owner name: GENERAL INSTRUMENT HOLDINGS, INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:GENERAL INSTRUMENT CORPORATION;REEL/FRAME:033593/0361

Effective date: 20130415

AS Assignment

Owner name: GOOGLE TECHNOLOGY HOLDINGS LLC, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MOTOROLA MOBILITY LLC;REEL/FRAME:034718/0001

Effective date: 20141028

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION