ENHANCEMENT OF BROADCAST SIGNALS THROUGH
AUTOMATIC SIGNAL DETECTION AND PROVIDING
OF SERVICES THROUGH DATA NETWORKS AND DEVICES
Field of the Invention
The invention relates to the field of providing services that enhance broadcast media content.
Background of the Invention Broadcast radio and TV are ubiquitous features of modern day life and reach populations worldwide. The broadcast infrastructure is mature and sufficiently efficient that radio and TV broadcasts over radio frequencies (Rf) are free to end users, subsidized only by advertising content. Cable and satellite systems have evolved to expand the number of channels available to users, with "basic" services priced by the connection, not by the number of channels, and "premium" services priced at a very nominal rate per user with respect to the original production costs. Broadcast systems are extremely efficient in providing rich and varied content to large numbers of people. The key to this efficiency is the delivery of identical content in a one-way direction to masses of users.
Recently, the value of broadcast systems has been enhanced by making such systems interactive. The simplest way to accomplish such interactivity is to broadcast a phone number or web URL that the recipient manually enters into a telephone or PC web browser. Such systems require the user to write down or remember the phone number or URL and manually enter it into a separate device. Consequently, such systems have obvious drawbacks such as lack of ease and accuracy. Some such systems also log the time of the broadcast and permit later recall and use of the content that was broadcast at that time. Other systems improve on this approach by adding encoded data to the broadcast media that is not perceptible to human viewers or listeners. These systems include digital radio and systems that insert data in the vertical blanking interval of a video signal or in the video signal itself. A modified receiver or a separate device detects the inserted data and permits the user to access a new service or communication channel. These systems, however, require that the broadcast content be modified to include additional data, that new devices be provided to users, and also require a new service infrastructure.
Other broadcasters produce a parallel path by manually creating data prior to the broadcast that is transmitted via a different channel and allows interaction with the broadcast program. For example, MTV provides a timed interactive experience to PC users in their broadcast WebRiot TV show and PC simulcast. Users are allowed to play along with the broadcast TV show and have their responses feed back into the show. This permits a broadcaster to enhance its broadcast through manual creation of synchronized content. This content, including any embedded advertising or links, is owned and managed by the owner of the broadcast content.
Following a similar strategy, broadcasters may create new digital channels designed to provide fully integrated interactivity. Interactive TV (ITV) systems offer viewers the ability to receive additional content to the broadcast signal or to respond to the broadcast signal to purchase or consult with, for example, a product manufacturer. For example, an interactive TV signal with a set-top box may allow receipt and transmission of user input and display of the interactive information. This concept is being deployed as part of early ITV systems, such as the BskyB system in England. A clear disadvantage of the system is the requirement to create embedded enhancements that alter content and to deploy new hardware (i.e., set-top boxes) at each user location. This technique also requires a new broadcast infrastructure, and therefore can only be realized with the aid or cooperation of the broadcaster.
The above-described systems have one or more of the following five limitations: 1) they require modification to broadcast content, 2) they can only be realized by the broadcaster or owner of the broadcast content, 3) they require new broadcast equipment and/or customer premises equipment, 4) they require a new response network and infrastructure, and 5) they are created prior to broadcast and hence cannot create enhancements in real time during an event such as a sporting event or news broadcast. These limitations either delay the roll out of broadcast interactivity or increase it's cost due to lack of competition. These restrictions are unfortunate due to the low cost and increasing availability of separate interactive devices and communications channels such as those provided by telephone and the Internet.
Summary of the Invention The system disclosed herein utilizes existing broadcast signals and existing interactive channels to provide interactive enhancement and response to broadcast signals without requiring any modification to broadcast content, permission from the broadcaster or other content provider, or new equipment at the broadcast or user locations. Hence, it can be deployed quickly, inexpensively, and by any third party.
The disclosed system processes a broadcast signal to identify its content and subsequently delivers an interactive service that enhances the utility of the broadcast signal. Electronic and computer-implemented techniques for identifying the content of known audio or video signals are well known in the industry. Alternatively, a person may be given the task of identifying the content of a broadcast signal. The processing is preferably done by a processing unit that can compare known content to the content of a broadcast signal or to a portion of such broadcast content such as the specific occurrence of a spoken word or image. In accordance with one aspect of the invention, the broadcast of a recorded song, video, or advertisement can be detected. In another aspect, digital data such as closed caption text in a video image can be identified, and used to identify the image. In accordance with another aspect of the invention, a keyword such as a product name or an image such as a company logo can be identified.
Once a broadcast signal (or a portion thereof) is identified, an interactive application server produces a service associated in some way with the broadcast content that is accessible over a separate and distinct communication channel from the one used to carry the broadcast signal. The system thus uses some aspect of a broadcast signal's content to create a service related to the signal's content. In accordance with one embodiment, this service can be a standard web service that is accessible via a web browser. In another embodiment, the service can be accessible by a wireless device using an application interface such as the
Wireless Application Protocol (WAP). In yet another embodiment, the service could be accessed through a wired or cellular telephone via the Public Switched Telephone Network. Alternatively, a special purpose device may be used to access the enhanced services. In one embodiment, such a special purpose device may take the form of a TV remote control with a wireless local area or wide area network connection to a network server.
The service that is provided over these channels enhances the receiver's experience of the broadcast channel. It also enables commerce associated with the stimuli of the broadcast
content by providing ease of response to offers to buy and direct responses to broadcast content. For example, the user may receive more information about a concert associated -with a broadcast song, buy the CD that contains the song, or respond to a voting process to choose a favorite artist. The service may be associated with the broadcaster or content owner (e.g., a radio ad) but need not be. For example, a record distribution company may allow users to buy CDs that contain the song being broadcast on a radio station using a WAP phone which accesses a site based on the broadcast channel identification.
Brief Description of the Drawings These and other aspects of the invention will be better appreciated when taken in conjunction with the detailed description and accompanying drawings in which:
Fig. 1 is a block diagram of a preferred embodiment of a system for enhancing broadcast content;
Fig. 2 is a block diagram of a preferred embodiment of processing system 103 shown in Fig. 1 and its relationship to other system components;
Fig. 3 is a block diagram of a preferred embodiment of audio processor 204 shown in Fig. 2 and its relationship to other system components;
Fig. 4 is a flowchart depicting a preferred embodiment of the steps performed by the system to create a database of characteristics that characterize the content of a plurality of predetermined signals;
Fig. 5 is a flowchart depicting a preferred embodiment of the steps performed by the system to provide broadcast-enhancing services to a user;
Figs. 6 A-C illustrate a preferred embodiment of one service that may be provided via a digital phone handset; and Fig. 7 is a diagram of a preferred embodiment of a special purpose device adapted to receive services provided by processing system 103.
Detailed Description of the Preferred Embodiments A preferred embodiment of a system for enhancing broadcast signals is shown in Fig. 1. As shown in Fig. 1 , the system comprises a broadcast facility 100 comprising an antenna
106 adapted to broadcast radio signals (e.g., AM or FM radio signals) via an Rf broadcast network 101 to a plurality of radios such as a radio 102.
Radio 102 is preferably in possession of a user who has tuned the radio to the broadcast signal's frequency. The user also preferably has in his or her possession a digital cellular phone 105 with interactive capabilities such as Short Message Service (SMS) or Wireless Application Protocol (WAP). This enables the user to transmit and receive information via a wireless data network 104 (e.g., the GSM Digital cellular network). Digital cellphonel05 preferably comprises a digital display that is used to display Wireless Markup Language (WML) data passed from the server and a series of scroll and select buttons that allow the user to scroll down and select items from a menu. A keypad is also provided to allow the user to enter numbers and alphanumeric characters as part of the WML service. The keypad is also used as a normal phone keypad when the phone is used for radio communication.
Also shown in Fig. 1 is a processing system 103 that receives the same broadcast radio signal as radio 102. As described in more detail below, processing system 103 is adapted to process the received audio stream to identify its content. For example, processing system 103 may be adapted to identify particular songs, advertisements, or keywords that are broadcast by broadcast location 100.
For each identified type of information, processing system 103 creates conesponding digital information or services to enhance the broadcast information or provide commerce or other interaction opportunities. These interactive or information services may then be delivered to the user of digital cell phone 105 via digital cellular network 104. No modifications or enhancements are required to the broadcast content, the Rf broadcast network, equipment 102 and 105, or wireless data network 104.
Fig. 2 is a block diagram of a preferred embodiment of processing system 103 and its relationship to other system components. As shown in Fig. 2, processing system 103 preferably comprises a radio receiver 203 for receiving a broadcast radio signal. Radio receiver 203 demodulates the received broadcast signal and presents the audio stream, like that heard by the user, to an audio processor 204.
Audio processor 204 is adapted to compare the received audio stream to a library of audio signals stored in a database 205. Audio processor 204 is also connected to an output database 206 and a web server 207. Web server 207 connects to an IP Network 208 which in turn connects to wireless data network 104 via a wireless gateway 209. Web server 207 is thus able to transmit data to and receive data from digital cell phone 105.
Fig. 3 is a block diagram of a preferred embodiment of audio processor 204 and its relationship to other system components. As shown in Fig. 3, audio processor 204 preferably comprises a preprocessor 302 that is connected to database 205 and processes the stored library of recorded signals to extract parameters from or create transforms of the signals so that they can be compared to received broadcast signals. The output of preprocessor 302 is stored in a database 303.
Audio processor 204 further comprises a preprocessor 306 for processing received broadcast radio signals. Preprocessor 306 and database 303 are both connected to a conelator or comparator 307 that is adapted to compare the processed received signals to the processed signals stored in database 303. The output of correlator 307 is provided to a detector 308 that determines whether the received radio signal matches any of the stored signals. Detector 308 is connected to a detected signal database 206 and web server 207. A plurality of parallel audio processors 204 may be used to process multiple channels broadcast in a particular geographic area or over a particular cable broadcast system. In an alternative embodiment, audio processor 204 may comprise a voice recognition processor adapted to identify a set of ke words or phrases embedded in the content.
Operation of a prefened embodiment of the present invention will now be described in connection with the flowcharts of Figs. 4 and 5. The flowchart of Fig. 4 depicts steps performed by the system to create database 303. The flowchart of Fig. 5 depicts steps performed by the system to provide a broadcast-enhancing service to a user.
Beginning with Fig. 4, in step 402, a library of recorded signals is stored in database 301. These signals may include: copies of songs that may be broadcast, advertisements, or key words such as "New York Mets." In step 404, preprocessor 302 retrieves a recorded signal and processes it to extract parameters or generate signal transforms to facilitate comparison of the stored signals to received broadcast signals, as described below. In step
406, preprocessor 302 deposits the extracted data in database 303. Each record in database 303 preferably comprises the type of signal (e.g., advertisement, key word, etc.), an identifier of the specific signal, and the extracted parameters or transforms. For example, the record might contain the Fourier transform of the signal. Alternatively, it might contain digital data such as closed caption text or digital radio sidebands embedded in the signal.
Steps 404, 406 are repeated for each signal in the database. In a preferred embodiment, database 303 is created before commencement of system operation and may be
updated periodically (using the process steps of Fig. 4) when, for example, new tunes or new advertising messages are created.
Turning now to Fig. 5, in step 502, radio 102 receives a broadcast signal from network 101 and plays the audio content of the radio signal on its speaker. Concurrently, in step 504, receiver 203 of processing system 103 receives the same broadcast radio signal and forwards it to audio processor 204.
In step 506, the received signal is archived for quality control and replay purposes in recorded signal database 305 of audio processor 204 using conventional audio or digital radio techniques. In step 508, the received signal is processed by preprocessor 306 to prepare the signal for identification processing. This processing may include, for example, noise removal or a Fourier Transform.
In step 510, correlator 307 compares the processed received signal to all candidate signals stored in database 303. The output of this comparison for each signal candidate is sent to detector 308. In step 512, detector 308 determines whether the received signal matches any candidate signal.
If a match is found, the system proceeds to step 514 where detector 308 creates a database record that includes time, broadcast station, the identified signal, and a recording of part of the received signal for quality control purposes. Detector 308 stores the record in output database 206. In step 516, detector 308 also informs web server 311 that the signal has been detected and of the presence of the newly created record and it's identity in output database 206. As known, in the art, although shown as a single component, element 207 may comprise several components including a web server, application server, and database server.
In step 518, web server 207 creates a service comprising content that is correlated to the identified signal. The content may be formatted using a protocol such as Wireless Mark Up Language (WML) or as a short message service (SMS) message that is intended for services utilized with a digital cellular phone. Digital cell phone 105 may use an existing second generation (2G) digital network such as the GSM, TDMA, or CDMA networks, an enhanced second generation digital network (2.5 G), or a third generation (3G) network. In creating aspects of the service, web server 207 may use standard and well known procedures and capabilities to provide an e-commerce service component, including authentication, security, billing, and linkage to other services in the Internet for advertising or order
fulfillment. Web server 207 may create such service content after a received broadcast signal has been matched to a candidate signal, or alternatively may have pre-stored services content for certain candidate signals that need only be retrieved from memory when a match is detected. In step 520, digital cell phone 105 displays a number of service options to the user.
As shown, for example, in Fig. 6A, the user may be offered weather information, sports information, stock market information, and broadcast-enhancing information. In the example of Fig. 6A, this fourth option is designated by the name radwap (TM) and the user has saved the server WAP address of this service as one of his or her favorite services. The keypad may be used to scroll to the radwap service and then select it.
In step 522, if the user chooses the radwap service, web server 207 causes digital cell phone 105 to prompt the user to identify the radio station that he or she is listening to. Ih step 524, the user enters an identifier for the radio station he or she is listening to. For example, if the listener is listening to 101.1 FM, the user might enter 1011, as illustrated in Fig. 6B. The cellular system can be interrogated for the location (e.g., city) of digital cell phone 105 so that web server 207 can determine the identity of the radio station from its frequency.
In step 526, web server 207 transmits the created content or service to the user's digital cell phone 105 via wireless gateway 209. For example, as shown in Fig. 6C, web server 207 may cause to be displayed on digital cell phone 105 the name of the song currently being played and several options relating to the song. These may, for example, include an option to buy a CD that contains the song, to save the song, to download the song, or to win a prize in a contest related to the song. Selection of any option may lead to additional screens where information such as user passwords or credit card information may be entered prior to CD purchase. If the user chooses to download the song, web server 207 may provide the user with several options relating to the download. For example, the user may be permitted to specify that he or she wishes to save the song to a device (e.g., an MP3 player) located either in digital cell phone 105 or another location (e.g., the user's home PC). Alternatively, the user may be permitted to download the song to digital cell phone 105 for use as the phone's ring tone. As will be apparent, the particular preferred embodiment described above is for a system that enhances broadcast radio signals by providing the listener with advanced services via his or her cell phone that relate to the radio broadcast. It will be recognized, however that
the techniques described in the above preferred embodiment may be applied to other environments and components.
For example, the disclosed techniques may be applied to broadcast, cable, or satellite video signals where similar audio processing may be used or where video imaging processing, processing of an embedded digital signal, or text processing of closed caption text may also be used. In addition to a digital cellular network, the disclosed techniques may also be implemented using a wired internet connection, a wireless Local Area Network, a satellite communications network, or other two-way digital radio network. Alternative devices and associated IP protocols may be used with the disclosed techniques such as a voice phone with Voice Extensible Mark Up Language (VXML), a PC with Hypertext Mark Up Language
(HTML), a Personnel Digital Assistant, Internet appliance or two-way pager with HTML, Tiny HTML (THTML), Web Clipping, or other IP based service protocols, or hybrid PDA/cell phones communicating over a packet network. The disclosed techniques can also be applied to a series of services. Merchandise related to the broadcast content such as digital or analog tape video recordings, posters, or group clothing may also be sold. Contests or gaming may be associated with the broadcast channel.
In a preferred embodiment, in response to a broadcast advertisement a user may request additional information concerning the advertised item (via e-mail or mail), purchase the product, subscription, or service being advertised, or request that a representative contact the user via digital cell phone 105 or an alternate telephone. Identification of the content
(e.g., keywords or images in the broadcast) may also be used to create hypertext links to information or commerce provided by a third party. In a further preferred embodiment, during broadcast of a sporting event, web server 207 may transmit to digital cell phone 105 information concerning players or teams participating in the sporting event. In a preferred embodiment, the user may specify to web server 207 that he or she wishes digital information to be transmitted to a second device (i.e., a device other than digital cell phone 105). For example, if the broadcast signal included a URL for a company providing an advertised product or service, the user may request that web server 207 transmit the URL via e-mail to the user's home PC. More generally, services provided by processing system 103 may include transmission of data to digital cell phone 105 or elsewhere. The transmitted data may include images, phone numbers, bookmarks to be saved by, e.g., digital
cell phone 105 for later viewing, programming listings for upcoming television broadcasts on a channel being viewed by the user or other channels, or other data.
In a preferred embodiment, the disclosed techniques may be applied to provide enhanced television services to a viewer via a television remote control unit adapted to provide Internet connectivity. A preferred embodiment of such a unit is shown in Fig. 7.
As shown in Fig. 7, a remote control unit 700 comprises a keypad 702. Keypad 702 preferably comprises a plurality of buttons to allow a user to enter commands such as channel control (704) and volume control (706). In addition, keypad 702 preferably comprises additional buttons for controlling TV on/off and a numeric keypad comprising buttons 0 through 9.
Unit 700 is further preferably provided with an infrared transmitter element for communicating commands entered by the user to a TV or set-top box. Unit 700 also preferably comprises a display 710 for displaying information to the user including services information supplied by web server 207, as described below. Keypad 702 may also be used to communicate messages from the user to web server 207.
During operation of this preferred embodiment, processing system 103 may be adapted to monitor television signals broadcast by television stations and to generate services relating to those broadcast transmissions, in a manner analogous to that described above in connection with steps 502-516 of Fig. 5. Processing system 103 may then provide these services to the user via unit 700, in a manner analogous to that described above in connection with steps 518-524 of Fig. 5.
In an alternative prefened embodiment, the above embodiment may be implemented by programming a PDA comprising an infrared transmitter and wireless data transmission capabilities to act as a television remote control and communicate with web server 207 as described above. In this preferred embodiment, buttons for controlling the user's TV and for transmitting user messages to web server 207 may be implemented as "soft" buttons displayed on the PDA's display screen.
In addition, unit 700 may be adapted to transmit a data message to web server 207 each time the user changes the channel that he or she is watching. Web server 207 may then automatically modify the enhanced services that it is transmitting to the user from those associated with the channel that the user was watching, to those associated with the channel that the user is now watching.
In a preferred embodiment, web server 207 may also be adapted to provide a service that informs the user of content that matches the user's preferences on another broadcast • channel. For example, the user may specify in his or her profile that he or she likes Beatles' songs. When processing system 103 detects broadcast of a Beatles' song on another station, web server 207 may transmit a data message to the user including the frequency of the station playing the song. Alternately, a user may be watching TV news and be informed that the weather or sports is currently being broadcast on another station. The system may also be used for content monitoring to verify and log the broadcast of particular media for verification, statistical analyses, or campaign-effectiveness measurement. It will be appreciated that, although the invention has been described in the context of specific illustrative embodiments, those skilled in the art will be able to devise numerous alternative arrangements which, although not explicitly shown or described herein, nevertheless embody the principles of the invention.