US20020054242A1

US20020054242A1 - Information processing apparatus, information processing method, and distribution media

Info

Publication number: US20020054242A1
Application number: US10/022,419
Authority: US
Inventors: Taro Suito; Masashi Ohta; Katsunari Miyata
Original assignee: Sony Corp
Current assignee: Sony Corp
Priority date: 1998-11-30
Filing date: 2001-12-20
Publication date: 2002-05-09
Also published as: TW465239B; JP2000165806A; US6459735B1; CN1256588A; EP1006685A2; KR20000035766A

Abstract

A commercial candidate section detector detects a commercial candidate section on the basis of a quiet section and a scene change point. A commercial characteristic quantity detector judges whether the commercial candidate section has various characteristics of commercials, and adds a predetermined value to an commercial characteristic value on the basis of the judgment result. The commercial characteristic quantity detector compares the final commercial characteristic value with a predetermined threshold value, and judges on the basis of the comparison result whether the commercial candidate section is a commercial section.

Description

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is related to U.S. Patent Application Numbers ______ and ______, both of which are filed on the same date as this application. These applications correspond with Attorney Docket Nos. SONY-Q9137 and SONY-Q9138. Both applications are titled INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND DISTRIBUTION MEDIA, both have the same inventors as this application and both are assigned to the assignee of the present invention. Both of said applications are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an information processing device and method, and an information supply medium, and particularly to an information processing device and method and an information supply medium for detecting commercials contained in a television broadcast.

2. Description of the Related Art

When a recorded television broadcast is reproduced, in order to satisfy users' needs for continuously watching only a program, there has been provided a video recorder (e.g., a video cassette recorder or “VCR”) having a so-called commercial cut function which fast feeds a commercial advertisement (hereinafter referred to as a “commercial” or a “CM”).

A commercial detection algorithm used in such a video recorder is based on the following characteristics which are unique to commercials or characteristic of a number of commercials. That is, if the following characteristics are confirmed, the portion thus confirmed is detected as a commercial: a quiet section of 0.1 to 2.0 seconds exists at the start and end times of each commercial; a scene change point of an image exists at each quiet section; the duration of each commercial is equal to an integral multiple of 15 seconds; and (for commercials broadcast outside of the United States, e.g., in Japan) the sound multiplex mode changes from a monaural mode for programs to a stereo mode for commercials.

In television broadcast systems of America and Europe, commercials may be detected on the basis of a black frame or blue frame existing between a program and a commercial.

Therefore, according to a conventional commercial detection algorithm, any commercial in which at least one of the above characteristics is not present, such as a Japanese commercial having a monaural sound multiplex mode, cannot be detected.

Further, according to the conventional commercial detection algorithm, if the above characteristics are contained in a part of a program, that portion of the program is erroneously detected as a commercial.

If the characteristics of commercials which have been used in the conventional algorithm were altered or abolished (for example, if the duration of commercials were changed to an integral multiple of 14 seconds or if inserting of the black frame or blue frame were abolished in America or Europe), no commercial would be detected.

Furthermore, according to the detection method of the quiet section in the conventional commercial detection algorithm, the average level of sound over a section is calculated, and if the average level thus calculated is below a predetermined threshold value, the section concerned is detected as a quiet section. Therefore, the detection precision for the quiet section is different between a case of bad reception of a television broadcast (e.g., if the signal to noise ratio (“S/N”) of sound signals is low due to a weak electric field) and a case where the electric field is not weak, so that no commercial can be reliably detected.

The measurement of the time required for a commercial in the conventional commercial detection algorithm is performed by counting the number of frames, while the number of frames per second is set to about 30 frames. However, since an error of several frames often occurs in the actual broadcast, a margin of error is provided to the threshold value to identify the needed time in connection with the error. Accordingly, erroneous detection or non-detection of commercials occurs due to the margin of error.

Further, when a commercial spot of a TV station is broadcast between a commercial and a program for a short time, e.g., for 5 seconds, the spot is something like a commercial from the perspective of viewers. However, the spot is not detected because it does not have the foregoing characteristics of commercials.

SUMMARY OF THE INVENTION

It is an object of the invention to provide a method, apparatus and computer program for correctly detecting commercials contained in television broadcasts.

It is a further object of the present invention to reduce or eliminate the need to view commercials when reproducing previously recorded television broadcasts.

It is a feature of the present invention to stop recording of a television broadcast when a commercial is detected.

It is an additional feature of the present invention to instead index commercials when recording a television broadcast in order to permit the high speed fast forwarding through such commercials, yet preserve the ability to view such commercials according to the wishes of a viewer.

It is an advantage of the present invention to reduce the amount of time a viewer must spend in viewing a desired television program.

It is a further advantage of the present invention to reduce the amount of recording media storage space necessary to record a desired television program, by not recording commercials interspersed with the desired television program.

It is yet another advantage of the present invention to allow a viewer to index commercials to permit the viewing of such commercials when desired by the viewer.

According to one aspect of the invention, there is provided an apparatus for processing a television signal which includes: means for receiving a television signal; means for detecting a commercial candidate section in the television signal; means for determining characteristics of the commercial candidate section; and means for judging whether or not the commercial candidate section is a commercial, based on the characteristics detected by the determining means.

According to another aspect of the invention, there is provided an apparatus for processing a television signal which includes: means for receiving a television signal; means for detecting a commercial candidate section in the television signal; means for determining a frequency of scene changes in the commercial candidate section and for increasing a commercial characteristic value if the frequency of scene changes is above a predetermined level; means for determining whether or not the audio signal in the commercial candidate section has a periodicity, for calculating a level of periodicity detected and for increasing a commercial characteristic value if the calculated periodicity is above a predetermined level; means for determining a level of continuity of the periodicity of the audio signal in the commercial candidate section and for increasing a commercial characteristic value if the continuity is above a predetermined level; means for determining a level of repetition of the video signal in the commercial candidate section and for increasing a commercial characteristic value if the repetition is above a predetermined level; means for detecting a telop in the video signal of the commercial candidate section and for increasing a commercial characteristic value if a telop is detected; means for detecting a character in the video signal of the commercial candidate section and for increasing a commercial characteristic value if a character is detected; means for determining the quietness rate of the commercial candidate section and for increasing a commercial characteristic value if the quietness rate is below a predetermined level; and means for judging whether or not the commercial candidate section is a commercial, based on the commercial characteristic value resulting from the operations of the frequency determining means, the periodicity determining means, the continuity determining means, the repetition determining means, the telop determining means, the character determining means and the quietness rate determining means.

According to another aspect of the invention, there is provided an apparatus for processing a television signal which includes: means for receiving the television signal; means for determining a quiet threshold value from the television signal; means for making a comparison of the audio signal and the quiet threshold value; and means for judging whether or not a portion of the television signal is a commercial candidate section according to the comparison.

According to another aspect of the invention, there is provided an apparatus for processing a television signal which includes: a receiver for receiving a television signal; a first detector for detecting a commercial candidate section in the television signal; a second detector for detecting characteristics of the commercial candidate section; and a judgment circuit for judging whether or not the commercial candidate section is a commercial, based on the characteristics detected by the second detector.

According to another aspect of the invention, there is provided an apparatus for processing a television signal which includes: a receiver for receiving a television signal; a commercial candidate section detecting circuit for detecting a commercial candidate section in the television signal; a commercial characteristic value counter for storing a previous commercial characteristic value, for receiving a commercial characteristic value signal and for adding a commercial characteristic value from the commercial characteristic value signal to the previous commercial characteristic value; a frequency determining circuit for determining a frequency of scene changes in the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if the frequency of scene changes is above a predetermined level; a periodicity determining circuit for detecting whether or not the audio signal in the commercial candidate section has a periodicity, for calculating a level of periodicity detected and for sending a commercial characteristic value signal to the commercial characteristic value counter if the calculated periodicity is above a predetermined level; a continuity determining circuit for determining a level of continuity of the periodicity of the audio signal in the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if the continuity is above a predetermined level; a repetition determining circuit for determining a level of repetition of the video signal in the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if the repetition is above a predetermined level; a telop detecting circuit for detecting a telop in the video signal of the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if a telop exists; a character detecting circuit for detecting a character in the video signal of the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if a character is detected; a quietness rate determining circuit for determining the quietness rate of the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if the quietness rate is below a predetermined level; and a judgement circuit for judging whether or not the commercial candidate section is a commercial, based on the commercial characteristic value in the commercial characteristic value counter.

According to another aspect of the invention, there is provided an apparatus for processing a television signal which includes: a receiver for receiving the television signal; a quiet threshold value determining circuit for determining a quiet threshold value from the television signal; a comparitor for making a comparison of the audio portion of the television signal and the quiet threshold value; and a judging circuit for judging whether or not a portion of the television signal is a commercial candidate section according to the comparison.

According to another aspect of the invention, there is provided a method for processing a television signal which includes the following steps: receiving a television signal; detecting a commercial candidate section in the television signal; determining characteristics of the commercial candidate section; and judging whether or not the commercial candidate section is a commercial, based on the characteristics detected in the determining step.

According to another aspect of the invention, there is provided a method for processing a television signal which includes the following steps: receiving a television signal; detecting a commercial candidate section in the television signal; determining a frequency of scene changes in the commercial candidate section and increasing a commercial characteristic value if the frequency of scene changes is above a predetermined level; determining a periodicity of the audio signal in the commercial candidate section and increasing a commercial characteristic value if the periodicity is above a predetermined level; determining a continuity of a periodicity of the audio signal in the commercial candidate section and increasing a commercial characteristic value if the continuity is above a predetermined level; detecting a repetition of the video signal in the commercial candidate section and increasing a commercial characteristic value if the repetition is above a predetermined level; searching for a telop in the video signal of the commercial candidate section and increasing a commercial characteristic value if a telop is detected; searching for a character in the video signal of the commercial candidate section and increasing a commercial characteristic value if a characters is detected; determining the quietness rate of the commercial candidate section and increasing a commercial characteristic value if the quietness rate is below a predetermined level; and judging whether or not the commercial candidate section is a commercial, based on the commercial characteristic value after the preceding steps are performed.

According to another aspect of the invention, there is provided a method of processing a television signal which includes the following steps: receiving the television signal; determining a quiet threshold value from the television signal; making a comparison of the audio signal and the quiet threshold value; and judging whether or not a portion of the television signal is a commercial candidate section according to the comparison.

According to another aspect of the invention, there is provided a provision medium for providing a program which is readable by a computer to control an apparatus to execute a detection routine for detecting commercials included in a television broadcast, wherein the detection routine includes the following steps: receiving a television signal; detecting a commercial candidate section in the television signal; detecting characteristics of the commercial candidate section; and judging whether or not the commercial candidate section is a commercial, based on the characteristics detected in the determining step.

According to another aspect of the invention, a provision medium provides a program which is readable by a computer to control an apparatus to execute a detection routine for detecting commercials included in a television broadcast, wherein the detection routine includes the following steps: receiving a television signal; detecting a commercial candidate section in the television signal; determining a frequency of scene changes in the commercial candidate section and increasing a commercial characteristic value if the frequency of scene changes is above a predetermined level; determining a periodicity of the audio signal in the commercial candidate section and increasing a commercial characteristic value if the periodicity is above a predetermined level; determining a continuity of a periodicity of the audio signal in the commercial candidate section and increasing a commercial characteristic value if the continuity is above a predetermined level; detecting a repetition of the video signal in the commercial candidate section and increasing a commercial characteristic value if the repetition is above a predetermined level; searching for a telop in the video signal of the commercial candidate section and increasing a commercial characteristic value if a telop is detected; searching for a character in the video signal of the commercial candidate section and increasing a commercial characteristic value if a characters is detected; determining the quietness rate of the commercial candidate section and increasing a commercial characteristic value if the quietness rate is below a predetermined level; and judging whether or not the commercial candidate section is a commercial, based on the commercial characteristic value after the preceding steps are performed.

According to another aspect of the invention, a provision medium provides a program which is readable by a computer to control an apparatus to execute a detection routine for detecting commercials included in a television broadcast, wherein the detection routine includes the following steps: receiving the television signal; determining a quiet threshold value from the television signal; making a comparison of the audio signal and the quiet threshold value; and judging whether or not a portion of the television signal is a commercial candidate section according to the comparison.

These and other objects, features and advantages will become apparent when considered with reference to the following description and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a first embodiment of a video recorder to which the present invention is applied; [0034]
FIG. 2 is a block diagram showing a first embodiment of [0035] commercial detection circuit 3 of FIG. 1;
FIG. 3 is a flowchart showing the operation of [0036] commercial detection circuit 3;
FIG. 4 is a flowchart showing the processing of quiet threshold [0037] value determining unit 11 of FIG. 2;
FIG. 5 is a diagram showing the operation of quiet threshold [0038] value determining unit 11 of FIG. 2;
FIG. 6 is a diagram showing the operation of quiet threshold [0039] value determining unit 11 of FIG. 2;
FIG. 7 is a diagram showing the operation of quiet threshold [0040] value determining unit 11 of FIG. 2;
FIG. 8 is a flowchart showing the quiet section detection processing of step S[0041] 2 of FIG. 3;
FIG. 9 is a flowchart showing the scene change detection processing of step S[0042] 3 of FIG. 3;
FIG. 10 is a diagram showing scene change detection processing; [0043]
FIG. 11 is a diagram showing information recorded in a memory in an embodiment of commercial [0044] candidate section detector 16;
FIG. 12 is a diagram showing an embodiment of the processing of commercial [0045] candidate section detector 16;
FIG. 13 is a diagram showing information recorded in the memory in another embodiment of commercial [0046] candidate section detector 16;
FIG. 14 is a diagram showing the processing of an embodiment of commercial [0047] candidate section detector 16;
FIG. 15 is a diagram showing the processing of an embodiment of commercial [0048] candidate section detector 16;
FIG. 16 is a block diagram showing a second embodiment of [0049] commercial detection circuit 3 of FIG. 1;
FIG. 17 is a diagram showing the processing of an embodiment of [0050] commercial block detector 21 of FIG. 16;
FIG. 18 is a diagram showing the processing of an embodiment of [0051] commercial block detector 21 of FIG. 16;
FIG. 19 is a block diagram showing a third embodiment of [0052] commercial detection circuit 3 of FIG. 1;
FIG. 20 is a flowchart showing the operation of an embodiment of commercial [0053] characteristic quantity detector 31 of FIG. 19;
FIG. 21 is a flowchart showing an embodiment of the periodicity detection processing of sound signals of step S[0054] 43 of FIG. 20;
FIG. 22 is a diagram showing an embodiment of the periodicity detection processing of sound signals; [0055]
FIG. 23 is a flowchart showing an embodiment of the continuity detection processing of step S[0056] 44 of FIG. 20;
FIG. 24 is a flowchart showing an embodiment of the repetition detection processing of step S[0057] 45 of FIG. 20;
FIG. 25 is a flowchart showing an embodiment of the steps of telop detection processing of step S[0058] 46 of FIG. 20;
FIG. 26 is a diagram showing an embodiment of the processing of one character of a telop; [0059]
FIG. 27 is a flowchart showing an embodiment of the quietness rate detection processing of step S[0060] 48 of FIG. 20;
FIG. 28 is a diagram showing an embodiment of quietness rate detection processing; and [0061]
FIG. 29 is a block diagram showing a second embodiment of a video recorder to which the present invention is applied.[0062]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A first embodiment of a video recorder to which the present invention is applied will be described with reference to FIG. 1. [0063]
[0064] Tuner 2 of a recording system decodes an RF signal of television broadcast input from a terminal 1 and outputs a video signal, an audio signal, an AGC signal and, optionally, a signal representing a sound multiplex mode (hereinafter referred to as a “sound multiplex mode signal” or “audio multiplex mode signal”) to commercial detection circuit 3. Tuner 2 outputs the video signal and the audio signal to a delay circuit 4.
[0065] Commercial detection circuit 3 judges whether the signal input from the tuner 2 is a commercial or not, and outputs “0” or “1” as a control signal to switch 6 in accordance with the judgment result. When the control signal represents “1,” switch 6 is switched off, and when the control signal represents “0,” switch 6 is switched on.
[0066] Delay circuit 4 delays the video signal and the audio signal input from tuner 2 by only the time required to perform the processing of commercial detection circuit 3 (in this embodiment, 1 minute), and then outputs the processed signals to modulation circuit 5. Modulation circuit 5 compresses the video signal and the audio signal input from delay circuit 4 in a predetermined format (for example, the MPEG2 format), and further modulates the signal by a predetermined modulation method (for example, EFM modulation). The modulated signal thus obtained is output to write-in circuit 7 through switch 6. Write-in circuit 7 records the input modulated signal on magnetic tape 8.
[0067] Reading circuit 9 of a reproduction system reads and decodes signals recorded in magnetic tape 8 in accordance with a command input from a user, and supplies the signal thus decoded to a monitor (not shown).
The medium in which the video signal and the audio signal are recorded is not limited to magnetic tape: the medium may be an optical disc, a magneto-optical disc, a hard disc, a semiconductor memory or the like. [0068]
FIG. 2 shows a first embodiment of [0069] commercial detection circuit 3 of FIG. 1. In commercial detection circuit 3, the sound multiplex mode signal input from tuner 2 is supplied to quiet threshold value determining unit 11 and commercial candidate section detector 16. The video signal is supplied to delay unit 14 and scene change detector 15, the audio signal is supplied to A/D converter 12, and the AGC signal is supplied to quiet threshold value determining unit 11.
Quiet threshold [0070] value determining unit 11 calculates a threshold value used in the quiet section detection processing by using the sound multiplex mode signal, the audio signal digitized in A/D converter 12, the AGC signal or scene change detector 15 (or a combination thereof), and supplies the threshold value thus calculated to quiet section detector 13.
[0071] Quiet section detector 13 compares the level of the digitized audio signal input from A/D converter 12 with the threshold value supplied from quiet threshold value determining unit 11 to detect quiet sections, and outputs the comparison result to scene change detector 15.
[0072] Scene change detector 15 compares two frame images which are simultaneously input (a current frame and a previous frame which is delayed by the time corresponding to one frame [{fraction (1/30)} second] in delay unit 14), thereby detecting the presence or absence of a scene change in the quiet section, and then outputs the result to commercial candidate section detector 16.
Commercial [0073] candidate section detector 16 converts the information of the scene change supplied from the sound multiplex mode signal and the scene change detector 15 to binary signals, stores the binary signals into the built-in memory, and detects a commercial candidate section on the basis of the foregoing information. Note that in alternative embodiments, some of which are described below, other information can be used to detect a CM candidate.
Thereafter, commercial [0074] candidate section detector 16 outputs “1” as the control signal to the switch 6 in the commercial candidate section, and outputs “0” as the control signal in sections other than the commercial candidate section. Information from the previous minute is stored in the memory contained in commercial candidate section detector 16. In this case, the capacity of the memory is equal to: 60 (seconds)×30 (frames)×2 (data)×1 (bit).
Next, the operation of [0075] commercial detection circuit 3 will be described with reference to the flowchart of FIG. 3. The commercial detection processing is started when each signal is input from the tuner 2 to the commercial detection circuit 3. In step S1, quiet threshold value determining unit 11 of commercial detection circuit 3 outputs to quiet section detector 13 the threshold value which is calculated beforehand (as described below) and used in the quiet section detection processing of step S2.
The details of the quiet threshold value processing will now be described with reference to the flowchart of FIG. 4. In the following description, quiet threshold [0076] value determining unit 11 performs many different functions. In practice, these functions could be performed by one component or by two or more components.
In step S[0077] 11, the A/D converter 12 digitizes an audio signal (analog signal) of a predetermined minute of time input from tuner 2 at a predetermined sampling frequency and a predetermined quantizing level, and outputs the digital audio signal thus obtained (for example, FIG. 5A) to quiet threshold value determining unit 11. In other embodiments, lengths of time other than one minute may be used. For embodiments in which digital signals are input, the digitizing step is omitted.
In step S[0078] 12, quiet threshold value determining unit 11 converts the digital audio signal input from A/D converter 12 to absolute values as shown in FIG. 5B, and in step S13 calculates the average of the absolute values of the samples (FIG. 5C).
In step S[0079] 14, quiet threshold value determining unit 11 compares the average value obtained in step S13 with the threshold value which has been stored until that time, and the smaller of these two values is stored as the new threshold value.
In step S[0080] 15, quiet threshold determining unit 11 judges whether the processing of steps S11 to S14 has been carried out on all the channels. If it is judged that the processing has not been carried out on all the channels, the processing goes to step S16. In step S16, quiet threshold value determining unit 11 outputs a channel switching signal to tuner 2. The channel is switched in accordance with the channel switching signal.
Thereafter, if it is judged in step S[0081] 15 that the processing from the steps S11 to the step S14 has been carried on all the channels, the quiet threshold value determining processing is completed. The quiet threshold value determining processing is repeated at a predetermined interval (for example, every 10 minutes) at all times.
Note that all channels need not be used in this process. In other embodiments of this process, not all of the channels are used. [0082]
Another method for determining the quiet threshold value will now be described. At a broadcast start time and a broadcast end time of each channel, a video signal exists but the level of an audio signal should be equal to 0. At these times, the audio signal is received and a value (n+Δ), obtained by adding the level n of the audio signal with a predetermined offset value Δ, may be set as a quiet threshold value. In this case, the broadcast start time and the broadcast end time of each channel are assumed to be known. [0083]
The quiet threshold value may also be determined by using the AGC signal from the [0084] tuner 2. That is, the level of the AGC signal from the tuner 2 and the amplitude of the audio signal plus noise have a predictable relationship to each other, as shown in FIG. 6. Accordingly, by estimating the amplitude of the audio signal plus noise from the level of the AGC signal, the threshold value corresponding to the amplitude thus estimated can be uniquely determined.
In addition, the buzz component of the audio signal is dependent on the brightness level of the video signal, and thus the quiet threshold value may be determined by using the average value of the brightness level of the video signal. For example, when the average value of the brightness level is high, the buzz component of the audio signal is increased and consequently the level of the audio signal is increased for a fixed period. Therefore, the threshold value is set to a higher value than the usual case. When the average value of the brightness level is low, the buzz component of the audio signal is reduced and consequently the level of the audio signal is reduced for a fixed period. Therefore, the threshold value is set to a lower value than the usual case. The average value Y[0085] _Aof the brightness of the video signal is calculated according to the following equation in scene change detector 15:
Y _A=(ΣD _ij)/(n×m),
where i=1 to n, j=1 to m and D[0086] _ijrepresents a pixel value at coordinate (i, j) of the image corresponding to the audio signal, as shown in FIG. 7.
The quiet threshold value may also be determined on the basis of the audio multiplex mode signal. That is, the level of the audio signal may be the quiet threshold value when the audio multiplex mode is changed from a bilingual broadcast to a stereo broadcast. [0087]
Returning to FIG. 3, in step S[0088] 2 quiet section detector 13 detects a quiet section on the basis of the threshold value which is input from quiet threshold value determining unit 11 in step S1. The details of the quiet section detection processing will be described with reference to the flowchart of FIG. 8.
In step S[0089] 21, the A/D converter 12 digitizes an audio signal (an analog signal) of a predetermined minute of time input from tuner 2 at a predetermined sampling frequency and a predetermined quantization level, and outputs the digital audio signal thus obtained (for example, as shown in FIG. 5A) to quiet threshold value determining unit 11. As noted above, for embodiments in which a digital signal is input, the digitizing step is omitted. In step S22, the digital audio signal input from A/D converter 12 is converted to absolute values as shown in FIG. 5B, and in step S23 the average absolute value of the samples thus obtained is calculated (FIG. 5C) by quiet section detector 13.
In step S[0090] 24, quiet threshold value determining unit 11 judges whether the average value obtained in step S23 is smaller than the threshold value input from quiet threshold value determining unit 11. If it is judged that the average value is smaller than the threshold value, the processing goes to step S25. In step S25, quiet section detector 13 judges this section as a quiet section, and outputs the information thereof to scene change detector 15.
Conversely, if it is judged in step S[0091] 24 that the average value is not smaller than the threshold value, the processing goes to step S26. In step S26, quiet section detector 13 judges this section as a section which is not quiet and outputs the information thereof to scene change detector 15.
The processing returns to the step S[0092] 3 of FIG. 3. In step S3, scene change detector 15 detects the scene change of two input consecutive frames and outputs the detection result to commercial candidate section detector 16. The details of the scene change detection processing will be described with reference to the flowchart of FIG. 9.
In step S[0093] 31, scene change detector 15 judges whether the information input from quiet section detector 13 indicates a quiet section. If it is judged that the information input indicates the presence of a quiet section, the processing goes to step S32.
In step S[0094] 32, the scene change detector 15 calculates the inverse correlation value E of two successive frame images. Specifically, as shown in FIG. 10, the sum of the absolute values of the differences between pixel values of corresponding pixels of an image input through delay unit 14 (a delay image) and an image which is not input through delay unit 14 (a through image) is calculated according to the following equation as the inverse correlation value:
E=(|D _ij −S _ij|)
D[0095] _ijand S_ijrepresent the pixel values at the coordinate (i, j) of the delay image and the through image, respectively. The inverse correlation value is larger as the degree of the correlation between the frames is reduced. On the other hand, as the degree of the correlation between the frames is increased, the inverse correlation value is smaller.
Another method of calculating the inverse correlation value E is to use histograms of the pixel values of the delay image and the through image. Alternatively, the delay image and the through image may be divided into predetermined blocks and then the inverse correlation value may be calculated from the differences of the pixel values in some or all of the blocks. [0096]
In step S[0097] 33, scene change detector 15 judges whether the inverse correlation value obtained in step S32 is larger than a predetermined threshold value. If the inverse correlation value is larger than the predetermined threshold value (the degree of the correlation between the frames is smaller), the processing goes to step S34.
In step S[0098] 34, scene change detector 15 judges whether a scene change exists between the two successive frames and outputs the result to commercial candidate section detector 16.
Conversely, in step S[0099] 33, if the inverse correlation value is not larger than the predetermined threshold value (the degree of the correlation between the frames is larger), the processing goes to step S35.
In step S[0100] 35, scene change detector 15 judges that no scene change exists between the two successive frames, and outputs the information thereof to commercial candidate section detector 16.
In step S[0101] 31, if it is judged that the information does not indicate a quiet section, the information is output to commercial candidate section detector 16 and the processing returns to the step S4 of FIG. 3.
In step S[0102] 4, commercial candidate section detector 16 judges the commercial candidate section on the basis of the frame-based audio multiplex mode signal of the previous minute of stored binary data in the built-in memory and the information from the scene change detector 15. Note that in other embodiments, the audio multiplex signal need not be used to determine commercial candidate sections.
As shown in FIG. 11, the following data are stored in the memory contained in commercial candidate section detector [0103] 16: with respect to the audio multiplex mode signal (Audio_Multi[]), “1” is recorded for the stereo mode while “0” is recorded for the monaural mode and the bilingual mode, and with respect to the information (Scene_Change[]) input from the scene change detector 15, “1” is recorded for a frame which is a scene change corresponding to the scene change while “0” is recorded for a frame which is not a scene change.
As shown in FIG. 12A, the commercial [0104] candidate section detector 16 refers to the built-in memory to partition the sections at every frame (scene change point) in which the signal indicating the scene change is equal to 1 (in this case, the scene change sections 0 to 10) and to divide the number of frames constituting each scene change section by 30, thereby calculating the time of the section. Further, as shown in FIG. 12B, commercial candidate section detector 16 sets a continuous frame having an audio multiplex mode signal of 1 as a stereo section. Still further, as shown in FIG. 12C, commercial candidate section detector 16 judges as a commercial candidate section a section for which the time of a single scene change section (or the total time of plural scene change sections adjacent to the scene change section concerned) is equal to an integral multiple of 15 seconds and which is a stereo section, and outputs “1” as a control signal to the switch 6 in response to such a commercial candidate section. Commercial candidate section detector 16 outputs “0” as a control signal to the switch 6 in response to sections other than a commercial candidate section.
[0105] Switch 6 is switched off when the control signal is “1” and switched on when the control signal is “0”. Switch 6 is supplied with the modulated video signal and the audio signal input from modulation circuit 5. The video signal and the audio signal are in synchronization with the control signal from commercial candidate section detector 16, because these signals are delayed by only one minute in delay circuit 4. Therefore, only the video signal and audio signal of the program are supplied to those parts to the circuit which are subsequent to switch 6, and the video signal and audio signal of the commercial candidate sections are not supplied thereto. Accordingly, only the program is recorded in magnetic tape 8.
Next, another embodiment of the commercial [0106] candidate section detector 16 will be described. A general tendency of commercials is that they have many scene changes. In this embodiment, the characteristic of multiple scene changes is used to identify a commercial candidate section.
As shown in FIG. 13, the memory contained in commercial [0107] candidate section detector 16 records not only the information as shown in FIG. 11, but also information (No_Sound[]) indicating a quiet section which is an output of quiet section detector 13 (“0” for a sound portion and “1” for a quiet portion). In this case, the following capacity is needed to the built-in memory: 60(seconds)×30(frames)×3(data)×1(bit).
Commercial [0108] candidate section detector 16 refers to the built-in memory to extract a quiet section as shown in FIG. 14A, extract a scene change point as shown in FIG. 14B, partition the scene change section at every scene change point in the quiet section (in the case of FIG. 14C, scene change sections 0 to 10) and divide the number of frames constituting each scene change section by 30, thereby calculating the time of the section.
Commercial [0109] candidate section detector 16 extracts a section which is grouped so that the time of the single scene change section (or the total time of plural scene change sections adjacent to the scene change section concerned) is equal to an integral multiple of 15 seconds, as shown in FIG. 14D, and, in this embodiment, whose audio multiplex mode signal is “1,” as shown in FIG. 14E. Then, commercial candidate section detector 16 groups the sections and judges that a section which is grouped at an integral multiple of 15 seconds, as shown in FIG. 14F, and is a stereo section (in this case, sections a to c, and sections d and e), is a commercial candidate section. Further, as shown in FIG. 14G, commercial candidate section detector 16 compares a predetermined threshold value (for example, “1”) with the frequency of the scene change in the commercial candidate sections (in this case, the commercial candidate sections a, c, d and e) at both sides of the adjacent commercial candidate section, and judges that a commercial candidate section (in this case, the commercial candidate sections a and c) having a scene change frequency larger than the threshold value is a commercial section. Commercial candidate section b, sandwiched between the commercial candidate sections a and c, is also judged as a commercial section.
Commercial [0110] candidate section detector 16 outputs “1” as a control signal to the switch 6 in response to a commercial section, and outputs “0” as a control signal to the switch 6 in response to sections other than a commercial section.
Next, another embodiment of the commercial [0111] candidate section detector 16 will be described. In this embodiment, when a characteristic unique to commercials is altered (for example, the time of a commercial is changed from an integral multiple of 15 seconds to an integral multiple of 14 seconds or the like), the reference value used for the judgment of the commercial candidate section is changed accordingly.
In the above operation, commercial [0112] candidate section detector 16 refers to the built-in memory (e.g., as shown in FIG. 11) to partition the scene change section at every frame whose scene change signal is “1” (in this case, scene change sections 0 to 10) and divide the number of frames constituting each scene change section by 30, thereby calculating the time of the section as shown in FIG. 15A. In this embodiment, commercial candidate section detector 16 sets as a stereo section a continuous frame whose audio multiplex mode signal is “1,” as shown in FIG. 15B.
Here, commercial [0113] candidate section detector 16 compares the time of the stereo section with the time of a non-stereo section which is adjacent to the stereo section concerned. If the time of the stereo section is sufficiently shorter, the stereo section (for example, scene change sections 1 to 4 and scene change sections 8 and 9) is judged as a commercial candidate section.
Since the total time of the commercial candidate sections (the [0114] scene change sections 1 to 4) is equal to 56 seconds, commercial candidate section detector 16 judges that the time of commercials is on a 7-second basis, a 14-second basis or a 28-second basis. Further, since the scene change is carried out at a time interval of 14 seconds or 28 seconds, commercial candidate section detector 16 finally judges that the time of commercials is equal to an integral multiple of 14 seconds.
When the above result (the judgement that the time of a commercial is equal to an integral multiple of 14 seconds) is continuously obtained at a predetermined number of times or more, commercial [0115] candidate section detector 16 judges that the time of a commercial has been changed to an integral multiple of 14 seconds, and then uses an integral multiple of 14 seconds as a reference value for the detection of commercial candidate sections.
When a characteristic unique to commercials is altered (the alteration of the commercial time as described above, the abolition of the insertion of a black frame or blue frame in America and Europe or the like), the reference value or other reference information for the commercial section judgment which is altered in accordance with the altered absolute characteristic may be supplied from outside of the device to commercial [0116] candidate section detector 16. In this case, in commercial candidate section detector 16, reference information for the detection of commercial sections may be stored in a rewritable recording medium such as a flash memory.
Next, a second embodiment of [0117] commercial detection circuit 3 will be described with reference to FIG. 16.
This construction is obtained by adding [0118] commercial block detector 21 to the construction shown in FIG. 2. Commercial block detector 21 detects a commercial block comprising a plurality of commercials sandwiched between programs by using the frame number of a single commercial candidate section detected in commercial candidate section detector 16, and outputs a control signal to switch 6 on the basis of the detection result.
According to this construction, the commercial block can be detected by using the following characteristic of commercials: when a plurality of commercials is continuously broadcast, in terms of an individual commercial, each commercial has an error of about 5 frames relative to a standard frame number. However, in terms of a commercial block (continuous plural commercials), each commercial block has an error of about 3 frames for the standard frame number. For example, when four commercials each having a commercial time of 15 seconds are consecutively broadcast, the number of frames of each commercial is equal to 450±5 (=15×30±5). However, the number of frames of each commercial block is not equal to 1800±20 (=15×4×30±5×4), but is equal to 1800±3. [0119]
Next, the operation of the device will be described with reference to FIGS. 17 and 18. [0120]
The [0121] commercial block detector 21 reads out the frame number of each commercial candidate section a through e (sections 1+2, 3, 4, 8 and 9 of FIG. 17A) detected by commercial candidate section detector 16 and adds the frame number to the frame number of a block of adjacent commercial candidate sections (a “commercial candidate block”) to judge whether the total number of frames is within a permitted range of the error (±3). If commercial block detector 21 judges that the total number of frames is within the permitted range of the error, the commercial candidate block is judged as a commercial block. Conversely, if commercial block detector 21 judges that the total number of frames of the error is not within the permitted range of the error, the commercial candidate block is judged not to be a commercial block.
In this example, as shown in FIG. 17, the total number of frames of commercial candidate sections a through c ([0122] sections 1 to 4) is equal to 1803 (=453+446+904), and the error (3) relative to the standard frame number 1800 (=60 seconds×30 frames) is within the permitted range, so that this commercial candidate block is judged as a commercial block. Note that in this instance, scene change sections 1 and 2 have a total time of about 15 seconds and were judged to comprise a single commercial candidate section (a).
On the other hand, the total number of frames of adjacent commercial candidate sections d and e (quiet [0123] scene change sections 8 and 9) is equal to 910 (=455+455) and the error (10) relative to the standard frame number 900 (=30 seconds×30 frames) is out of the permitted range, so that this commercial candidate block is judged not to be a commercial block.
When the commercial candidate section is input from the commercial [0124] candidate section detector 16, commercial block detector 21 judges quiet scene change sections 1 to 4 (commercial block A) and quiet scene change sections 7 to 10 (commercial block B) as commercial blocks, as shown in FIG. 18C.
Furthermore, [0125] commercial block detector 21 reads out from commercial candidate section detector 16 the number of frames of a section (scene change sections 5 and 6) which is sandwiched between the commercial blocks A and B and is not a commercial candidate section. If the number of frames (in this case, 150) is below a predetermined threshold value (for example, 300 frames (10 seconds)), commercial block detector 21 judges the section not to be part of a program, but something like a commercial, and then includes the sandwiched section with the preceding and following commercial blocks A and B. That is, commercial block detector 21 judges the scene change sections 1 to 10 as one commercial block, as shown in FIG. 18D.
[0126] Commercial block detector 21 outputs “1” as a control signal to the switch 6 when a commercial block is detected, and outputs “0” as the control signal to the switch 6 when a commercial block is not detected.
Next, a third embodiment of [0127] commercial detection circuit 3 will be described with reference to FIG. 19.
This construction is obtained by adding commercial [0128] characteristic quantity detector 31 to the first embodiment of FIG. 2. Commercial characteristic quantity detector 31 is supplied with two front and rear frame images (video signals), digitized audio signals, information (e.g., as shown in FIG. 11) stored in the memory contained in commercial candidate section detector 16 and information about the commercial candidate section detected by commercial candidate section detector 16. Commercial characteristic quantity detector 31 judges whether the supplied information has various characteristics of commercials and detects commercial sections on the basis of the resulting judgment. Commercial characteristic quantity detector 31 outputs a control signal to switch 6 in accordance with the detection result.
The operation of commercial [0129] characteristic quantity detector 31 will be described with reference to the flowchart of FIG. 20. The commercial characteristic quantity detection processing is executed on each commercial candidate section detected by the commercial candidate section detector 16. Although commercial characteristic quantity detector 31 is described below as having many functions, these functions could be performed by separate elements.
In the embodiment described below, commercial [0130] characteristic quantity detector 31 assigns the same value to each commercial characteristic, but these values could be varied to indicate that some commercial characteristics are more likely than others to be associated with a commercial. For example, the telop detection of step S46 and the character detection of step S47 could be assigned a higher value than other commercial characteristics. Moreover, any of the criteria used to detect or verify commercial candidate sections could also be used by commercial characteristic quantity detector 31, e.g., the number of frames of a commercial candidate group or of a commercial candidate, quiet scene change sections or the audio multiplex mode.
In step S[0131] 41, commercial characteristic quantity detector 31 initializes the commercial characteristic value to 0. In step S42, commercial characteristic quantity detector 31 counts the scene change frequency of the commercial candidate section input from the commercial candidate section detector 16, and judges whether the scene change frequency is above a predetermined threshold value (e.g., 5 per 15 seconds). If it is judged that the measured scene change frequency is above the predetermined threshold value, “1” is added to the commercial characteristic value. Conversely, if it is judged that the measured scene change frequency is less than the predetermined threshold value, no addition is carried out on the commercial characteristic value. This processing is based on a general observation that commercials often have numerous scene changes. In step S42, commercial characteristic quantity detector 31 detects the periodicity of the audio signal of a commercial candidate section. The periodicity detection processing of the audio signal is based on the observation that repeated phrases of background music (e.g., “jingles”) are often used in commercials. In the periodicity detection step, the rhythm of background music may also be detected.
The periodicity detection processing of the audio signal will be described with reference to the flowchart of FIG. 21. In step S[0132] 61, as shown in FIG. 22, commercial characteristic quantity detector 31 detects as peaks those points at which the level of the audio signal input from A/D converter 12 is greater than or equal to a predetermined threshold value (FIG. 22B). FIG. 22A shows the audio signal as an analog signal.
In step S[0133] 62, commercial characteristic quantity detector 31 judges whether the peaks detected in step S61 have a longer periodicity than a predetermined period (several seconds). The judgment of the periodicity may be performed by using a FFT (Fast Fourier Transform) or by merely measuring the intervals between peaks. If it is judged that the detected peaks have a longer periodicity than the predetermined period, the processing goes to step S63.
In step S[0134] 63, commercial characteristic quantity detector 31 adds “1” to the commercial characteristic value. If it is judged in step S62 that the detected peak does not have a longer periodicity than the predetermined period, no addition is carried out on the commercial characteristic value, and thus step S63 is skipped.
Returning to step S[0135] 44 of FIG. 20, commercial characteristic quantity detector 31 detects the continuity of the audio signal and the video signal of the commercial candidate section in step S44. The continuity detection processing will be described with reference to the flowchart of FIG. 23.
In step S[0136] 71, commercial characteristic quantity detector 31 judges whether the periodicity of the peaks of the audio signal continues for a time which is equal to 95% or more of the commercial candidate section. The measurement of the continuity time of the periodicity of the peaks is performed by using the information obtained in step S43. If it is judged that the periodicity of the peaks of the audio signal continues for a time which is equal to 95% or more of the commercial candidate section, the processing goes to step S72.
In step S[0137] 72, commercial characteristic quantity detector 31 adds “1” to the commercial characteristic value.
If it is judged in step S[0138] 71 that the periodicity of the peaks of the audio signal does not continue for a time which is equal to 95% or more of the commercial candidate section, no addition is carried out on the commercial characteristic value and step S72 is skipped.
In step S[0139] 73, commercial characteristic quantity detector 31 calculates the difference between the pixel values of corresponding pixels of two successive images and detects pixels whose differential values are below a predetermined threshold value, that is, pixels having no motion between the two images. In step S74, commercial characteristic quantity detector 31 generates a histogram of the pixel values of the pixels detected in step S73, and detects the pixel value indicating the maximum value as a representative color of the background. The processing of steps S73 and S74 is repetitively carried out in the commercial candidate section at a predetermined interval.
In step S[0140] 75, commercial characteristic quantity detector 31 refers to plural representative colors of the background detected in step S74, and judges whether the same pixel value is continuously detected as the representative color of the background. If it is judged that the same pixel value is continuously detected as the representative color of the background, the processing goes to step S76.
In step S[0141] 76, commercial characteristic quantity detector 31 adds “1” to the commercial characteristic value.
If it is judged in step S[0142] 75 that the same pixel value is not continuously detected as the representative color of the background, no addition is carried out on the commercial characteristic value and step S76 is skipped.
Returning to step S[0143] 45 of FIG. 20, the commercial characteristic quantity detector 31 detects the repetition of the image in step S45. The repetition detection processing is based on the observation that the same images are often repeated in commercials. For example, a ketchup commercial of 15 seconds might include a green forest scene (3 seconds), a blue sky scene (2 seconds), a green forest scene (2 seconds), a red ketchup scene (3 seconds), a green forest scene (2 seconds) and a red ketchup scene (3 seconds).
The repetition detection processing of this image will be described with reference to the flowchart of FIG. 24. [0144]
In step S[0145] 81, commercial characteristic quantity detector 31 segments the commercial candidate section at scene change points. In step S82, commercial characteristic quantity detector 31 generates a histogram of the pixel values of each scene segmented in step S81 with 16 gradations, and detects the pixel value indicating the maximum value as a representative color of the scene.
In step S[0146] 83, commercial characteristic quantity detector 31 judges whether the representative color of the scene is repeated (the same representative color is detected in different scenes) in the commercial candidate section. If it is judged that the representative color of the scene is repeated, the processing goes to step S84.
In step S[0147] 84, commercial characteristic quantity detector 31 adds “1” to the commercial characteristic value.
If it is judged in step S[0148] 83 that the representative color of the scene is not repeated, no addition is carried out on the commercial characteristic value, and thus step S84 is skipped.
Returning to the step S[0149] 46 of FIG. 20, commercial characteristic quantity detector 31 detects a “telop” (which could be a logo, a graphic, a mark, a slogan, or the like), as shown in FIG. 26A. The telop detection processing is based on the general observation that a telop is often displayed at or near the end time of a commercial. However, this routine could search for a telop at any part of a commercial candidate section.
The telop detection processing will be described with reference to the flowchart of FIG. 25. In step S[0150] 91, commercial characteristic quantity detector 31 detects pixels which have the same pixel value and are adjacent to other pixels whose number is larger than a predetermined number in each frame image of a predetermined time (for example, 5 seconds) from the end of a commercial candidate section, as shown in FIG. 26(B).
In step S[0151] 92, commercial characteristic quantity detector 31 searches and regionalizes pixels having the same pixel value as the pixels detected in step S91, as shown in FIG. 26(C).
In step S[0152] 93, commercial characteristic quantity detector 31 judges whether the area obtained in step S92 is continuous in time (i.e., if the same area exists in the prior frame and in the next frame). If it is judged that the area is continuous in time, the processing goes to step S94.
In step S[0153] 94, commercial characteristic quantity detector 31 detects the centroid of all the areas on an image, and calculates the average value of the distance between the centroid and each point on the areas. Commercial characteristic quantity detector 31 further compares the average value thus calculated with a predetermined threshold value to judge whether the areas are dense not. If it is judged that the areas are dense, the processing goes to step S95.
In step S[0154] 95, commercial characteristic quantity detector 31 adds “1” to the commercial characteristic value.
If it is judged in step S[0155] 93 that the areas are not continuous in time, no addition is carried out on the commercial characteristic value, and thus step S95 is skipped.
Returning to step S[0156] 47 of FIG. 20, by using the same method as the telop detection processing as described above, commercial characteristic quantity detector 31 judges whether characters exist on an image for a time longer than a predetermined time from the start to the end of the commercial candidate section. If it is judged that characters exist on the image for a time longer than the predetermined time, “1” is added to the commercial characteristic value. Conversely, if it is judged that the characters do not exist on the image for a time longer than the predetermined time, no addition is carried out to the commercial characteristic value. This processing is based on a general observation that characters are often displayed in commercials.
In step S[0157] 48, commercial characteristic quantity detector 31 detects the rate of a quiet section other than the start and end points of the commercial candidate section. The quietness rate detection processing is based on the general observation that quiet sections are rare in commercials. The quietness rate detection will be described with reference to the flowchart of FIG. 27.
In step S[0158] 101, commercial characteristic quantity detector 31 reads out the information (e.g., the information shown in FIG. 11) stored in the memory of the commercial candidate section detector 16, and detects the frame number of the quiet section other than the start and end points of the commercial candidate section, as shown in FIG. 28. Further, commercial characteristic quantity detector 31 divides the frame number of the quiet section thus detected by the number of all the frames constituting the commercial candidate section to calculate the quietness rate.
In step S[0159] 102, commercial characteristic quantity detector 31 judges whether the quietness rate calculated in step S101 is smaller than a predetermined threshold value. If it is judged that the quietness rate is smaller than the predetermined threshold value, the processing goes to step S103.
In step S[0160] 103, commercial characteristic quantity detector 31 adds “1” to the commercial characteristic value.
If it is judged in step S[0161] 102 that the quietness rate is not smaller than the predetermined threshold value, no addition is carried out to the commercial characteristic value, and thus step S103 is skipped.
Returning to step S[0162] 49 of FIG. 20, commercial characteristic quantity detector 31 judges whether the commercial characteristic value is larger than a predetermined threshold value or not. If it is judged that the commercial characteristic value is larger than the predetermined threshold value, the processing goes to step S50. In step S50, commercial characteristic quantity detector 31 judges the commercial candidate section to be a commercial section.
If it is judged in step S[0163] 49 that the commercial characteristic value is not larger than the predetermined threshold value, the processing goes to step S51. In step S51, commercial characteristic quantity detector 31 judges the commercial candidate section not to be a commercial section.
In connection with this judgment result, commercial [0164] characteristic quantity detector 31 outputs “1” as a control signal to switch 6 in a commercial section, and outputs “0” as a control signal to switch 6 out of the commercial section.
As described above, in a video recorder (FIG. 1) of this embodiment, only a program of a television broadcast is recorded on [0165] magnetic tape 8, and no commercial is recorded on magnetic tape 8. Accordingly, when magnetic tape 8 is reproduced, only the program is displayed.
Some users want to record commercials, although these users typically fast-forward or skip the commercials when the recording is played. Therefore, a second embodiment of a video recorder which satisfies these needs, and to which the present invention is applied, will be described with reference to FIG. 29. This embodiment is obtained by removing [0166] switch 6 from the construction shown in FIG. 1 and supplying the commercial detection result of commercial detection circuit 3 to write-in circuit 7.
In the recording system of the second embodiment, write-[0167] in circuit 7 records all the video and audio signals (programs and commercials) of a television broadcast input from modulation circuit 5 on magnetic tape 8, and records information about the commercials (the time positions of the commercials, etc.) at predetermined positions on magnetic tape 8 on the basis of information from commercial detection circuit 3.
In the reproduction system, when a command for reproducing only the program (a command for removing commercials) is input from a user, read-[0168] out circuit 9 removes (fast-forwards or the like) the commercials on the basis of the information regarding the commercials recorded at predetermined positions of magnetic tape 8, demodulates only the program and supplies it to a monitor (not shown).
The present invention is not limited to a video recorder; it may be applied to a television receiver, a tuner, etc. [0169]
A computer program to perform the processing described above can be supplied to users not only via a recording medium such as a magnetic disc, CD-ROM or the like, but also a network supply medium such as Internet, a digital satellite or the like. [0170]
As described above, according to the information processing device, the information processing method and the supply medium of the present invention, a commercial candidate section is detected, the characteristics of commercials are detected in the commercial candidate section, a predetermined value is added to a commercial quantity value in accordance with the detection result, and it is judged on the basis of the commercial quantity value whether the commercial candidate section is a commercial or the like. Therefore, commercials contained in a television broadcast can be reliably detected. [0171]
Although only certain embodiments have been described in detail, those having ordinary skill in the art will certainly understand that many modifications are possible without departing from the teachings thereof. All such modifications are intended to be encompassed within the following claims. [0172]

Claims

We claim:

1. An apparatus for processing a television signal, wherein the television signal comprises programs and commercials, the apparatus comprising:

receiving means for receiving a television signal;

detecting means for detecting a commercial candidate section in the television signal;

determining means for determining characteristics of the commercial candidate section; and

judgement means for judging whether or not the commercial candidate section is a commercial, based on the characteristics detected by the determining means.

2. The apparatus of claim 1, wherein:

the commercial candidate section comprises scene changes; and

the determining means comprises means for determining a frequency of the scene changes in the commercial candidate section.

3. The apparatus of claim 1, wherein the determining means comprises:

periodicity detecting means for detecting whether or not the audio signal in the commercial candidate section has a periodicity; and

periodicity calculation means for calculating the periodicity detected by the periodicity detecting means.

4. The apparatus of claim 1, wherein:

the commercial candidate section comprises an audio signal with periodicity; and

the determining means comprises means for determining a degree of continuity of the periodicity of the audio signal.

5. The apparatus of claim 1, wherein:

the commercial candidate section comprises a video signal with repetition; and

the determining means comprises means for determining the repetition of the video signal in the commercial candidate section.

6. The apparatus of claim 1, wherein the determining means comprises means for determining whether or not a telop exists in the video signal of the commercial candidate section.

7. The apparatus of claim 1, wherein the determining means comprises means for determining whether or not a character exists in the video signal of the commercial candidate section.

8. The apparatus of claim 1, wherein the determining means comprises means for determining a quietness rate in the commercial candidate section.

9. The apparatus of claim 1, wherein:

the television signal comprises quiet frames; and

the detecting means comprises means for detecting quiet frames of the television signal which are disposed between a program and a commercial candidate section.

10. An apparatus for processing a television signal, wherein the television signal comprises programs, commercials and scene changes, the apparatus comprising:

receiving means for receiving a television signal;

commercial candidate section detecting means for detecting a commercial candidate section in the television signal;

frequency determining means for determining a frequency of scene changes in the commercial candidate section and for increasing a commercial characteristic value if the frequency of scene changes is above a predetermined level;

periodicity determining means for determining whether or not the audio signal in the commercial candidate section has a periodicity, for calculating a level of periodicity detected and for increasing a commercial characteristic value if the calculated periodicity is above a predetermined level;

continuity determining means for determining a level of continuity of the periodicity of the audio signal in the commercial candidate section and for increasing a commercial characteristic value if the continuity is above a predetermined level;

repetition determining means for determining a level of repetition of the video signal in the commercial candidate section and for increasing a commercial characteristic value if the repetition is above a predetermined level;

telop detecting means for detecting a telop in the video signal of the commercial candidate section and for increasing a commercial characteristic value if a telop is detected;

character detecting means for detecting a character in the video signal of the commercial candidate section and for increasing a commercial characteristic value if a character is detected;

quietness rate determining means for determining the quietness rate of the commercial candidate section and for increasing a commercial characteristic value if the quietness rate is below a predetermined level; and

judgement means for judging whether or not the commercial candidate section is a commercial, based on the commercial characteristic value resulting from the operations of the frequency determining means, the periodicity determining means, the continuity determining means, the repetition determining means, the telop determining means, the character determining means and the quietness rate determining means.

11. An apparatus for processing a television signal, wherein the television signal comprises an audio signal and a video signal, the apparatus comprising:

receiving means for receiving the television signal;

determining means for determining a quiet threshold value from the television signal;

comparing means for making a comparison of the audio signal and the quiet threshold value; and

judging means for judging whether or not a portion of the television signal is a commercial candidate section according to the comparison.

12. The apparatus of claim 11, wherein:

the television signal comprises scene changes;

the determining means comprises means for detecting the scene changes and means for detecting the audio signal during the scene changes; and

the determining means determines the quiet threshold value based on the audio signal at the scene changes.

13. The apparatus of claim 11, wherein:

the television signal comprises an audio multiplex mode signal;

the determining means comprises means for detecting the audio multiplex mode from the audio multiplex mode signal and means for detecting the audio signal when the audio multiplex mode changes; and

the determining means determines the quiet threshold value based on the audio signal at the time the audio multiplex mode changes.

14. The apparatus of claim 11, wherein:

the television signal comprises a program start time;

the determining means comprises means for detecting the program start time and means for detecting the audio signal at the program start time; and

the determining means determines the quiet threshold value based on the audio signal at the program start time.

15. The apparatus of claim 11, wherein:

the television signal comprises a program end time;

the determining means comprises means for detecting the program end time and means for detecting the audio signal at the program end time; and

the determining means determines the quiet threshold value based on the audio signal at the program end time.

16. The apparatus of claim 11, wherein:

the television signal comprises an AGC signal;

the determining means comprises means for detecting the AGC signal and means for detecting the audio signal; and

the determining means determines the quiet threshold value based on the audio signal and the AGC signal.

17. The apparatus of claim 11, wherein:

the video signal comprises a brightness level;

the determining means comprises means for determining the brightness level and means for detecting the audio signal; and

the determining means determines the quiet threshold value based on the audio signal and the brightness level.

18. The apparatus of claim 11, wherein:

the television signal is organized in channels;

the determining means comprises selecting means for selecting one or more channels of the television signal; and

the determining means determines the quiet threshold value based on the audio signal of each channel selected by the selecting means.

19. The apparatus of claim 18, wherein the determining means determines the quiet threshold value at a predetermined time interval.

20. The apparatus of claim 18, further comprising:

storing means for storing a previous quiet threshold value;

quiet threshold value comparing means for determining whether the new quiet threshold value based on the audio portion of each selected channel of the television signal is smaller than the previous quiet threshold value; and

means for outputting the new quiet threshold value to the storing means if the quiet threshold value comparing means determines that the new quiet threshold value is smaller than the previous quiet threshold value.

21. An apparatus for processing a television signal, wherein the television signal comprises programs and commercials, the apparatus comprising:

a receiver for receiving a television signal;

a first detector for detecting a commercial candidate section in the television signal;

a second detector for detecting characteristics of the commercial candidate section; and

a judgment circuit for judging whether or not the commercial candidate section is a commercial, based on the characteristics detected by the second detector.

22. The apparatus of claim 21, wherein:

the commercial candidate section comprises scene changes; and

the second detector detects the frequency of the scene changes in the commercial candidate section.

23. The apparatus of claim 21, wherein:

the commercial candidate section comprises an audio signal which has a periodicity; and

the second detector detects the periodicity of the audio signal in the commercial candidate section.

24. The apparatus of claim 21, wherein:

the second detector detects a continuity of the periodicity of the audio signal in the commercial candidate section.

25. The apparatus of claim 21, wherein:

the commercial candidate section comprises a video signal with repetition; and

the second detector detects repetition of the video signal in the commercial candidate section.

26. The apparatus of claim 21, wherein the second detector determines whether or not a telop exists in the video signal of the commercial candidate section.

27. The apparatus of claim 21, wherein the second detector determines whether or not a character exists in the video signal of the commercial candidate section.

28. The apparatus of claim 21, wherein the second detector detects a quietness rate of the commercial candidate section.

29. The apparatus of claim 21, wherein:

the television signal comprises quiet frames; and

the first detector detects quiet frames of the television signal which are disposed between a program and a commercial candidate section.

30. An apparatus for processing a television signal, wherein the television signal comprises programs, commercials and scene changes, the apparatus comprising:

a receiver for receiving a television signal;

a commercial candidate section detecting circuit for detecting a commercial candidate section in the television signal;

a commercial characteristic value counter for storing a previous commercial characteristic value, for receiving a commercial characteristic value signal and for adding a commercial characteristic value from the commercial characteristic value signal to the previous commercial characteristic value;

a frequency determining circuit for determining a frequency of scene changes in the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if the frequency of scene changes is above a predetermined level;

a periodicity determining circuit for detecting whether or not the audio signal in the commercial candidate section has a periodicity, for calculating a level of periodicity detected and for sending a commercial characteristic value signal to the commercial characteristic value counter if the calculated periodicity is above a predetermined level;

a continuity determining circuit for determining a level of continuity of the periodicity of the audio signal in the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if the continuity is above a predetermined level;

a repetition determining circuit for determining a level of repetition of the video signal in the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if the repetition is above a predetermined level;

a telop detecting circuit for detecting a telop in the video signal of the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if a telop exists;

a character detecting circuit for detecting a character in the video signal of the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if a character is detected;

a quietness rate determining circuit for determining the quietness rate of the commercial candidate section and for sending a commercial characteristic value signal to the commercial characteristic value counter if the quietness rate is below a predetermined level; and

a judgement circuit for judging whether or not the commercial candidate section is a commercial, based on the commercial characteristic value in the commercial characteristic value counter.

31. An apparatus for processing a television signal, wherein the television signal comprises an audio portion, the apparatus comprising:

a receiver for receiving the television signal;

a quiet threshold value determining circuit for determining a quiet threshold value from the television signal;

a comparitor for making a comparison of the audio portion of the television signal and the quiet threshold value; and

a judging circuit for judging whether or not a portion of the television signal is a commercial candidate section according to the comparison.

32. The apparatus of claim 31, wherein:

the television signal comprises scene changes;

the quiet threshold value determining circuit comprises a scene change detector and an amplitude detector for measuring the amplitude of the audio portion of the television signal during the scene changes; and

the quiet threshold value determining circuit determines the quiet threshold value based on the audio portion of the television signal at the scene changes.

33. The apparatus of claim 31, wherein:

the television signal comprises an audio multiplex mode signal;

the quiet threshold value determining circuit comprises an audio multiplex mode detector for determining the state of the audio multiplex mode signal and an amplitude detector for measuring the amplitude of the audio signal when the audio multiplex mode changes; and

the quiet threshold value determining circuit determines the quiet threshold value based on the audio signal at the time the audio multiplex mode changes.

34. The apparatus of claim 31, wherein:

the television signal comprises a program start time;

the quiet threshold value determining circuit comprises a program start time detector and an amplitude detector for measuring the amplitude of the audio signal the audio signal at the program start time; and

the quiet threshold value determining circuit determines the quiet threshold value based on the amplitude of the audio signal at the program start time.

35. The apparatus of claim 31, wherein:

the television signal comprises a program end time;

the quiet threshold value determining circuit comprises a program end time detector and an amplitude detector for measuring the amplitude of the audio signal the audio signal at the program end time; and

the quiet threshold value determining circuit determines the quiet threshold value based on the amplitude of the audio signal at the program end time.

36. The apparatus of claim 31, wherein:

the television signal comprises an AGC signal;

the quiet threshold value determining circuit comprises a voltage detector for detecting amplitude of the AGC signal and an amplitude detector for measuring the amplitude of the audio signal; and

the quiet threshold value determining circuit determines the quiet threshold value based on the audio signal and the AGC signal.

37. The apparatus of claim 31, wherein:

the video signal comprises a brightness level;

the quiet threshold value determining circuit comprises a brightness detection circuit for determining the brightness level of the video signal and an amplitude detector for measuring the amplitude of the audio signal; and

the quiet threshold value determining circuit determines the quiet threshold value based on the amplitude and the brightness level.

38. The apparatus of claim 31, wherein:

the television signal is organized in channels;

the quiet threshold value determining circuit comprises a channel selector for selecting one or more channels of the television signal; and

the quiet threshold value determining circuit determines the quiet threshold value based on the audio portion of each channel of the television signal selected by the selecting means.

39. The apparatus of claim 38, wherein the quiet threshold value determining circuit determines the quiet threshold value at a predetermined time interval.

40. The apparatus of claim 38, further comprising:

a memory for storing a previous quiet threshold value;

a quiet threshold value comparing circuit for comparing the quiet threshold value based on the audio portion of each channel of the television signal with the previous quiet threshold value and for outputting the smaller of the two values to the memory.

41. A method for processing a television signal, wherein the television signal comprises programs and commercials, the method comprising:

a receiving step of receiving a television signal;

a detecting step of detecting a commercial candidate section in the television signal;

a determining step of determining characteristics of the commercial candidate section; and

a judgement step of judging whether or not the commercial candidate section is a commercial, based on the characteristics detected in the determining step.

42. The method of claim 41, wherein:

the commercial candidate section comprises scene changes; and

the determining step comprises the step of determining the frequency of the scene changes in the commercial candidate section.

43. The method of claim 41, wherein the determining step comprises:

a periodicity determining step of determining whether or not the audio signal in the commercial candidate section has a periodicity; and

a periodicity calculating step of calculating the periodicity determined in the periodicity determining step.

44. The method of claim 41, wherein:

the determining step comprises the step of determining the continuity of the periodicity.

45. The method of claim 41, wherein:

the commercial candidate section comprises a video signal which has repetition; and

the determining step comprises the step of determining the amount of repetition of the video signal in the commercial candidate section.

46. The method of claim 41, wherein the determining step comprises the step of determining whether a telop exists in the video signal of the commercial candidate section.

47. The method of claim 41, wherein the determining step comprises the step of determining whether a character exists in the video signal of the commercial candidate section.

48. The method of claim 41, wherein the determining step comprises the step of determining a quietness rate of the commercial candidate section.

49. The method of claim 41, wherein:

the television signal comprises quiet frames; and

the detecting step comprises the step of detecting quiet frames of the television signal which are disposed between a program and a commercial candidate section.

50. A method for processing a television signal, wherein the television signal comprises programs, commercials and scene changes, the method comprising:

a receiving step of receiving a television signal;

a commercial candidate section detecting step of detecting a commercial candidate section in the television signal;

a frequency determining step, comprising the steps of determining a frequency of scene changes in the commercial candidate section and of increasing a commercial characteristic value if the frequency of scene changes is above a predetermined level;

a periodicity determining step, comprising the steps of determining a periodicity of the audio signal in the commercial candidate section and of increasing a commercial characteristic value if the periodicity is above a predetermined level;

a continuity determining step, comprising the steps of determining a continuity of a periodicity of the audio signal in the commercial candidate section and of increasing a commercial characteristic value if the continuity is above a predetermined level;

a repetition detecting step, comprising the steps of detecting a repetition of the video signal in the commercial candidate section and of increasing a commercial characteristic value if the repetition is above a predetermined level;

a telop searching step, comprising the steps of searching for a telop in the video signal of the commercial candidate section and of increasing a commercial characteristic value if a telop is detected;

a character searching step, comprising the steps of searching for a character in the video signal of the commercial candidate section and of increasing a commercial characteristic value if a characters is detected;

a quietness rate determining step, comprising the steps of determining the quietness rate of the commercial candidate section and of increasing a commercial characteristic value if the quietness rate is below a predetermined level; and

a judgement step of judging whether or not the commercial candidate section is a commercial, based on the commercial characteristic value after the preceding steps are performed.

51. A method of processing a television signal, wherein the television signal comprises an audio signal and a video signal, the method comprising:

a receiving step of receiving the television signal;

a determining step of determining a quiet threshold value from the television signal;

a comparing step of making a comparison of the audio signal and the quiet threshold value; and

a judging step of judging whether or not a portion of the television signal is a commercial candidate section according to the comparison.

52. The method of claim 51, wherein the television signal comprises scene changes and wherein the determining step comprises:

a step of detecting the scene changes;

a step of detecting the audio signal during the scene changes; and

a step of determining the quiet threshold value based on the audio signal at the scene changes.

53. The method of claim 51, wherein the television signal comprises an audio multiplex mode signal and wherein the determining step comprises:

a step of detecting the audio multiplex mode from the audio multiplex mode signal;

a step of detecting the audio signal when the audio multiplex mode changes; and

a step of determining the quiet threshold value based on the audio signal at the time the audio multiplex mode changes.

54. The method of claim 51, wherein the television signal comprises a program start time and wherein the determining step comprises:

a step of detecting the program start time;

a step of detecting the audio signal at the program start time; and

a step of determining the quiet threshold value based on the audio signal at the program start time.

55. The method of claim 51, wherein the television signal comprises a program end time and wherein the determining step comprises:

a step of detecting the program end time;

a step of detecting the audio signal at the program end time; and

a step of determining the quiet threshold value based on the audio signal at the program end time.

56. The method of claim 51, wherein the television signal comprises an AGC signal and wherein the determining step comprises:

a step of detecting the AGC signal;

a step of detecting the audio signal; and

a step of determining the quiet threshold value based on the audio signal and the AGC signal.

57. The method of claim 51, wherein the video signal comprises a brightness level and wherein the determining step comprises:

a step of determining the brightness level;

a step of detecting the audio signal; and

a step of determining the quiet threshold value based on the audio signal and the brightness level.

58. The method of claim 51, wherein the television signal is organized in channels and wherein the determining step comprises:

a selecting step of selecting each channel of the television signal; and

a calculating step of calculating the quiet threshold value based on the audio portion of each channel of the television signal selected by the selecting step.

59. The method of claim 58, wherein the calculating step calculates the quiet threshold value at a predetermined time interval.

60. The method of claim 58, further comprising:

a storing step of storing a previous quiet threshold value;

a quiet threshold value comparing step of determining whether the new quiet threshold value based on the audio portion of each channel of the television signal is smaller than the previous quiet threshold value; and

a step of outputting the new quiet threshold value to the storing step if the new quiet threshold value is smaller than the previous quiet threshold value.

61. A provision medium for providing a program which is readable by a computer to control an apparatus to execute a detection routine for detecting commercials included in a television broadcast, the detection routine comprising:

a receiving step of receiving a television signal;

a determining step of detecting characteristics of the commercial candidate section; and

62. The provision medium of claim 61, wherein:

the commercial candidate section comprises scene changes; and

63. The provision medium of claim 61, wherein:

the second determining step comprises the step of determining the periodicity of the audio signal in the commercial candidate section.

64. The provision medium of claim 61, wherein:

the determining step comprises the step of determining a level of continuity of the periodicity of the audio signal in the commercial candidate section.

65. The provision medium of claim 61, wherein:

the determining step comprises the step of determining a level of repetition of the video signal in the commercial candidate section.

66. The provision medium of claim 61, wherein the determining step comprises the step of determining whether a telop exists in the video signal of the commercial candidate section.

67. The provision medium of claim 61, wherein the determining step comprises the step of determining whether a character exists in the video signal of the commercial candidate section.

68. The provision medium of claim 61, wherein the determining step comprises the step of determining a quietness rate of the commercial candidate section.

69. The provision medium of claim 61, wherein:

the television signal comprises quiet frames; and

70. A provision medium for providing a program which is readable by a computer to control an apparatus to execute a detection routine for detecting commercials included in a television broadcast, the detection routine comprising:

a receiving step of receiving a television signal;

71. A provision medium for providing a program which is readable by a computer to control an apparatus to execute a detection routine for detecting commercials included in a television broadcast, the detection routine comprising:

a receiving step of receiving the television signal;

72. The provision medium of claim 71, wherein the television signal comprises scene changes and wherein the determining step comprises:

a step of detecting the scene changes;

a step of detecting the audio signal during the scene changes; and

73. The provision medium of claim 71, wherein the television signal comprises an audio multiplex mode signal and wherein the determining step comprises:

a step of detecting the audio signal when the audio multiplex mode changes; and

74. The provision medium of claim 71, wherein the television signal comprises a program start time and wherein the determining step comprises:

a step of detecting the program start time;

a step of detecting the audio signal at the program start time; and

75. The provision medium of claim 71, wherein the television signal comprises a program end time and wherein the determining step comprises:

a step of detecting the program end time;

a step of detecting the audio signal at the program end time; and

76. The provision medium of claim 71, wherein the television signal comprises an AGC signal and wherein the determining step comprises:

a step of detecting the AGC signal;

a step of detecting the audio signal; and

77. The provision medium of claim 71, wherein the video signal comprises a brightness level and wherein the determining step comprises:

a step of determining the brightness level;

a step of detecting the audio signal; and

78. The provision medium of claim 71, wherein the television signal is organized in channels and wherein the determining step comprises:

a selecting step of selecting each channel of the television signal; and

79. The provision medium of claim 78, wherein the calculating step calculates the quiet threshold value at a predetermined time interval.

80. The provision medium of claim 78, further comprising:

a storing step of storing a previous quiet threshold value;

a step of storing the new quiet threshold value if the new quiet threshold value is smaller than the previous quiet threshold value.