WO2002025940A1

WO2002025940A1 - Picture-in-picture

Info

Publication number: WO2002025940A1
Application number: PCT/EP2001/010319
Authority: WO
Inventors: Miroslav Trajkovic
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2000-09-20
Filing date: 2001-09-06
Publication date: 2002-03-28

Abstract

A television having a processor that increases the viewability of a picture-in-picture display (PIP) by producing a tracking window in a primary image display area and tracking a corresponding image portion in the PIP. The television has a user input device that receives a user input to enable a user to move the tracking window to any given portion of the primary image display area. The user input also enables a user to adjust the size of the tracking window and to adjust a zoom level of the PIP.

Description

This invention generally relates to a method and device to enhance home television usage. Specifically, the present invention relates to a picture-in-picture display.

It is very common for televisions to have a capability of displaying more than one video display on the television display at the same time. Typically, the display is separated into two or more portions wherein a main portion of the display is dedicated to a selected video data stream (e.g., a given television channel). A second video data stream is simultaneously shown in a display box that is show as an inset over the display of the first data stream. This inset box is typically denoted as a picture-in-picture display ("PIP"). This PIP provides the functionality for a television viewer to monitor two or more video data streams at the same time. This may be desirable for instance at a time when a commercial segment has started on a given television channel and a viewer wishes to "surf additional selected television channels during the commercial segment, yet does not wish to miss the return from the commercial segment. At other times, a viewer may wish to search for other video content or just view the other content without missing content on another selected channel.

In any event, PIP has a problem in that the PIP is typically shown in an inset box that is substantially smaller than the entire television display. Techniques are known to produce the reduced size PIP, however, with this reduction in size there is a reduction in the viewability of the displayed image. People buy larger television sets to increase the size of displayed images so that the images are in effect, more viewable. Yet, with the addition of PIP features into televisions, a smaller, less viewable image has been introduced on to these larger displays. These reduced PIP images produce a marked reduction in the ability of a viewer to follow events or content flow of selected content.

Accordingly, it is an object of the present invention to overcome the disadvantages of the prior art. It is a further object of the present invention to provide a method and device for increasing the viewability of PIP images. To this end, the invention provides what is defined by the independent claims. The dependent claims define advantageous embodiments.

In a preferred embodiment, a television has a processor that increases the viewability of a picture-in-picture display (PIP) by producing a tracking window in a primary image display area and thereby, track a corresponding image portion in the PIP. The television has a user input device that receives a user input to move the tracking window to any given portion of the primary image display area. The user input also enables a user to adjust the size of the tracking window and to adjust a zoom level of the PIP.

In one embodiment, the processor captures an image frame in the primary image display area for placement of the tracking window. The processor may utilize known object recognition techniques to identify the image portion corresponding to the position of the tracking window. These techniques may be further utilized to size the tracking window to the identified image portion.

The following are descriptions of embodiments of the present invention that when taken in conjunction with the following drawings will demonstrate the above noted features and advantages, as well as further ones. It should be expressly understood that the drawings are included for illustrative purposes and do not represent the scope of the present invention. The invention is best understood in conjunction with the accompanying drawings in which: FIG. 1 shows an illustrative image tracking system in accordance with an embodiment of the present invention;

FIG. 2 A shows an enlarged view of a display at an initial time Tl containing illustrative objects than can be tracked in accordance with an embodiment of the present invention; and FIG. 2B shows an enlarged view of the display at a time T2 subsequent to the initial time Tl. In the discussion to follow, certain terms will be illustratively discussed in regard to specific embodiments or systems to facilitate the discussion. However, as would be readily apparent to a person of ordinary in the art, these terms should be understood to encompass other similar applications and embodiments wherein the present invention could be readily applied.

FIG. 1 shows an illustrative image tracking system 100 in accordance with an embodiment of the present invention including a display 110, operatively coupled to a processor 120, and a remote control device 130. The processor 120 and the remote control device 130 are operatively coupled as is known in the art via an infrared (IR) receiver 125, operatively coupled to the processor 120, and an IR transmitter 131, operatively coupled to the remote control device 130.

The display 110 may be a television receiver or other device enabled to reproduce audiovisual content for a user to view and listen to. The processor 120 is operable to produce a picture-in-picture display (PIP) on the display 110 as is know by a person of ordinary skill in the art. Further, the processor 120 is operable to provide a PIP display in accordance with an embodiment of the present invention.

The operation of the illustrative system shown in FIG. 1 will be described herein below with reference to FIGs. 2 A, 2B. The remote control device 130 contains buttons for operation in accordance with the present invention. Specifically, the remote control device 130 contains a PIP button 134, a swap button 132, a tracking window button 133, tracking window sizing buttons 136A, 136B, tracking window zoom buttons 135A, 135B, and tracking window position control buttons 137A, 137B, 137C, 137D. The PIP button 134 initiates a PIP function to open a PIP 210A (e.g., see, FIG. 2A) on the display 110 and the swap button 132 swaps each of the images shown on the PIP 210A portion and on a primary display portion 210B of the display 110 as is known.

FIG. 2 A shows an enlarged view of the display 110 at an initial time Tl containing illustrative objects than can be tracked in accordance with an embodiment of the present invention. Specifically, objects such as an airplane object 220, a truck object 230, a walking person object 240, and a running person object 250 are shown in the primary display portion 210B . It should be expressly understood that although the present invention is described with regard to objects, a person of ordinary skill in the art would readily appreciate that other picture portions may be tracked and still be well within the bounds of the present invention. Specifically, it is known in the art that color(s), contrast, object detection, motion, etc. may be utilized by a processor, such as the processor 120, for tracking a given portion of a video image amongst a sequence of frames. Further, it is known that even multiple color portions may be tracked simultaneously amongst a sequence of frames. Much attention in the prior art has also been paid to the tracking of a human body amongst a sequence of frames. Any of the above-mentioned systems for tracking a given portion of a video image from a video data stream (e.g., a television data stream) may be utilized in accordance with the present invention.

The operation of a tracking system in accordance with an embodiment of the present invention can be described as follows. First, a user initiates an action, such as depression of the tracking window button 133, to bring up a tracking window 260. In FIG. 2 A, the tracking window 260 is shown centered within the primary display portion 210B. The centered position of the tracking window 260 may be the initial position after the user depresses tracking window button 133 or it may be the position after the user has positioned the tracking window 260 utilizing the window position control buttons 137A, 137B, 137C, 137D. The window position control buttons 137A, 137B may respectively adjust the tracking window 260 up and down. The window position control buttons 137C, 137D may respectively adjust the tracking window 260 left and right.

Next, the user sizes the tracking window 260 utilizing the window sizing buttons 136A, 136B. For example, the window sizing button 136A may enlarge the size of the tracking window 260 and the window sizing button 136B may reduce the size of the tracking window 260. By sizing the tracking window 260, the user adjusts the portion of the video image shown that the system, in accordance with the present invention, will track in the PIP 210A. It should be noted that although the tracking window is illustratively shown and described as a window surrounding an object that is to be tracked, in fact the tracking window may not surround the object. In some embodiments, the tracking window may actually be a single indicated point, or a grouped plurality of points on the primary display area 210B. In these embodiments, the tracking window may not surround an object or region of interest. In either case, the device in accordance with the present invention may utilize well known object detection and identification techniques to determine the bounds of a selected object. In this way, the device in accordance with the present invention may determine the bounds of a selected object, region, etc. for tracking in the PIP 210A. The single indicated point may be a result of reducing the size of the tracking window 260 utilizing window sizing button 136B, or it may be a result of a predetermined initial condition for the tracking window 260. Accordingly, the term tracking window as utilized herein should be understood to encompass these and other related visual indications for facilitating a user to select what they wish to track. Clearly, the tracking window 260 is envisioned to encompass any indication that may range from a single point to any portion of the primary display area 210B and may take on any suitable shape. For instance, in one embodiment, after a user selects a point for the tracking window 260, the system in accordance with the present invention may perform an object recognition step and automatically size the tracking window to the area of interest. In this case, the tracking window may be presented visually as a highlighting or other indication around the recognized object. Automatically placing the visual indication of the tracking window 260 around a selected portion of the video image may facilitate the user identifying and repositioning the tracking window in the event that an incorrect portion of the video image is recognized by the system.

Next, the system tracks the selected portion of the video image in the PIP 210A. The truck object 230 is shown coincidentally centered within the tracking window 260 in the primary display portion 210B. However, as should be clear from the discussion above, the tracking window 260 may be positioned on any portion of the primary display area 210B. In accordance with the present invention, the truck object 230 is tracked and is shown centered in the PIP 210A. Further, the PIP 210A illustratively shows a portion of the airplane object 220 merely due to the airplane objects relative position with regard to the tracked truck object 230.

As should be clear from FIG. 2 A, the PIP 210A is not an ordinary display of the television image as is known in the art. In accordance with the present invention, the PIP 210A shows an enlarged portion of the television image with a selected object (e.g., the truck object 230) tracked and centered within the PIP 210A. As discussed above and as should be readily appreciated, other portions of the television image may also be tracked in accordance with a present invention by the user selecting the other portions of the television image utilizing the tracking window 260.

Next, the user may optionally adjust the degree of zoom that is shown in the PIP 210 through use of the zoom buttons 135 A, 135B. Electronic zoom techniques are known in the art wherein additional pixels may be interpolated to produce a zoomed image (e.g., an enlarged image). The zoom button 135A may increase the degree of zoom of the selected portion of the television image shown in the PIP 210A, thereby increasing the size of the selected portion in the PIP 210A. In accordance with the present invention, this increases the viewability of a portion of the television image of interest. In one embodiment, depression of the zoom button 135 A may also increase the size of the PIP 210A. The zoom button 135B may decrease the degree of zoom of the selected portion of the television image shown in the PIP 210A and/or decrease the size of the PIP 210A.

FIG. 2B shows an enlarged view of the display 110 at a time T2 subsequent to the initial time Tl shown in FIG. 2 A. In FIG. 2B, at the time T2, the objects shown have changed position with regard to the objects position shown at the time Tl . However, in accordance with a present invention, the truck object 230 is still shown centered within the tracking window 260 and the PIP 210A even though the relative position of the truck object 230 has moved to the right of the primary display portion 210B. Further, the position of the airplane object 220 has changed relative positions in the primary display 210A as well as in the PIP 210B since the airplane object 220 has not been selected by the user for tracking. Next, the system in accordance with the present invention will continue to track the selected portion of the television image as long as the selected portion is within the primary display portion 210A. Tracking of the selected portion is discontinued due to the selected portion no longer being present in the primary display portion 210A. Of course, tracking of the selected portion may be discontinued at anytime by the user once again depressing the tracking window button 133.

In accordance with another embodiment of the present invention, when the user first depresses the tracking window button 133, a frame of the displayed television image is captured by the processor 120 and is continuously displayed on the display 110. In this way, placement of the tracking window 260 is facilitated in the event of a quickly changing television image, such as a television image of a football game, etc. Once the user selects the portion of the video image that the user wishes to track, the primary display area may revert back to displaying the video data stream.

Finally, the above-discussion is intended to be merely illustrative of the present invention. Numerous alternative embodiments may be devised by those having ordinary skill in the art without departing from the scope of the following claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means can be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

8 29.05.2001CLAIMS:

1. A television comprising: a display (110) having a primary image display area (21 OB) and a picture-in-picture display area (PIP) (210A); and a processor (120) operatively coupled to the display (110) and configured to produce a tracking window (260) in the primary image display area (210B) and to track a corresponding image portion in the PIP (210 A) in response to user input.

2. The television of Claim 1 , comprising a user input device (130) configured to receive a user input and to provide signals to the processor (120) to move the tracking window (260) to any given portion of the primary image display area (210B).

3. The television of Claim 1 , comprising a user input device (130) configured to receive a user input and to provide signals to the processor (120) to adjust the size of the tracking window (260).

4. The television of Claim 1 , comprising a user input device (130) configured to receive a user input and to provide signals to the processor (120) to adjust a zoom level of the PIP (210 A).

5. The television of Claim 1 , wherein the processor (120) is configured to capture an image frame in the primary image display area (210B) for placement of the tracking window (260).

6. The television of Claim 1 , comprising a user input device (130) configured to receive a user input and to provide signals to the processor (120) to adjust the position of the tracking window (260) in the primary display area (210B), wherein the processor (120) is configured to identify the image portion corresponding to the position of the tracking window (260).

7. The television of Claim 6, wherein the processor (120) is configured to size the tracking window (260) to the identified image portion.

8. An image signal processing comprising: - placing a tracking window (260) over a portion of a video image displayed in a primary display area (210B) of a television display; tracking a corresponding portion of the video image in a picture in picture display (PIP)(210A).

9. The method of Claim 8, comprising capturing an image frame in the primary image display area (210B) for placement of the tracking window (260).

10. A software program sequence stored on a computer readable medium, the software program sequence comprising program portions for enabling a processor to carry out the method of claim 8.

11. A user input device (130) configured to receive a user input and to provide signals to move a tracking window (260) to any given portion of a primary image display area (210B); and/or - to provide signals to adjust a size of the tracking window (260), and/or to provide signals to adjust a zoom level of a PIP (210A).