WO2010027250A2

WO2010027250A2 - Tracking and projection system and method thereof

Info

Publication number: WO2010027250A2
Application number: PCT/MY2009/000132
Authority: WO
Inventors: Ching Hau Chan; Woon Hon Hock; Sheau Wei Chan; Wooi Hen Yap; Siu Jing Then; Yen San Yong
Original assignee: Mimos Berhad
Priority date: 2008-09-03
Filing date: 2009-09-02
Publication date: 2010-03-11
Also published as: WO2010027250A3; MY171817A

Abstract

The present invention discloses a system and method for object tracking and image projection which may be used for a surveillance system or for providing information to users by way of a display adjacent to users within the monitored area. The system comprises a video camera (200), a light projector (208), housing for the light projector (210), a plurality of motors (202, 204, 206) for providing movements to the light projector (208), a support pole (216), and a mirror (212) for enabling the light projector (208) to illuminate or project images at the targeted object for tracking by way of reflecting therefore diverting the illumination from the light projector (208). There is further provided a control and processing system (300) connected to the projector (208) and the camera (200) for providing overall system control and further processing of the captured images by the camera (200). The system of the present invention provides continuous tracking of an object within the monitored area upon suitably prompted by the control and processing system (300).

Description

TRACKING AND PROJECTION SYSTEM AND METHOD THEREOF

Field of Invention

The present invention relates to a video camera system and more particularly to a video camera in combination with a display system enhanced by monitoring, tracking and displaying objects or images as an indication based on said monitoring and tracking; or for providing information within a predetermined area.

Background of Invention

Video camera equipment and system have been widely used since decades ago for various purposes, whereby such system is typically employed to residential, commercial, government and law enforcement agencies worldwide.

At present, one of the prominent uses relative to video camera equipments is to form a video security surveillance system. It is known that such system plays a significant role in providing a means for ensuring and increasing security in public places and within buildings. Most surveillance systems of the prior art necessitates a plurality of video cameras installed within the predetermined areas therefore providing a plurality of video motion pictures or views for display through connecting monitors which are normally controlled by an operator. Comprehendingly, for security purposes, the said operator carries out the monitoring task in a remote control room which is normally positioned or located at a predetermined distance from the monitored area, hi the event that potential hazard or illegal intrusion is detected from the plurality of motion views provided by the video cameras, the operator is obliged to notify the closest security personality of such event so as to investigate the occurrence. The notification step may involve verbal communication means.

One of the major concerns in the existing system as described above is the time required to notify the said security personality or any person in charge. A minor delay of time to notify the security personality to take action upon detected an intruder for instance, may result to a substantial gap of reaction time which results to delayed reaction, particularly in crowded areas. In addition, such circumstance may not be favorable for highly pertinent surveillance areas, as a delayed reaction is one of the crucial causative factors related to security failure.

It is commonly known and highly advantageous for the human society to have a security system which is able to provide immediate reaction for any type of unexpected motions within an area which may represent security problem.

Accordingly the latter shortcoming therefore puts forward various advances and improvements endeavors for innovators of the industry in providing systems offer efficiency in the said vital aspect as briefly described in the preceding paragraphs. However, existing prior arts have yet to effectively solve the tribulations in relation to the said criteria.

Accordingly, there is a great need to provide a video camera aided surveillance or monitoring system that is able efficiently alleviate the discussed drawback and is able to immediately bring the operator of the system and the relevant person in charge attention and thus to react promptly when required.

It is therefore a primary object of the present invention to provide a video camera aided tracking and projection system with the ability to monitor, detect and immediately notify the operator and the nearest authorized person on any unexpected motions or security problems related occurrences within a predetermined area or areas.

It is yet another object of the present invention to provide a video camera aided tracking and projection system that allows immediate reaction from the authorized person upon detection of any unexpected motions or security problems related occurrences within a predetermined area or areas.

It is further an object of the present invention to provide a video camera aided tracking and projection system to automatically and continuously tracking an object within the monitored area if prompted. These and other objects, features and advantages of the present invention will be apparent from the following detailed description of preferred embodiments thereof.

Summary of Invention

The present invention discloses a method of tracking an object and providing image projection within an area comprising the steps of: i) capturing images based on said area; ii) extracting objects within the captured images; iii) classifying objects as extracted from step ii); iv) assigning object identification based on object features and characteristics for each of the objects from step (iii); v) determining the objects field of view; vi) selecting an object for tracking based on the identification from iv); vii) aiming at the object for tracking from step vi); and viii) automatically and continuously tracking the object from step vii) or display an image adjacent to the object from step vii).

The invention further discloses a system for tracking an object and providing image projection within an area comprising: a support pole (216); at least one means for capturing images (200) fixed to at least one end of the said support pole (216); at least one means (208) for providing illumination and projection of images; and at least one means for reflecting the illumination from the said means for providing illumination and display of images (208). In accordance with another embodiment of the present invention there is provided at least one means (202) comprising a support member (202a) configured to provide horizontally rotational movement for the means for providing illumination and projection of images relative to the support pole (216); wherein the support member (202a) is configured to support the means for illumination and projection of images.

In yet another embodiment of the present invention there is further provided at least one means (204) configured to provide pan and tilting movements to the means for providing illumination and projection of images (208);

In another embodiment of the present invention there further at least one means (206) for providing tilting movements of the means for re-reflecting the illumination (212) from the means for providing illumination and projection of images (208);

In another embodiment of the present invention there is provided a control and processing system (300) for controlling the means for capturing images (200), means for providing illumination and display of images (208) and the means for providing horizontal, pan and tilting movements (202, 204, 206);

In accordance with another embodiment of the present invention the means for providing the horizontally rotational movement (202) and the support member (202a) for the means for providing illumination and projection of images (208) is positioned above the means for capturing images (200);

hi another embodiment of the present invention the means for providing illumination and projection of images (208) is positioned in a manner such that it is supported by the support member (202a) of the means for providing horizontally rotational movement (202) for the means for providing illumination and projection of images (208);

In accordance with yet another embodiment of the present invention, the means for reflecting the illumination from the means for providing illumination and projection of images (212) is positioned in a manner such that it is within the illumination path of said means for providing illumination and projection of images (208) for reflecting the illumination from the means for providing the illumination and display of images (208);

In accordance with yet another embodiment of the present invention, the means for providing illumination and display of images (208) illuminates an object or display an image adjacent to an object when prompted.

Brief Description of Drawings

The invention will be more understood by reference to the description below taken in conjunction with the accompanying drawings herein:

FIG 1 illustrates the preferred embodiments of the present invention;

FIG 2 illustrates the tilting and pan movements of the projector and mirror in one embodiment of the present invention;

FIG 3 shows the camera providing wide angle of monitoring within an area;

FIG 4 shows an operative effect of the present invention, whereby the light projector is prompted to display an image adjacent to an object which is an intruder;

FIG 5 shows another operative effect in one embodiment of the present invention, whereby the light projector is prompted to continuously track and display the image adjacent to the intruder;

FIG 6a - FIG 6b show another operative effect in one embodiment of the present invention whereby the light projector is prompted to illuminate an object; and to display informative images to patrons within the monitored area. FIG 7 shows a flowchart for steps involved in tracking and projection abilities based on one embodiment of the present invention.

Detailed Description of Preferred Embodiments

In line with the above summary, the following description of a number of specific and alternative embodiments is provided to understand the inventive features of the present invention. It shall be apparent to one skilled in the art, however that this invention may be practiced without such specific details.

The system of the present invention as shown in FIG 1 comprises of a video camera (200), a first motor (202) with a support member (202a) attached thereto, a second motor (204), a third motor (206), light projector (208), a housing (210) for the light projector (208) a mirror (212) and a support pole (216).

There is further included an electronic device or the likes referred herein as the control and processing system (300) for performing the important tasks including image processing, system control and display of captured images with respect to the surveillance area to the operator, the said electronic assembly or device may be in the form of, but not confined to, a computer system suitably in communication or connected by way of suitable cables or wirelessly to the video camera (200) and light projector (208) of the system. The control and processing system (300) is configured with suitable software which allows remote controlling of the camera (200) and the motors (202,204,206), whereby the movements of the motors (202,204,206) are dependant on the observation of the operator. The system (300) of the present invention is further configured to be able to classify and track objects within a single image or sequences of images, which will be described in detail herein with reference to the flowchart as shown in FIG 7.

The camera (200) is preferably a wide angle digital camera disposed in a dome- shaped portion which is attached to a base portion, whereby said camera (200) is able to provide a substantially wide coverage within an area. The primary function of the camera (200) is to capture images and is connected to the control and processing system (300) for providing the captured images to the control and processing system (300). The camera (200) is configured to be accordingly fixed at the base section to at least one end of a support pole (216), in a manner such that the dome-shaped portion extends downwardly relative to the base portion if the support pole (216) is fixed on the ceiling for instance.

Preferably, the support pole (216) is formed from a suitable material into an elongated cylindrical member. The interior portion of the support pole (216) may be hollowed so as to channel the connecting cable or the likes from the control and processing system (300) to the camera (200). Another essential embodiment of the present invention is the light projector (208).

The light projector (208) may be a standard or conventional light projector having at least one end for light emission, and adapted for connection to power supply and to the control and processing system (300) illumination or projection of images to a location or at one point within an area.

A housing means (210) for the projector (208) is formed from a suitable material into a shape but not limiting to a hollowed rectangular member. The rectangular member therefore comprises four side panels, a base section and at least one open end, said open end for allowing the emission or illumination by the projector (208) upon disposed therein. It is formed with predetermined dimension sufficient to accommodate or support the light projector (208). It is understood that the housing (210) may further house the cables, power supply or the likes of the light projector (208). Additionally, at least one section, preferably at least one side panel of the rectangular member is formed extensively at a greater height than that of the other side panels therefore extending upwardly relative to the base section of the rectangular member.

There is further provided a mirror (212) configured for re-directing the light emission or illumination from the projector (208) to another position for accurate aiming or tracking of the selected object. The said mirror (212) may be in various shapes, provided that is it able to reflect or re-direct the illumination from the projector (208) to the desired spot or angle. As seen in FIG 1 and FIG 2, the system further comprises three motors (202, 204,

206), said motors are configured to be attached or connected to the light projector (208) and the housing means (210) in such that the motors (202, 204, 206) is able to provide horizontal rotation, pan and tilt movements for the light projector (208), and with the mirror (212) attached to one of the motors (206) there is provided continuous tracking of an object within the camera's (200) view. The motors used for the purpose of the present invention are conventional electric motors or the likes which is suitably configured for converting electric energy into mechanical energy and thus providing physical movements for respective components of the present invention. It is apparent for a person skilled in the art that any form of motion devices may substitute said motors.

Suitably, the movements of said motors (202, 204, and 206) may be controlled remotely by an operator of the system via the control and processing system (300), whereby the movements and orientations are dependant on the selected object for tracking, which is determined by the operator based on his monitoring or captured views from the camera (200).

Following the above and now referring to FIG 2, the first motor (202) with the support member (202a), which is preferably, but not confined to, a disc -like or disc- shaped member, is fixed to the support pole (216) with suitable fastening means and is positioned above the camera (200). In this position, the support pole (216) may serve as an axial point with respect to the support member (202a) which is the disc-like member and the first motor (202) movements during use. The second motor (204) is in contact with the first motor (202), and is connected to the projector housing means (210) at one end. In combination, it is noted that the first (202) and second (204) motors play a major role in allowing the projector (208) to rotate horizontally in addition to the pan and tilt movements of the light projector (208).

In order to provide horizontal and pan movements to the light projector (208) the first motor (202) is suitably attached or connected to the disc-like member (202a) such that it is able to support and therefore provide the horizontally rotational movement as shown in FIG 2, using the pole (216) as a rotation axis.

The third motor (206) is attached to the mirror (212), so as to enable the tilting movement of said mirror (212) while re-directing or reflecting the emission or illumination from the light projector (208) during use.

Upon assembled, at least one end of the support pole (216) is fixed or mounted to the support structure which may include a ceiling structure or any of the likes, and one end of said pole (216) supports the camera (200) and the motors (202,204,206). In the event that the support pole (216) is mounted to the ceiling of a building, camera (200) of the assembly (100) is directed downwards or at an opposite direction to that of the ceiling. As briefly described in the preceding paragraphs and still referring to FIG 2, the first motor (202) is connected to the disc- like member (202a) such that it is able to provide horizontal rotational movement of the said disc- like member (202a). The second motor (204) is positioned or supported by the disc like member (202a) and in contact with the base section of the housing (210) for the projector (208) in a manner such that the second motor (204) is able to provide tilting movements for the housing (210) and therefore, providing corresponding movements for the light projector (208).

The third motor (206) is secured at the tip end of the extended side panel of the housing (216), as seen in FIG 1 and FIG 2. Coupled to the third motor

(206) is the mirror (212), whereby the mirror (212) is attached to the motor (206) in a manner such that the motor (206) is able to provide tilting movements for the mirror (212). In this position, the mirror (212) is positioned relative to the projector (208) such that the reflective surface of the mirror (212) is preferably positioned facing the light or illumination outlet of the projector (208), therefore within the light or illumination path of said projector (208).

It is noted that the tilting movements of the mirror (212) depend significantly on the tilting movements of the projector (208) provided by second motor (204). This is because the mirror (212) should be in a position which enables it to reflect the emission or illumination for the projector (208) and thus re-directing the illumination to the desired positioned. Accordingly, the illumination direction of the projector (208) is therefore diverted by the reflecting surface mounted on said motor (206).

It is noted that the movements of the light projector (208) and the mirror (212) may be synchronized remotely by the operator or by automated synchronization.

The camera (200) and light projector (208) are suitably in communication with a control and processing system (300) for image processing, display of the captured images and for overall system control. It is apparent for the person skilled in the art that the role of the control and processing system (300) with respect to the invention may not be confined to the aforementioned. Comprehendingly, signals and captured images are sent to the control and processing system (300) for processing and thus providing information to the operator.

During use, the camera (200) provides a wide angle monitoring or coverage area as suitably seen in FIG 3. The operator therefore is able to monitor the predetermined area based on the view of the camera (200). Referring now to FIG 4 as an exemplary for the operative effect of the present invention, in the event that there is suspected intrusion within the monitored area based on the images captured by the camera (200), the operator immediately notifies the authorized person within the said monitored area by way of illuminating a "danger" sign or the likes within the projector (208) adjacent to the suspected object or person. This way, the authorized person is warned immediately and thus reacts promptly to the situation. In this process, the motors (202, 204, and 206) are suitably operated such that the desired position to provide the illumination is achieved. Further, the said object or person is continuously tracked devoid of any interruption with the assistance of the illumination or emission from the projector (208) as shown in FIG 5.

In another instance of occurrence, the operator may detect dropped object or hazardous object located on the ground within the monitored area. In order to notify the authorized person within the said monitored area on the said object, the operator may highlight the object by way of directing the emission of light on the said object which would grab the immediate attention of the authorized person or any nearest person. This course of action may also apply in the event that there is suspected theft, burglary, shoplifter, or the likes, whereby as shown in FIG 6a, upon detected, the operator may highlight the suspected person which would therefore notify the authorized person to immediately react. The highlighted or illuminated person or thief will appear obvious to the nearest security person within the monitored area even in crowded areas, and consequently the security person can act immediately. In the event that the thief attempts to run off, the projector (208) tracks and thus continuously highlights the thief which therefore serves as a lead for the security to trail the thief.

It is understood that the illumination and display images may range from words to colors, whichever is suitable for the circumstance. With such notifications or display, there is provided prompt interactions between the operator and the authorized person(s) within the monitored area. Another usage of the system in accordance with the preferred embodiments of the present invention is to notify or provide information to the patrons within the monitored area as shown in FIG 6b, whereby the said information may include advertisements on current sales, current headline news, information on exhibits for an art gallery, and many more. The system may also be deployed for interactive games, whereby with the tracking feature, the system may display virtual characters which may be interacted based on the participant's movements.

FIG 7 provides the flowchart related to the tracking and projection process in accordance with another preferred embodiment of the present invention. Referring to

FIG 7, the captured images of the monitored area from the camera (200) are forwarded to the control and processing system (300). The said control and processing system (300) is configured with the appropriate software to filter the background and extracts the moving binary large objects or blobs. These blobs may contain audio, images or other related multimedia objects captured within the monitored area. The blobs are accordingly classified, for image content acquisition whereby in this classifying step, each nature of each blob is determined, as in whether the blob is human, vehicle or other unrecognized objects, for instance. In the event that the blob is classified as a human or vehicle, the system then proceeds to assign an identification number or the likes to the objects which may be human or vehicle or etc. It is noted that the classification and assigning the identification number of objects may be performed based on the characteristics and features specified by users. Subsequently, the system determines the detected objects within the field of view, which may be within the 180° angle of direction. The current and previous locations of the said object are taken into account to confirm the field of view.

Each of the objects within the field of view are accordingly tracked and forwarded for a response containing the relevant input from the user. If there is no user input on the tracked object, the said object may be ignored and another tracked object if forwarded to request for input or further instructions from the operator. This step is repeated for every object within the field of view.

In the event that the operator identifies an intruder, captured images containing views of the intruder is selected. The operator immediately lock the intruder as the object targeted for tracking and appropriately prompted the system for next course of action, whereby the projector (208) may be prompted to aim at the object prior to providing an image display or object illumination. The object aiming is aided by the motors (202, 204, and 206). The locking of target may be based on the features and characteristics constrained by the operator when acquired the object for tracking. In the event that the object illumination is selected, the projector (208) is prompted to aim at the center region of the targeted object and thus highlighted. It is noted that with the constraints provided by the operators on the features or characteristics of the targeted object, the system therefore locks the target and thus automatically tracks the object within the monitored area until appropriate action is taken by the authorized person within the said monitored area. Alternatively, if the operator selects the image display, the system is therefore prompted to search for a flat area to display the desired image. In this process, the pixels in the targeted object's field of view are accordingly extracted and the field of view is accordingly segmented into five sections. Based on the five sections, the texture complexity for each section is calculated in order to determine any smooth area or any section with the least complex textures. Upon located the flat area, the projector (208) is prompted to aim at the flat area and subsequently the desired image is accordingly displayed. Suitably, the image display may be rotated or adjusted based on the position of the selected object.

It is understood that any one of steps involved in the tracking and projection process as described above may be repeated in order to achieve the desired results.

Although illustrative embodiments of the present invention have been described herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various other changes and modifications may be effected therein, without departing from the scope of invention.

Claims

1. A method of tracking an object and providing image projection within an area comprising the steps of: i) capturing images based on said area; ii) extracting objects within the captured images; iii) classifying objects as extracted from step ii); iv) assigning object identification based on object features and characteristics for each of the objects from step (iii); v) determining the objects field of view; vi) selecting an object for tracking based on the identification from iv); vii) aiming at the object for tracking from step vi); and viii) automatically and continuously tracking the object from step vii) or display an image adjacent to the object from step vii).

2. A method as claimed in Claim 1 wherein the method further includes the step of filtering the background of the captured images.

3. A method as claimed in Claim 1 wherein the tracking of object may include continuously illuminating the object .

4. A method as claimed in Claim 1 to 2 wherein illuminating the object and projection of images adjacent to an object is performed using a projector (208).

5. A method as claimed in Claim 1 wherein the capturing images is performed using a camera (200).

6. A system for tracking an object and providing image projection within an area comprising: a support pole (216);

at least one means for capturing images (200) fixed to at least one end of the said support pole (216);

at least one means (208) for providing illumination and projection of images;

at least one means for reflecting the illumination from the said means for providing illumination and projection of images (208);

at least one means (202) comprising a support member (202a) configured to provide horizontally rotational movement for the means for providing illumination and projection of images (208) relative to the support pole (216), and wherein the support member (202a) is further configured to support the means for illumination and projection of images (208); at least one means (204) for providing pan and tilting movements to the means for providing illumination and projection of images (208);

at least one means (206) for providing tilting movements of the means for reflecting the illumination (212) from the means for providing illumination and projection of images (208);

a control and processing system (300) for controlling the means for capturing images (200), means for providing illumination and display of images (208) and the means for providing horizontal, pan and tilting movements (202, 204, 206);

wherein the means for providing the horizontally rotational movement (202) comprising the support member (202a) for the means for providing illumination and projection of images (208) is positioned above the means for capturing images (200);

wherein the means for providing illumination and projection of images (208) is positioned in a manner such that it is supported by the support member (202a) of the means for providing horizontally rotational movement (202) for the means for providing illumination and projection of images (208);

wherein the means for reflecting the illumination from the means for providing illumination and projection of images (212) is positioned in a manner such that it is within the illumination path of said means for providing illumination and projection of images (208) for reflecting the illumination from the means for providing the illumination and projection of images (208);

wherein the means for providing illumination and display of images (208) illuminates an object or display an image adjacent to an object when prompted.

7. A system as claimed in Claim 6 wherein there is further included a housing (210) for the means for providing illumination and display or images (208).

8. A system as claimed in Claim 7 wherein the housing (210) is a rectangular member having at least one side panel being formed with a greater height than that of the other side panels.

9. A system as claimed in Claim 6 wherein the control and processing system (300) is further used for processing captured images and provides display to the operator or user of the system.

10. A system as claimed in Claim 6 wherein the pan and horizontally rotational movement of the means for providing illumination and display of images (208) is a disc like member (202a) provided with a first motor (202), means for providing the tilting movement of the means for providing illumination and projection of images is a second motor (204) and the means for providing tilting movement of the means for reflecting the illumination and projection (212) is a third motor (206)

11. A system as claimed in Claim 10 wherein the third motor (206) and the means for reflecting the illumination from the means for providing illumination and projection of images (208) is attached at the tip portion of one side panel formed with a greater height than that of the other panels of the housing (210).

12. A system as claimed in Claim 6 wherein the support member (202a) of the means for providing horizontally rotational movement for the means for providing illumination and projection of images (208) is a disc-like member.

13. A system as claimed in Claim 6 wherein the means for capturing images is a camera (200).

14. A system as claimed in Claim 6 wherein the means for providing illumination and projection of images (208) is a projector.

15. A system as claimed in Claim 6 wherein the means for reflecting the illumination from the means for providing illumination and projection of images (208) is a mirror (212).