An intelligent method and an intelligent system for integrated TV messaging or advettising.
DESCRIPTION
Field of the invention
The present invention relates generally to the field of advertising at events, at place of location, commercials integrated within subsequent TV broadcasts of such activities, the combination of advertisements at location and the broadcasting of the combined message (advertisements at location and the event) and framework for the control and management of such an advertisement system, where the commercial messages may be targeted according to the socio-economic profile(s) of the audience.
In particular, the invention relates to a method for integrated TV messaging in live event broadcasting and in later re-runs from such events, using at least one TV camera at an event site, where at least one TV camera is able to establish at least one parameter among environmental and behavioural parameters, and constitutes an obligatory sensor in a set of at least one sensor among environmental sensors and behavioural sensors at the event site. The at least one sensor is connected to a control unit.
The present invention is a further development of the applicant's prior Norwegian Patent No. 316485 and International Patent Application PCT/NO03/00308.
Background The challenge to transmit and distribute images from events electronically to a distant audience, from a mobile or a fixed studio, in a timely matter, like activities at sports arenas or fields, concert locations, arrangements at a field for horse riding, car or racing competitions, city or rural activities, natural or in nature activities involving humans, animals, machines or vehicles related to sports, leisure, work or the like, to relay images or activities from one spot to another within a compound like a shopping mall, building, sports arena or field, one needs a rather sophisticated broadcasting facility. From outside a fixed studio this necessitates an Outside Broadcasting (OB)
facility, including selection of technical and production arrangement, TV or image signal contribution and direct transmission by both terrestrial landlines and/or satellite. Such a facility will most likely involve, but is not limited to, a Mobile Control Room (MCR) for the coordination of the events in question, mobile editing suites, portable laptop edit suites, accompanying software with "fly away" capabilities, cameras, video machines, virtual playback, super slow motion systems, standard lighting equipment and specialized systems like intellabeams (intelligent lighting systems) and digital dimmers, mobile generators, microwave links in various bands etc. To provide electronic advertisements from such events, traditional means have been to interrupt ordinary programming with commercial breaks, fit in advertisements in natural stops of the event, like the breaks between rounds in boxing matches, or to provide passive signs, mechanical changeable boards or even non-intelligent electronic boards, displays or stationary signs at the location, or fixed on the objects in question, like printed advertisements on racing cars, on clothing on football players, skiers (downhill, cross country etc.), artists etc.
Prior art US patent No. 6,408,278 discloses an information display system for displaying still pictures or video, and the system comprises among other things a control unit that chooses display content in accordance with inter alia socio- economic or behavioural parameter values, that is with targeting of display content toward different groups of individuals. Norwegian patent No. 316.485, belonging to the proprietor of the present invention, discloses a system for "out of home" advertising by means of e.g. display boards at sites where both behavioural, socio-economical and environmental parameters are considered when changing advertising messages in the display boards. This prior art does however not concern TV broadcasting situations. Some prior art related to broadcasting and TV advertising, not related to the interruption of regular programming or the viewing of passive signs or mechanical boards at the sports arena, does exist. WO 01/54401 relates to a video-signal, for instance a TV-signal, for the transmittance via a broadcasting system, which is adapted to represent a scene for transmitting via the broadcasting system, and which is adapted to reproduce in the scene a part-image which fills a part of the image and which is independent of the scene, wherein the part-image represents an advertising message.
US Patent Application No. 20020080272 describes an information serving system including a broadcasting station module (TV), user module, and advertising agency module. The broadcasting station module broadcasts pieces of broadcast information including at least video information. The user module receives the broadcast information, selects an image component composing an image from the received broadcast information, and produces specification information indicative of the selected image component. This allows a viewer viewing a broadcasted program to acquire detailed information about a desired image component. In US Patent Application No. 20020072966 each advertiser defines its own target groups based on the preferred characteristics of the advertiser to form advertiser-specific target groups. Each person who is enabled to receive the advertisements (e.g., a subscriber to a cable TV service) is associated with appropriate target groups of the different advertisers, and this target group information may be stored at the subscriber's side or accessible by the subscriber's device such as a set top box or PVR. Based on the stored information, the advertisements that have been associated with the target groups of the subscriber, are then selected and displayed to the subscriber on a TV. US Patent Application No. 20020087978 WO 0254754 is strongly related to US Patent Application No. 20020072966 in the sense that it necessitates a subscription set top box and associated subscriber channel. The set top box monitors for triggers to provide advertisements to the subscriber's television. The advertisements can be downloaded in advance of any advertisement opportunity triggers to a local storage means associated with the set top box, if available. WO 0064165 is also subscriber based. It outlines a system for managing advertisements in a digital video environment, including methods for selecting suitable advertising based on subscriber profiles, and substituting advertisements in a programme stream. The advertisement management system of the invention manages the sales and insertion of digital video advertisements in cable TV, switched digital video and streaming internet based environments. WO 0215570 describes an advertising method for displaying an advertisement mixed with an image for television broadcasting, mobile communication terminals or image storage media. The procedure describes the simultaneous displaying of an advertisement from e.g. the upper left corner to the upper right corner of a screen mixed on the screen with an ordinary program image.
Summary of invention
None of the above methods outline an integrated TV messaging system, based on real or virtual images, both at the location of the event, and/or to the distant viewer with socio-economic or timing algorithmic capabilities. To integrate the advertising system within the broadcasting facility, would add a new dimension to the commercialization of events. In this way one could obtain a better socio - or demographic targeting of the advertisements based on nationality, race, age, income, sex, education or any other means of classifying people in demographic or socio - economic groups, by means of integrated programmable display units at the event location or virtual signs, boards, images or message either at the location of the event, on the object in question (cars, players, balls, horses etc.) or provided electronically within or being part of the broadcasting system and giving the appearance of being present at the location or on the (moving) main objects of the arrangement or event. The advertisement system may provide real time communications from and to the receiver (consumer) by e.g. telephone or the internet, enabling the ordering and billing of goods and services, providing further information or sound, in real time, according to the advertisements or messages shown at the display unit, both locally and for the distant TV viewer. Thus, in accordance with the invention there is provided a method for integrated TV messaging in live event broadcasting and in later re-runs from such events, using at least one TV camera at an event site, where at least one TV camera is able to establish at least one parameter among environmental and behavioural parameters, and constituting an obligatory sensor in a set of at least one sensor among environmental sensors and behavioural sensors at the event site. The at least one sensor is connected to a control unit. The method in accordance with the invention is characterized in that an immediately changeable message is combined into a broadcast image, using messaging control algorithms stored in the control unit, the algorithms operating on - at least one parameter among established environmental parameters and established behavioural parameters at the event site, and - at least one parameter among socio-economic parameters established for spectators at the event site and socio-economic parameters established for a target TV audience.
Favourable embodiments of the method of the invention, appear from the appended dependent claims 2-20. In another aspect, the present invention comprises a system such as defined precisely in the appended claim 21, and favourable embodiments thereof appear from the dependent claims 22-34.
Short description of the drawings
In the following, a detailed description will be given by reference to the appended drawings, of which figs. 1-6 shows, schematically, the interplay between TV cameras, various sensors and a broadcasting system, at event sites of various types, illustrating embodiments of the present invention.
Detailed description Besides traditional technologies like cathode ray tubes (CRT), large multicolour flat panel displays based on e.g. combinations of smectic A, smectic C, nematic or cholesteric Liquid Crystals (LCD), Light Emissive Diodes (LED), Organic Light Emissive Diodes (OLED), Plasma Display Panels (PDP) or other emerging technologies, may revolutionize the outdoor advertising market. In a modern society, both in rural and urban areas, various demographic or socio-economic groups, based on nationality, race, address, income, education, age, sex etc., will pass or be at certain (public) locations as a function of weekday, time, weather condition, type of event at an (sports) arena or field etc. Equivalent^, grouping by socio - economic parameters are also valid for TV audiences, which are watching the same broadcasted events. From an advertisement point of view, a challenge is to target these various groups of people selectively and in a cost effective way, with the additional possibility of real time interaction(s) or communication(s) between the advertiser and/or the displayed message(s) and the receivers (consumers), both locally and towards the distant TV viewer. Research and statistical analysis may be required to obtain adequate basic time varying socio-economic data as a function of location, season, weekday, time, the TV channel in question etc.
At the same time it would be beneficiary to vary the exposure time of each message or advertisement and/or the information/message itself, as a function of input from at least one sensor among environmental and behavioural sensors, e.g. with changing weather conditions, or the noise/accumulated voice or dB level or when a particular athlete or sports team is entering the field, reaching a geographical point or some action is occurring at a certain location at the field or track. This can be combined with real time interactions or communications between the transmitted message(s) and the receiver(s) [consumer(s)] for the transmission of additional information, like sound, music or enabling the ordering of goods or services related to the message(s), with billing routines, both locally and for the distant TV viewer. In this way one may obtain profitability by optimising the exposure to the audience, and at the same time the transmitter (or advertiser) may gather valuable statistics and/or billing data. At the same time it is advantageous for these systems to allow for the communication and/or the interaction between the advertiser, advertisement(s) or message(s) and the receiver(s) by any wireless means, like bluetooth, telephone, wap telephone, internet (palm top) etc., in real time, allowing the receiver(s) the ability to gather further information about the advertisement(s) or message(s), sound or music, or to order the goods or services in question. The present invention, with reference to the boards or display units associated with the examples to follow, consists of a display system having at least one local display unit or screen and a local processing unit (CPU), which is labelled (8) in fig.1. The CPU could also be integrated within the board or display unit, which is not shown separately. The CPU may be connected by cable or wireless communications to the integrated TV system. The CPU may have one or several analogue or digital data interfaces with the capacity for the input of combinations of text, music, speech, graphics, video, motion pictures or any other means of audio or visual communications. The local CPU may be in communication with a central server or control unit which may communicate with receivers (consumers) and/or for the updating of the various timing algorithms. The CPU may be connected to a geographic positioning device (not shown), a data storage device (not shown) for the storage of combinations of text, music, speech, graphics, video, motion pictures or any other means of audio or visual communications and a real time communications unit (not shown) for the
communications from and to the receiver (consumer) by e.g. telephone or the internet, enabling the ordering and billing of goods and services, providing further information or sound, in real time, according to the advertisements or messages shown at the board. As an integral part of the CPU(s) is an algorithm or algorithms which is/are characterised by its/their ability to; Combine an immediately changeable message into a broadcast image. Govern the interface between data input, data storage and geographic positioning either locally or via a centrally located server unit. Provide data as analogue or digital signals to and from transceivers either directly or via a centrally located server unit. Guidance and control of transmission and the receiving of analogue and/or digital signals to one or several electronic boards or displays or projector sites by cable or wireless communications. Interaction with sensors. Communications from and to the receiver (consumer) either directly or via a centrally located server unit.
There will always be the possibility to intervene manually with the system, update the primary sequencing chain and/or secondary or replacement modules through cable or wireless means, either locally or from anywhere in the world, through a centrally located server unit or not.
The various behavioural and environmental sensors could be of a variety of combinations, represented by commercial or non-commercial types, like cameras, image (number) detectors, temperature indicators, (infrared) motion detectors, radars, humidity detectors, light intensity detectors, pressure detectors, lasers, microphones, wave -, particle -, emission -, radiation detectors etc.
Technologies for virtual image commercials at site of event or electronically added within the broadcasting system
Virtual image carrying commercial messages or containing advertisements in the form of 2D or 3D still pictures, 2D or 3D motion pictures, with or without sound and/or interaction by telephone, internet or any other wireless means, which interact
with the events, manually operated or by behavioural sensors like cameras (analogue, digital, temperature, motion and/or image based etc.), microphones, infrared detectors, lasers, radars, pressure indicator or any other type of commercial or tailor made behavioural detectors, or environmental sensor like light sensors, humidity detectors, cameras, microphones, thermometers, pressure gauges or detectors or any other type of commercial or tailor made environmental detectors, may be added at the location of the event and/or mixed in or added to the scene of the event or on items like vehicles, machines, cars, motorcycles, bicycles, structures, humans, animals, clothing, balls, skis or any other object associated with the particular activity, by any electronic means, and provided to the distant viewer by terrestrial and/or any wireless means. Although it is natural for humans to see in three dimensions, most displays can still show only two-dimensional images. Nevertheless, some 3D displays have been developed. The human eye gives a sense of depth to everything we see. This is because one eye focuses on the object in question at a slightly different angle than the other. The brain then combines the two images and calculates the distance to the object observed. This principle may be used for 3D displays. Screens that alternate at a frequency of 120 Hz between one image for the right eye and one for the left are relatively common today. Special shutter glasses prevent one eye from seeing the image destined for the other, so that each eye in fact sees only 60 images per second. The brain is unable to resolve this rapid alternation and therefore constructs an image in three dimensions. More convenient for practical applications are the auto-stereoscopic 3D LCD displays, which are manufactured by several major companies like Philips, Sanyo and Samsung. The principle behind this technology is simple. A grid of rod-shaped cylinder lenses or prisms is mounted in front of an LCD display. The grid directs the light from one row of pixels to one eye and the light from the neighbouring row to the other eye. The viewer's brain then combines the two images in such a way that a 3D effect is produced.
Laser Holography involves illuminating an object with a laser and then photographically record the pattern that is produced as the reflected light interferes with a reference beam. Illuminating this interference pattern with a laser beam produces a
3D image of the object floating in space. Unfortunately, this process is at present really only suitable for small, stationary objects. In the 1990s, a technique known as electro-holography was developed at the MIT. The technique can produce holographic videos, because acoustical-optical elements exert so much influence on the laser beams in real time, that the overlapping waves generate a 3D image for the viewer. Thus it is also possible to produce 3D images of models stored in computers. However, there is still no satisfactory way to supply the computing power needed to process the massive data volumes involved. An alternative solution was proposed back in the 1970s by Rϋdiger Hartwig from the University of Heidelberg. This involves using coloured lasers to illuminate a transparent helix of plexiglas rotating at 1 ,200 revolutions per minute. At such speeds, the helix itself is invisible, so that only the points hit by the laser beams can be seen. If the beams are moved quickly enough, it is possible to illuminate each point of the volume covered by the rotating helix, thereby generating a 3D image. This principle was first exploited in the mid-1980s and has since been developed by a number of companies. A new technique presented by the Heinrich Hertz Institute, Berlin, in 2003, dispenses with the use of a screen-type device completely. Here, two small projectors are used to project a left-hand and a right-hand image directly into each of the viewer's eyes. The viewer then sees a 3D object floating directly before his or her eyes. For an elaboration of (possible) 3D technologies, see e.g. w4.siemens.de.
The arena as a (partial) virtual studio
The event place, the actual sports arena, or location for the event could be designed as a virtual or partial virtual studio. Images within an arena may be created by using computer graphics (CG) and/or combined or added to with real images, where interacting advertisements could be part of the CG, and subsequently could be integrated or added into the broadcasted picture. However, existing CG technologies are still not equivalent to on-site pictures taken in real world. Therefore, one is still dependent on the creation of image components from real pictures instead of CG to get highly realistic images.
In a virtual studio, based on real pictures, to construct the event set, the broadcasting producer may choose images of small objects (three-dimensional image component) to place them within the image of an all-round ambient scene (environmental image component). Real pictures are used in combination with these image components, so that in comparison to the conventional approach, images of a more realistic electronic studio set can be obtained. And these images are equivalent to real-life pictures. The environmental image component consists of high-definition spherical surface images taken from every angle around the viewpoint, and creates an image in real time at high resolution. The three-dimensional image components have three-dimensional modelling data and the surface images of the objects as separate, individual data. They permit free alteration of the view and light. To obtain more flexible and realistic composite studio-set images in the future, one may aim to create a system that facilitates the use of high-quality sets. This could be achieved by separating the three-dimensional element from the irradiation conditions in the environmental image component and by reconstructing the finer textures in the three-dimensional image component. A key component of any virtual production system is a means of measuring the precise position and orientation of each (studio) camera, so that the virtual scene can be rendered from the appropriate viewpoint. There are systems available that provide camera position data. The essential difference between a conventional blue-screen studio and a virtual studio is the ability to move the studio arena cameras or maintain a fix on an object in motion by (a) stationary camera(s) whilst maintaining the correct registration between the foreground and the virtual background or components (screens or display units). The method used to measure the camera positions and orientations (or vice versa) will have a significant effect on the ease of use and realism that can be achieved, and may also be a significant factor in the cost of the installation. Ideally, a moving target camera position measurement system should: allow for unconstrained movement of an object with respect to line of sight of cameras. work with a wide variety of camera mountings (including manual and robotic pedestals, cranes and hand-held).
measure the object and orientation to a sufficient accuracy to introduce negligible drift or noise in the relative positions of real and virtual elements of the scene, measure the object parameters with minimal delay. Two approaches to the problem were proposed by G. Thomas et. al. ("A versatile camera positioning measurement system for virtual reality TV production", Int. Broadcasting Convention, Amsterdam, 1997), also see www.bbc.co.uk. The first method relies on markers placed in-shot, whereas the second uses markers out-of-shot, viewed by a second camera. The second method appears to be the most suitable for the majority of applications. In the following, the examples illustrated in the drawings will be described:
Example 1 The traditional way of covering, say an international soccer match, from a TV marketing point of view, is to install fixed boards on each side of the field and place cameras on both the opposing sides, and broadcast TV images to the home audience of one of the teams from cameras from one side of the arena, covering the boards on the opposite side of the field, and vice versa. With reference to figure 1 , during a football match (soccer), displays (2A, B, C ...) with or without behavioural and/or environmental sensors (3a, 3aa, 3b, 3c, 3cc etc.) are placed around the field (1) or at the sports arena. Several cameras (6A, B, C, D...) are placed around the stadium. If an activity (4) is occurring in the viewing area (5) or within a certain angular sector of a display (2A), like players from team alpha are attacking within the 16 meter sector of team bravo, sensors (3a) may detect this activity, or by switching on a camera (6A), a certain advertisement message is displayed on a board (2A). This combined activity, advertisement by means of a processing unit (8), message on a board (2A) and action (4) is distributed through the control room (7) via terrestrial or wireless means (9, 10) to distant TV viewers. If team alpha is (attacking) within the sector(s) of a display (2A), the advertising message is intended for the audience in favour of team alpha. An equivalent situation could occur, by activity in favour of team bravo, within the viewing sector of display (2C), by camera (6D), where the displayed advertisement would be intended for a (TV) audience in favour of team bravo. Regular advertisements, in accordance with a timing algorithm, could be displayed on (2A), (2B)
and (2C) when cameras (6A) and (6D) are not in action. Regular scheduling by the timing algorithm could be interrupted /changed/modified with sensor input above trigger values like a certain dB level recorded at (3a) and/or a photo (light) sensor (3aa, 3bb) registering the passing of a ball (goal). Individual players may be identified by image recognition and subsequent personalized or individually related advertisement messages could be displayed on a board (2A, 2B, 2C). The displays (2A, 2B, 2C) may be real or virtual at location and/or virtual within a virtual studio or broadcasting system.
Example 2 With reference to figure 2, at a European handball field or at an ice hockey arena (1) the display units (2A, 2B) could be part of or integrated within the actual field and/or located underneath the ice of a hockey court, in addition to or instead of being placed around the field with accompanying sensors, as described in example 1. Various sensors (3a, b, c, ...) could be placed around, on or within the field. If an activity is occurring at a certain location within the court and/or a sensor (3a, b, c, ..) may detect an activity, or a certain camera (4A) is switched on, a certain advertisement message is displayed on e.g. board (2A). This combined activity, advertisement by means of a processing unit (6), message on board (2A) and action is distributed through the control room (5) via terrestrial or wireless means (7, 8) to distant TV viewers (9). Other features are or could be equivalent as described in example 1. The displays (2A, 2B) may be real or virtual at location and/or virtual within a virtual studio or broadcasting system.
Example 3 With reference to figure 3, equivalent^ at a baseball match, displays (4A, B, C ...) with or without behavioural and/or environmental sensors (5A, B, C) are placed around the field (1) or at the sports arena. Several cameras (5A, B, C, D...) are placed around the stadium. In addition to combinations of features of example 1 and 2, a tracking system is added to the players. In this way a virtual advertisement can be added on the clothing (3, 13) to the player in action (2), either locally (8) or distantly (11) so that a different (virtual) advertisement could appear on the clothing to the player dependent on the network or distribution channel to which the viewer is connected. The
displays (4A, B, C, ...) may be real or virtual at location and/or virtual within a virtual studio or broadcasting system. A special "home run indicator", constituting e.g. a radar and/or a microphone (dB indicator), would be a natural component of such a system. When a home run is achieved, a particular advertising message would occur on the boards (A, B, C), Also, individual players may be identified by image recognition and subsequent personalized or individually related advertisement messages could be displayed on the same boards.
Example 4 With reference to figure 4, equivalently at a race car competition, displays (4A, B, C ...) with or without behavioural and/or environmental sensors (5A, B, C) are placed around the field (1 , 2). Several cameras (5A, B, C, D...) are placed around the track. In addition to combinations of features provided by examples 1 - 3, a tracking system is added to the cars. In this way a virtual advertisement can be added on the cars (3, 13), either locally (8) or distantly (11) so that a different (virtual) advertisement could appear on the car or at the arena dependent on the network or distribution channel to which the viewer is connected. The displays (4A, B, C, ...) may be real or virtual at location and/or virtual within a virtual studio or broadcasting system.
Example 5 With reference to figure 5, equivalently at an alpine (slalom, giant slalom, super giant slalom, downhill) ski competition track (1), behavioural and/or environmental sensors (3A, 3B ...) are place at the track. When the skier (2), whose identity is established by start sequence or image recognition, passes a sensor, a message associated with the skier, like the brand name of the skis, binding or ski booths he or she is wearing, or an advertisement related to the country from where the skier lives or is associated is shown on the board (3, A, B, C...). A camera (5, A, B, C), in the viewing sector (6) could capture the combined image of the skier (8) and the board (3). The pictures are distributed through a control room (9) via terrestrial or wireless means (11 , 12) to distant TV views (14). A virtual or tracking system could be added to the skiers (2, 8). In this way a virtual advertisement or message could appear be on the clothing (7, 15), either locally (10) or distantly (13) so that different (virtual)
advertisements or messages could appear on the clothing to the skier dependent on the network or distribution channel to which the viewer is connected. The board (3, A, B, C, ..) or displays may be real or virtual at location and/or virtual within a virtual studio or broadcasting system. In addition to the above features combinations provided by examples 1 - 4 could be added. An equivalent arrangement or system could be developed and installed for track and field competitions. E.g. at such an international competition, when a certain individual is entering a particular location of the field to perform e.g. long jump or pole vault, an adjacent display, within a certain camera view which is installed to capture the performance in question, the display may simultaneously present a message associated with the athlete. The message could be a brand with the same nationality as the athlete in question, a particular type of sports wear associated with the performer, etc. A natural element of the system would be a (world) record indicator, which, when an athlete achieves a record, yields a special advertising module on the displays in question. Any of the combinations of features discussed in the examples could be applicable to a track and field competition.
Example 6 With reference to figure 6, in addition to the features of examples 1 - 5, the combined message is distributed within a mall, building or the arena itself. In all examples or applications there may be the possibility for the communications from and to the local or distant receiver (consumer) by e.g. telephone, the internet or any other communication means (satellite phone, fibre optics, touch TV, identification card, credit card, debit card, smart card etc.), enabling the ordering and billing of goods and services, providing further information or sound, in real time, according to the advertisements or messages shown at the board or the virtual image. The present invention is not limited to the described system and algorithms, in the sense that any generic variations of the system or devices, such as various arrangements or rearrangements of the various components, various sizes and geometric designs of components, independent and automatic control and guidance of
components of the system, interconnection of several displays or images, continuous or part time connection to a central server or control unit, the placement of sensors or panels or any other components on movable fixtures, built in or partly or totally integrated systems or solutions, are all obvious variations to be derived by a skilled person in the art, subjected that this description of the stated invention is provided. Subsequently, all devices or systems that are functionally equivalent will be included by the scope of this invention, and any modifications of the invention that lie within the scope of the stated claims. Based on the above statements, all drawings and figures are to be interpreted illustratively and not in a limiting context. It is further presupposed that all the claims shall be interpreted to cover all generic and specific characteristics of the invention which are described, and that all aspects related to the invention, no matter the specific use language, shall be included. Thus, the stated references have to be interpreted to be included as a part of this invention's basis, methodology, mode of operation and apparatus.