WO2004021699A1

WO2004021699A1 - Head mount image display system device and image processing method thereof

Info

Publication number: WO2004021699A1
Application number: PCT/JP2003/010615
Authority: WO
Inventors: Yoichi Iba
Original assignee: Olympus Corporation
Priority date: 2002-08-30
Filing date: 2003-08-22
Publication date: 2004-03-11
Also published as: JP2004109994A

Abstract

In a head mount image display system device (1), direction data is supplied from a head direction detection section (5) of an HMD (2) via a head direction data relay section (9) of a controller (3) to a head direction data storage section (10). The head direction data storage section (10) outputs the current head direction data and past head direction data Δ t (time required for image generation processing by a computer) before to a direction displacement calculation section (11). According to a difference between these head direction data directions, the direction displacement calculation section (11) calculates a vertical and a horizontal shift amount of field of view generated by the processing time delay Δ t of the computer (4). After this, an image signal processing section (12) subjects the image supplied from an image generation section (4B) of the computer (4) to reverse-direction shifting by an amount equal to the vertical and the horizontal shift amount calculated by the direction displacement calculation section (11) and outputs the result to the HMD (2).

Description

Head mounted image display system apparatus and image processing method thereof

Light

Technical field

The present invention relates to a head-mounted image display system, and particularly to a head-mounted image display system device, which can generate a generated image from an image generation device based on a moving direction of a user's head regardless of a processing time involved in image generation calculation processing. The present invention relates to a head-mounted image display system device capable of real-time display on a 10-head image display device without discomfort, and an image processing method thereof.

BACKGROUND ART In recent years, a system having a head-mounted image display device, that is, a so-called head-mounted display (hereinafter, referred to as an HMD) has attracted attention.

This HMD is a display device worn on the user's head as described above, and is used as a visual display device for a virtual reality (hereinafter referred to as VR) system or a mixed reality (hereinafter referred to as MR) system. May be used.

A typical configuration of a VR or MR system is an HMD equipped with a head tracker (a sensor, hereinafter abbreviated as HT) for detecting a moving direction of a user's head, and an image display unit of the HMD. A controller having a controller for controlling an image display, a controller for generating and outputting an image to be displayed on an image display unit of the HMD, and a controller for controlling the entire system. The user wears the HMD on his head and uses this system.

In this type of system, the HT detects and measures the orientation of the user's head and outputs the orientation data to a computer via a controller. In response to this, the computer generates a visual image of the virtual environment based on the azimuth data by an image generating means using a computer graphic technology. That is, it generates a visual image that would be visible when the user enters the virtual space and turns his head in that direction. Then, the computer outputs the generated visual image to the HMD via the controller, and the HMD displays the visual image on an image display unit provided therein, thereby presenting it to the user as an aerial image.

By the way, in this kind of VR system, ideally, a series of flows in which the head direction is measured by the HT and the image is displayed on the HMD must be performed instantaneously. However, in reality, it takes 15 hours to perform the arithmetic processing to generate an image in a computer (this time is called At). Therefore, even if the user moves his or her head, there is no change in the visual field of the virtual space observed through the HMD at that moment, and after a lapse of Δt, the direction of the visual field changes to the head direction. They will start following changes. In the real world, there is no such delay in the world observed by the naked eye, and this delay is uncomfortable. 20 Similar technologies that have already been disclosed include, for example,

Japanese Patent Application Laid-Open No. 9-22846476 and Japanese Patent Application Laid-Open No. 8-191416 / 19 propose a gyro sensor in the former Japanese Patent Application Laid-Open No. 9-2846476. A means for detecting the amount of movement and / or rotation of the viewer's head is provided on the mounted head-mounted display, and the detected amount of movement and / or rotation is provided. Alternatively, there is disclosed a video display device which performs video processing for extracting and displaying a part of a video signal of an original video according to a change amount of a rotation angle.

Also, in the latter proposal of Japanese Patent Application Laid-Open No. 8-191419, the user's head position information detected by the rotation angle sensor from the wide-angle image signal stored in the frame memory of the signal processing unit is proposed. A head-mounted display system is disclosed in which a display image to be viewed by a user is cut out from a frame memory and displayed on a display based on the above.

However, in the above-described conventional VR system, as described above, the arithmetic processing for generating an image in a computer takes a time of Δt. There was a problem in that, even if the ten heads were moved, the images corresponding to the movements of the heads were not displayed in real time, causing a sense of discomfort. Conventionally, as a method of eliminating the discomfort caused by the effect of Δt, a method using a computer having a higher-performance arithmetic processing capability or a method of reducing the image density to reduce the arithmetic volume in image generation are considered. In the former method, 15 systems are expensive and difficult to realize, while the latter method can reduce the time of △ t, but the image becomes less dense This has the disadvantage of losing realism as a result.

In addition, Japanese Patent Application Laid-Open No. Hei 9-2846476 and Japanese Patent Application Laid-Open No.

In the conventional similar technology proposed in Japanese Patent No. 191419, a large-capacity frame for storing the original video 20 and a memory and a processing capability capable of generating an image with a wider angle of view are used. There was a problem that the system was expensive because it needed a computer that had it.

Therefore, the present invention has been made in view of the above-described problems, and it is possible to display an image corresponding to the 25th position of the head in real time at low cost and with a reduced image display delay due to image generation calculation processing. Head mounted image display system It is an object of the present invention to provide a device and an image processing method thereof.

Disclosure of the invention

A head-mounted image display system apparatus according to the present invention includes a display unit that can be attached to and detached from a user's head, and a display unit that is installed on the display unit and has at least a horizontal orientation of the user's head. A head-mounted type including: a direction detecting hand to be detected; and an image generating device that generates an image in accordance with the direction of the user's head from the direction detecting means and displays the generated image on the display means. In the image display system apparatus, an azimuth shift calculating means for calculating a difference between the current head azimuth data obtained by the azimuth detecting means and the head azimuth data for a predetermined time, and the azimuth shift calculating means Image processing means for at least horizontally shifting the position of the aerial image displayed on the display means in accordance with the azimuth displacement obtained by the method. With this configuration, it is possible to display an image corresponding to the azimuth of the head 15 in real time at low cost and with reduced image display delay due to the image generation operation processing.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a configuration diagram showing a first embodiment of a head-mounted image display system device according to the present invention, and showing a schematic system configuration of the head-mounted image display system device. FIG. 2 is a block diagram showing a circuit configuration which is a feature of the head-mounted image display system according to the present embodiment. FIG. 3 illustrates the principle of the head-mounted image display according to the present invention. FIG. 4 is an explanatory diagram for illustrating the principle of the head-mounted image display device according to the present invention. Those attributed to t FIG. 5 is a diagram showing a correction amount (△ 0) for correcting the position signal difference. FIG. 5 is an explanatory diagram for explaining the principle of the head mounted image display device of the present invention. FIG. 6 is a graph showing the relationship between (△ Θ) and t. FIG. 6 is an explanatory diagram for explaining the principle of the head-mounted image display device according to the present invention.

7A to 7C are diagrams showing image orientations when shifting based on (シ).

7E is an explanatory diagram for explaining the operation of the head-mounted image display device, FIG. 7A is a diagram showing time, and FIG. 7B is a diagram showing head orientation of the user. Yes, Figure 7C is a diagram showing the correct image according to the head orientation seen by the user, Figure 7D is a diagram showing the output image generated by the computer, and Figure 7E is a diagram showing the HMD FIG. 8 10 shows a second embodiment of the head-mounted image display system according to the present invention, and FIG. FIG. 9 is a block diagram showing a characteristic circuit configuration. FIG. 9 shows a third embodiment of the head-mounted image display system according to the present invention. FIG. 10 is a block diagram showing a circuit configuration of a head mounted type 15 image display system according to the present invention. Shows a fourth embodiment of a proc diagram showing a circuit configuration which is a feature of the corresponding portion mounted image Display system device.

BEST MODE FOR CARRYING OUT THE INVENTION 20 Hereinafter, embodiments of the present invention will be described with reference to the drawings.

First embodiment:

(Constitution)

1 to 7 show a first embodiment of a head mounted image display system according to the present invention, and FIG. 1 shows a schematic system configuration of the head mounted image display system. Figure 2 shows the head-mounted image display system. FIG. 3 is a block diagram showing a circuit configuration which is a characteristic of the device. FIGS. 3 to 6 are explanatory diagrams for explaining the principle of the head mounted image display device according to the present invention. FIG. 3 shows the image orientation caused by Δt. Fig. 4 shows the correction amount (ΔΘ) for correcting the azimuth data difference caused by △ t. Fig. 5 shows the correction amount (△

グラフ), and FIG. 6 shows the image orientations when shifting based on the correction amount (ΔΘ). 7A to 7E are explanatory diagrams for explaining the operation of the head-mounted image display device. FIG. 7A is time, FIG. 7B is the head orientation of the user, and FIG. The correct image according to the head orientation seen by the user, Figure 7D shows the output image generated by the computer, and Figure 7E shows the corrected image displayed on the HMD. 10 The head-mounted image display system according to the present embodiment has a simple configuration in which an existing conventional circuit is slightly changed, is inexpensive, reduces image display delay due to image generation calculation processing, and The feature is that it has been improved so that an image corresponding to the head orientation can be displayed correctly.

First, the principle of the head-mounted image display system device of the present invention will be described with reference to FIGS. In order to simplify the explanation, it is assumed that the orientation of the head rotates and shifts only around the vertical axis, that is, it is limited to the case where the head is horizontally swung. It shall be represented by

The head-mounted image display system apparatus of the present invention includes a head direction detecting means such as an HT for detecting 20 directions of movement of the user's head, and head direction information and images from the head direction detecting means. A head orientation data storage means for storing (recording) head orientation information with a time delay (A t, and the current time is t O) due to image generation processing by a computer as a generating device; azimuth 25 shift calculating means for calculating the head azimuth shift amount (also referred to as Δ0) according to t, and an image signal for performing image correction processing based on Θ to be obtained. Signal processing means is provided at least.

As described in the related art, in the head-mounted image display system device having the above configuration, it takes time Δt due to the image generation calculation processing based on the head orientation. In other words, this is shown in FIG. The solid line in FIG. 3 is the azimuth data # 5 detected by the head azimuth detecting means, and the wavy line indicates the center of the visual field generated by the computer as the image generating device. In this case, assuming that the image calculation process requires time Δt as described above, the wavy line is a curve obtained by horizontally moving the curve shown by the solid line rightward by Δt as shown in FIG.

FIG. 4 shows how the correction amount △ is obtained from the azimuth detection signal (the azimuth data and 10) obtained by the head azimuth detecting means. That is, by using the head direction data storage means and the direction change operation means, the correction amount △ 0 is obtained by subtracting the direction data measured Δt in the past from the current direction data. FIG. 5 shows an example in which the obtained correction amount △ 0 is shown in a graph. That is, in FIG. 5, the correction amount △ 0 indicated by the arrow 15 in FIG. 4 becomes the correction amount △ 0 indicated by the arrow in FIG. 4, and is the value at the current time t 0. In this case, if there is no displacement of the head, the correction amount ΔΘ becomes zero, and if the head moves at a constant speed, it becomes a straight line instead of a curved line.

In the present invention, the image signal processing means performs a correction process on an image generated from a computer based on the obtained correction amount of correction △ 0, whereby 分の t resulting from the image generation operation process is obtained. Regardless of the time delay, an image corresponding to the head direction can be obtained in real time in the same manner as when the image generation calculation processing was performed instantaneously. This is shown in FIG.

That is, FIG. 6 shows the visual field center direction of an image obtained by subjecting the image generated by the image generation unit of the computer to the image signal processing means by performing a 分 0 minute shift process. This is indicated by a two-point diagonal line. That is, as indicated by the two-point diagonal line, the visual field center direction of the corrected image is the same as the direction detected by the head direction detecting means (Fig.

3) can be seen from the comparison between Fig. 3 and Fig. 6.

For simplicity, the explanation has been given of the case in which the head is shifted only in the horizontal direction.

Instead of Θ, this can be represented by a matrix that indicates the azimuth, and Δ0 can be represented by a transformed matrix. However, because the sense of discomfort occurs more favorably in the horizontal direction than in the vertical and rotational directions, if the aim is to simplify the circuit configuration, correcting only the horizontal direction has a significant effect on reducing the discomfort. . 10 An embodiment of a head-mounted image display system that solves the above-mentioned problems by adopting such a principle is shown below.

As shown in FIG. 1, a head mounted image display system according to the present embodiment

Reference numeral 1 denotes a head mounted display (HMD) 2 provided with the head direction detecting means, and is electrically connected to the HMD 2 via a connection cable 3 A. It comprises a controller 3 for control and a computer 4 which is electrically connected to the controller 3 via a connection cable 3A and which has an image generation / operation processing unit. .

The HMD 2 includes a main body 2A having main components such as an image display section, and a mounting section 2B for mounting the main body 2A on a user's head. The main body 2A has a mounting portion 2B such as a crane attached to the base end portions on both sides, and the main body 2A is mounted and held on the head using the mounting portion 2B so that the main body 2A covers the eyes of the user. Is done. In addition, as the mounting portion 2B, other than the crane, any other device may be used as long as the device mounts and holds the main body 2A on the user's head. Although not shown, mounting part 2 B Both sides corresponding to the ears of the user are provided with an inner phone for reproducing a sound corresponding to the displayed image.

The controller 3 is an HMD controller for controlling an image display unit (described later) of the HMD 2, and controls display of a display image on the image display unit arranged on the main unit 2A of the HMD 2, controls volume, Performs various controls such as image quality and sound quality adjustment, and turning on / off the power supply 5 switch. The controller 3 is also provided with operating means (not shown) for performing the above-described various controls and a terminal group necessary for connecting the connection cable 3A.

The computer 4 is an image generation device having an image generation operation processing unit (described later), and generates and outputs an image based on the head direction data detected by the head direction detection means of the HMD 2. For example, when the head-mounted image display system is configured as a game machine, the computer 4 is a game machine capable of outputting a video signal.

Next, the electrical circuit configuration of such a head mounted image display device will be described with reference to FIG. 15 As shown in Fig. 2, inside the main body 2A of the HMD 2, a head orientation detection unit 5 as a head orientation detection means, a small image display element drive circuit 6, a small image display element 7, an aerial image projection The optical system 8 is mounted.

The head direction detection unit 5 is composed of HT or the like that detects the moving direction of the user's head, and converts the detected head direction data (direction vector) to the head of the control port 20 3. Output to bearing data relay unit 9.

The small-sized video display element 7 is constituted by, for example, an LCD for displaying an input image, and is arranged on the inner side surface of the main body 2A.

The small-sized video display element drive 6 controls the driving of the small-sized video display element 6, and causes the small-sized video display element 7 to display an image based on an input image signal. The aerial image projection optical system 8 is an optical system having a positive refractive power, and allows a user to view a display image displayed by the small image display element 7 as a more realistic aerial image. Regarding the structure of the HMD, when the main body 2A of the HMD 2 is mounted on the user's head by the mounting portion 23B, external light is transmitted to the aerial image optical system 8 and the small image display element 7. Although not shown, a shielding structure to prevent light from entering is also provided.

The controller 3 mainly includes a head direction data relay unit 9 that fetches and outputs the position data from the head direction detection unit 5 of the HMD 2, and a head direction from the head direction data relay unit 9. Head 10 data storage unit 10 for storing data (including head orientation data of time delay (A t) due to image generation processing by computer 4), and head orientation displacement amount according to Δt (Shift amount: Δ0) and an image signal processing unit 12 for performing image correction processing based on the obtained Δ0.

The head direction data relay unit 9 outputs the received direction data to the head position data storage unit 10 and the computer 4.

The computer 4 includes a control unit 4a for controlling the entire device and the system, and an image generation unit 4b for generating an image based on the azimuth data. Based on this, an image of the virtual space that looks like this direction is generated and output to the image signal processing unit 12 of the controller 3.

In the head-mounted image display system having the above configuration, heading data is supplied from the head head detection unit 5 of the HMD 2 to the head head data storage unit 10 via the head head data relay unit 9 of the controller 2. Then, it is recorded by the head direction data storage unit 10 of the controller 3. 25 Head direction data storage section 10 stores the latest recorded head direction data. The head direction data of the present time (to) and the head direction data of the past recorded by At (the time required for the image generation processing by the computer). Output to

Then, the heading change calculation unit 11 calculates the heading difference from the current head heading data and the head heading data already recorded by Δt in the past, and the computer 4 calculates the heading difference based on the heading data. The vertical and horizontal shift amounts of the visual field caused by the processing time delay Δt are obtained by arithmetic processing. The obtained shift amount is supplied to the image signal processing unit 12, and the image signal processing unit 12 receives the shift amount from the azimuth shift operation unit 1 for the image supplied from the image generation unit 4 B of the computer 4. The vertical and horizontal shift amounts calculated in step 1 are processed so as to be shifted in the opposite 10 directions by the same amount, and the processed image signal is output to the small image display element drive circuit 6 of the HMD 2.

The small video display element drive circuit 6 drives the small video display element 7 based on the corrected image signal, and displays an image having the same view direction as when the computer 4 instantaneously generated an image. . This allows the 15 users to view and check the aerial image in the virtual space through the aerial image projection optical system 8 without any discomfort.

In the present embodiment, the azimuth shift calculation unit 11 has been described in such a manner that the vertical and horizontal shift amounts of the visual field caused by the calculation time delay Δt of the computer 4 are calculated by the calculation processing. However, the present invention is not limited to this. For example, the amount of rotation may be obtained by arithmetic processing, and the image signal processing unit may be configured to perform processing including rotation amount correction.

(Action)

Next, an operation characteristic of the head mounted image display system according to the present embodiment will be described in detail with reference to FIG. 25 Note that, as shown in FIGS. 7A and 7B, the head-mounted image display> The start time t of the use of the device 1 is set to 12:00, the time required for the arithmetic processing of the computer is set to 5 seconds, and the change of the head movement of the user after Δt from the use start time is calculated. A description will be given assuming that the head movement change up to the present after 30 ° and further after △ t has been moved horizontally by 15 °.

Now, it is assumed that the user wears the head-mounted image display device 1 of the present embodiment 5 on his / her head and turns on the power.

At this time, as shown in FIG. 7B, the user's head 20 is facing the front (0 ° position) at the time of 12:00, and the actual image of the front is shown in FIG. 7C. Assuming that such a tree 30 is in the center and the person 31 is in the right direction, the image 20A is output from the computer 4. The video 4A shown in D is output to the image signal processing unit 12 of the controller 3, and as a result, the video is displayed on the small video display element 7 of the HMD 2.

The same video 8 A (see Fig. 7E) as 4 A is displayed.

After 0.5 seconds (△ t later 1 2: 0 0.05), the user's head

Assuming that 20 has moved 30 ° to the right, the actual 15 images seen at 30 ° are images of the tree 30 and the person 31 moving slightly to the left as shown in Figure 7C. However, as shown in FIG. 7D, the output video from the computer 4 takes a time に t required for the arithmetic processing of the arithmetic processing unit 4b, so that 1 2: At the times 0 0 and 0 5, an image corresponding to the head direction change could not be obtained, and the image 4 B, which is almost the same as the actual image 20 A (see Fig. 7C), is processed by the controller 3. This is output to the unit 12.

However, in the present embodiment, the heading difference calculation unit 11 is used to determine the heading difference from the current head heading data and the head heading data already recorded by Δt in the past. Based on this, the vertical and horizontal shift amounts of the field of view caused by the delay time Δt of the arithmetic processing time of the computer 4 are calculated by the arithmetic processing. (△ 0 shown in FIG. 4; shift unit 40 shown in FIG. 7E in this example), and then supplied by image signal processing unit 12 from image generation unit 4b of computer 4 The obtained image (video 4B) is processed so as to shift in the opposite direction by the same amount as the vertical and horizontal shifts calculated by the azimuth shift calculator 11 (see FIG. 6). The output to the small video display element drive circuit 65 of the HMD 2 results in the small video display element 7 having an image having the same field of view as when the computer 4 instantaneously generated an image, However, an image 8B (see FIG. 7E) substantially the same as the actual image 20B is displayed.

Then, after 0.5 seconds (△ t later: 12: 0.10), if the heads 20 of the ten users moved 45 ° rightward, this 45 ° The actual image seen in Fig. 7C is the image 20C with the tree 30 at the left end and the person 31 in the center as shown in Fig. 7C.The output image from the computer 4 is shown in Fig. 7D. As shown in the figure, the time △ t required for the arithmetic processing of the arithmetic processing unit 4 b is required. Therefore, at the time of 1 2: 0 0, 10, an image corresponding to the head direction change 15 is obtained. Therefore, the video 4C, which is substantially the same as the actual video 20B (the past video 4B before At), is output to the image signal processing unit 12 of the controller 3.

However, in the present embodiment, similarly to the above, the heading difference calculation unit 11 is used to calculate the heading difference between the current head heading data and the past 20 head heading data by Δt that has already been recorded. Based on this, the vertical and horizontal shift amounts of the field of view caused by the arithmetic processing time delay Δt of the computer 4 are obtained by arithmetic processing (in this example, the shift section 40A shown in FIG. 7E). Thereafter, the image signal processing unit 12 processes the image (video 4C) supplied from the image generation unit 4B of the computer 4 in the vertical and horizontal directions, which are processed by the azimuth shift calculation unit 11 25. Processing to shift the same amount as the shift amount in the opposite direction And output to the small video display element drive circuit 6 of the HMD 2, so that the small video display element 7 has an image with the same view direction as when the computer 4 instantaneously generates an image, that is, The actual picture

Video 8C (see Fig. 7E), which is almost the same as 20C, will be displayed.

Thus, the user can view and check the aerial image in the virtual 5 space through the aerial image projection optical system 8 without any discomfort.

As shown in FIG. 7D, there may be an area around the corrected image where the image indicated by the hatched portion in the figure is not displayed. As a means for preventing or reducing this, a visual field mask that covers the periphery of the display area of the small-sized video display element is preferably arranged in front of the small-sized video display element 7 shown in FIG.

Alternatively, by performing overscan driving of the small-sized image display element 7, the image signal may be protruded from the effective display area of the display element, and the image in which the peripheral image is truncated may be displayed on the display element.

(Effects) 15 Therefore, according to the present embodiment, a head-mounted type that is inexpensive and capable of reducing an image display delay due to image generation arithmetic processing and displaying an image corresponding to the head orientation in real time. An image display system device and an image processing method thereof can be realized.

By the way, the above Δ ΐ may differ depending on the arithmetic processing capacity of the computer used.In the present invention, this At is determined in advance by some means to determine the image generation time of the computer, and The value may be manually set in the position change calculation unit 11. Therefore, automatically setting up the above Δt is extremely effective in simplifying the setting operation by the user. An embodiment in which such an Δt can be automatically set up 25 will be described below. Second embodiment:

(Structure, function)

FIG. 8 is a block diagram showing a second embodiment of the head mounted image display system device according to the present invention, and showing a circuit configuration which is a feature of the head mounted image display system device. In FIG. 8, the same components as those of the five devices in the first embodiment are denoted by the same reference numerals, and description thereof will be omitted. Only different portions will be described.

In the present embodiment, a setup button operated when setting up the t measuring unit 13 and t in the controller 3 of the first embodiment.

In addition to the configuration of the first embodiment, the head orientation data relay unit 9 is provided with a new additional function, and other configurations are substantially the same as those of the first embodiment. is there.

As shown in FIG. 8, in the controller 3, the setup button 50 is an operation button operated when the user automatically measures Δt, and when pressed, the operation signal is transmitted to the head direction data. Relay units 9 and 15 供給 Supplied to a control unit (not shown) that controls the entire controller.

In this case, the control unit (not shown) controls the processing in the controller 3 according to the setup mode.

When the setup button 50 is pressed, the head direction data relay unit 19 transmits dummy direction data (hereinafter, referred to as first 20th position data) to the computer 4 based on the supplied operation signal. At the time when the time is output to the measurement unit 13 and a sufficiently long time (for example, 1 second) has elapsed from the assumed At, the azimuth data obtained by adding a predetermined displacement to the first azimuth data (hereinafter, 2) is output to the computer 4 and the t measurement unit 13.

The computer 4 generates 25 images (hereinafter, referred to as a first image) based on the first azimuth data by the image generation unit 4b, and the Δt measurement unit of the controller 3 13 and an image signal processing unit 12, and also generates an image based on the second azimuth data (hereinafter referred to as a second image). Similarly, the Δt measurement unit 13 and the image signal of the controller 3 Output to processing section 12.

The Δt measuring unit 13 starts counting from the time when the second azimuth data is received, stops counting when the second image is received, and counts the time up to 5. That is, the time from the transmission of the head direction data relay unit 9 to the computer 4 to the transmission of the second image to the controller 3, that is, Δΐ, is measured. Then, the At measurement unit 13 outputs the measured value Δt to the head orientation data storage unit 10 to be set.

In the measurement of Δt by the Δt measuring unit 13, it is necessary to detect that the first image sent from the computer 104 has been switched to the second image. If the luminance signal of the first image signal and the second image signal are time-differentiated, and when the timing at which the luminance signal changes sharply is detected, it is determined that the first image is switched to the second image. Alternatively, the switching of the image may be detected using other means. 15 When Δt is measured by the Δt measurement unit 13, the image pattern generated by the image generation unit 4 b of the computer 4 is registered in advance in the image generation unit 4 b or other storage means. Alternatively, based on this, the first image and the second image may be output to the Δΐ measuring unit 13 of the controller 3 to measure Δt. According to such a method, more stable measurement of Δt 20 can be performed.

Further, in the present embodiment, the image patterns to be registered in advance are a plurality of images having different numbers of polygons, At is determined for each image, and an average or weighted average is obtained from the obtained At. If Δ is used in the azimuth change operation unit 11, processing can be performed with higher accuracy. 25 Other configurations and operations are the same as in the first embodiment. You.

(Effect)

Therefore, according to the present embodiment, in addition to obtaining the same effects as those of the first embodiment, it is possible to automatically set Δt required for image generation arithmetic processing that differs depending on the arithmetic processing capability of a computer. This greatly contributes to the simplification of the user's 5 setting operations.

By the way, the head orientation detection unit 5 used in the first and second embodiments includes, for example, two types of sensors, a type using an acceleration sensor and a type using a magnetic sensor. However, the acceleration sensor has a high response, but has the drawback of causing drift10. On the other hand, the magnetic sensor has a drawback of no drift but poor response.

Therefore, by using two different types of sensors that detect the head orientation and configuring each of the two head orientation detectors to take advantage of their respective advantages, an expensive head orientation detector can be prepared. At the same time, it is possible to obtain the effect of reducing discomfort and to construct the system at low cost. Such an embodiment is shown in FIG.

Third embodiment:

(Structure, function)

FIG. 9 is a block diagram showing a head mounted image display system according to a third embodiment of the present invention, showing a circuit configuration which is a feature of the head mounted image display system. In FIG. 9, the same components as those in the device according to the second embodiment are denoted by the same reference numerals, and description thereof will be omitted. Only different portions will be described.

In this embodiment, two first and second head orientation detectors 5A and 5B are provided in place of the head orientation detector 5 used in the second embodiment. At the same time, in response to this, two

The first and second head orientation data relay sections 9A and 9B are provided.

That is, in the present embodiment, the acceleration sensor having a high response speed is used for measuring the azimuth required for the arithmetic processing of the azimuth change arithmetic unit 11, and the magnetic sensor without the drift is used for the image of the computer 4. It is used for the purpose of measuring the azimuth required by the generation unit 4b for image generation.

More specifically, as shown in FIG. 9, the first head orientation detection unit 5A is configured of a type to which a magnetic sensor is applied, and the detected heading data is transmitted to the first head orientation data relay unit. Output to computer 4 via 9A. On the other hand, the second head orientation detection unit 5B is configured by applying an acceleration sensor type, and the detected heading data is transmitted to the head orientation data relay unit 9B via the second head orientation data relay unit 9B. Output to storage unit 10.

According to the above configuration, as the head azimuth detecting means for detecting the azimuth required for the azimuth change calculating unit 11, the 215 th head azimuth detecting unit 5B composed of an acceleration sensor having a high response speed is used. Image generation unit of computer 4

4b is a head direction detecting means for detecting the direction required for image generation, using a first head direction detecting unit 5A composed of a magnetic sensor type without drift. Therefore, it is possible to perform motion processing that takes advantage of each feature, and it is possible to improve the detection accuracy at a low cost20 without providing an expensive head orientation detection sensor.

Other configurations and operations are the same as those of the second embodiment.

(Effect)

Therefore, according to the present embodiment, in addition to obtaining the same 25 effects as in the second embodiment, inexpensive head orientation detection sensors are not required. This makes it possible to improve the detection accuracy and greatly contributes to a reduction in discomfort.

Fourth embodiment:

(Structure, function)

FIG. 10 shows a fourth embodiment of the head mounted image display system according to the present invention, and is a block diagram showing a circuit configuration which is a feature of the corresponding part mounted image display system. In FIG. 10, the same components as those of the device according to the third embodiment are denoted by the same reference numerals, and description thereof will be omitted. Only different portions will be described.

In the present embodiment, the 110th head direction detecting unit 5A and the second head direction detecting unit 5B constituting the detecting means in the third embodiment are connected to one head direction detecting unit. The feature is that it has been replaced with Part 5C.

More specifically, as shown in FIG. 10, the head direction detection unit 5C is composed of a direction sensor unit 14 having one or more sensors and a direction sensor signal calculation unit 15; The azimuth sensor signal calculation unit 15 processes the 15 signals output by the unit 14 according to the movement of the head, converts the signals into azimuth coordinates, converts the first head azimuth data relay unit 9A and the second head The direction data A and the direction data B are output to the direction data relay unit 9B.

At this time, the azimuth sensor signal calculation unit 15

In order to generate A and azimuth data B, processing is performed using different signal processing circuits or 20 arithmetic processing. In other words, the generation of the azimuth data A is a process that focuses on suppressing the drift rather than the response speed, and the generation of the azimuth data B is a process that focuses on the faster response speed than suppressing the drift. Do. As for this response speed, it is desirable that the image displayed immediately after the head movement is detected is an image corresponding to the head direction. 25 Therefore, the response speed is almost lower than the video rate, that is, The response time until the corresponding azimuth data B is output is, for example, preferably 1/15 sec or less, more preferably 1/30 sec or less.

The azimuth sensor signal calculation unit 15 incorporates signal processing means such as two kinds of low-pass filters (LPF), band-pass filters (BPF), and noise filters having different time constants, for example. Correspondingly, the direction data A and the direction data B are output.

The signal processing means such as the LPF, the BPF, and the noise filter may be configured by hardware, or may be realized by software arithmetic processing. Of course, other than signal processing means other than LPF, BPF, and noise filter10, it is possible to generate azimuth data A that focuses on suppressing drift rather than response speed, and to achieve faster response speed by suppressing drift. Any signal processing circuit or arithmetic processing capable of realizing the generation of the azimuth data B with the main focus may be used.

According to the above configuration, one head direction having both functions of the first head 15 direction detection unit 5A and the second head direction detection unit 5B used in the third embodiment. The use of the detection unit 5C makes it possible to reduce the size and weight of the azimuth detection unit, thereby reducing the weight of the HMD 2 and greatly contributing to improved usability.

The sensors of the azimuth sensor unit 14 include, for example, an acceleration sensor, a gyro sensor, a gravity sensor, a magnetic sensor, a geomagnetic sensor, and the like, and are configured by one type or a combination of a plurality of types. ing. Of course, the sensors in the head direction detecting unit 5, the first head direction detecting unit 5A, and the second head direction detecting unit 5B which constitute the direction detecting means in the first to third embodiments are also provided. The same is true.

Further, in the first to fourth embodiments according to the present invention, the virtual sky is displayed without any sense of incongruity through the 25 small-sized image display device 7 and the aerial image projection optical system 8 of the HMD 2. Although the configuration for displaying the aerial image in between has been described, it is needless to say that, although not shown, audio corresponding to the aerial image can also be played back in conjunction.

Further, in the first to fourth embodiments, the head direction data relay unit 9, the head direction data storage unit 10, the azimuth shift operation unit 11, and the image signal processing unit (At measurement Although the configuration incorporated in the controller 3 has been described, the present invention is not limited to this. If each of these main constituent circuit groups is small and lightweight, the main unit 2A of the HMD 2 may be used. It may be configured so as to be built in the.

Further, the present invention is not limited to the first to fourth embodiments, and various changes and modifications can be made without departing from the spirit of the present invention.

Industrial applications

As described above, the head-mounted image display system device according to the present invention is a virtual reality display (HMD) having a head tracker having a head tracker for detecting the orientation of the user's head. (VR) system II, as an image display system adopted as a visual display device for mixed reality (MR) systems, and in various fields such as medical and academic fields where it is desirable to construct VR and MR systems, or in entertainment It is effective as an image display system in this game, and is particularly suitable for displaying images according to the head orientation in real time by reducing the image display delay due to 20 image generation arithmetic processing by a computer or the like.

twenty five

Claims

Range of request

1. Display means detachably attachable to the user's head, azimuth detecting means installed on the display means for detecting at least the horizontal azimuth of the user's head, and A head-mounted image display system device comprising: an image generation device that generates an image according to the orientation of the user's head, and displays the generated image on the display unit.

An azimuth shift calculating means for calculating a difference between the current head azimuth data obtained by the azimuth detecting means and the head azimuth data a predetermined time ago; and10 an azimuth displacement obtained by the azimuth shift calculating means. An image processing means for shifting at least a horizontal position of an aerial image displayed on the display means in accordance with an amount of the image data.

2. Display means that can be detachably attached to the user's head,

Azimuth detecting means installed on the display means for detecting at least a horizontal azimuth of the user's head;

An image generation device that generates an image according to the direction of the user's head from the direction detection unit, and displays the generated image on the display unit;

An azimuth shift calculating means for calculating a difference between the current head azimuth data obtained by the azimuth detecting means and the head azimuth data a predetermined time ago; 20 an azimuth displacement obtained by the azimuth shift calculating means Image processing means for at least horizontally shifting the position of the aerial image displayed on the display means according to the amount;

Head-mounted image display system device characterized by comprising:

twenty five

3. The head-mounted image display system described in claim 1 or claim 2. In the apparatus, the azimuth change calculating means and the image processing means are incorporated in the display means or a controller for controlling display of the display means.

4. The head-mounted image display system according to claim 1 or claim 2, wherein the azimuth detecting means includes two different five first and second azimuth detecting units for detecting the head azimuth of the user. The first azimuth detecting unit measures an azimuth necessary for the operation of the azimuth change calculating means, and the second azimuth detecting unit is required by the image generation device for image generation. It is characterized by measuring azimuth.

5. The head-mounted image display system 10 according to claim 1 or claim 2, wherein the image processing unit performs signal processing on a generated image signal from the image generation device, so that the display unit It is characterized in that the position of the aerial image to be displayed is shifted only in the horizontal direction.

6. The head-mounted image display system according to claim 1 or claim 2, wherein the fixed time is 15 hours required for the image generating device to generate an image, and a fixed time measuring means for actually measuring the time is provided. It is characterized by that.

7. The head-mounted image display system according to claim 6, wherein the fixed time measuring means is a dummy direction signal for notifying the image generating device of a change in head direction in order to measure the fixed time. And analyzing the image signal generated and output by the image generation device and measuring the time required for the image generation device to generate an image based on the signal.

8. In the head-mounted image display system according to claim 6, the predetermined time measuring means measures 25 times of generation of a plurality of pattern images defined in advance, and calculates a time determined by statistically processing them. The fixed time Characterized by:

9. Head-mounted image table according to claim 1 or claim 2:

In the apparatus, the azimuth detecting means generates azimuth data necessary for the calculation of the azimuth displacement calculating means and azimuth data required for the image generation device to generate an image. Five

10. The head-mounted image display system according to claim 9, wherein the azimuth detecting means includes a first signal processing for generating first azimuth data input to the azimuth displacement calculating means. Means, and second processing means for generating second azimuth data to be input to the image generation device. 10 11. The head-mounted image display system according to claim 10, wherein the first signal processing means performs processing with a high response speed.

12. The head-mounted image display system according to claim 11, wherein the response speed is approximately a video rate or a speed higher than the video rate.

13. The head-mounted image display system according to claim 11, wherein a response time of the first signal processing means is substantially 1/15 sec or less.

14. The head-mounted image display system according to claim 10, wherein the second signal processing means performs processing for suppressing drift.

15. An azimuth detecting procedure for detecting at least the horizontal azimuth of the user's head by the azimuth detecting means provided on the display means detachably attachable to the user's head; Generates an image according to the orientation of the user's head detected in Image generation procedure to be performed,

An azimuth shift calculating procedure for calculating a difference between the current head azimuth data obtained by the azimuth detecting means and the head azimuth data a fixed time ago to calculate an azimuth displacement amount;

An image processing step of, at least horizontally shifting a position of an aerial image displaying the generated image on the display means, according to the calculated azimuth displacement amount;

An image processing method for a head-mounted image display system device, comprising:

16. The image processing method for a head-mounted image display system according to claim 15, wherein the azimuth detecting step comprises: detecting the head orientation of the user using two different first azimuth directions. A first and second azimuth detecting unit, wherein the first azimuth detecting unit measures the azimuth necessary for calculating the azimuth displacement amount;

A step of measuring the azimuth required by the second azimuth detecting unit to generate the image, and

Characterized by having.

20

twenty five