US20070150138A1

US20070150138A1 - Memory management in event recording systems

Info

Publication number: US20070150138A1
Application number: US11/296,907
Authority: US
Inventors: James Plante
Original assignee: Individual
Current assignee: SmartDrive Systems Inc
Priority date: 2005-12-08
Filing date: 2005-12-08
Publication date: 2007-06-28
Also published as: US20090222163A1; US8880279B2; GB0811334D0; CA2632738A1; US20140098228A1; GB2447184A; US20160117872A1; US9226004B1; US9911253B2; CA2632738C; GB2447184B; US8374746B2; WO2007067775A3; WO2007067775A2

Abstract

In vehicle event recorders which capture video as discrete image frames, a managed loop memory and management system is provided to realize a virtual ‘timeline dilation’ effect. For a buffer memory of limited size, the maximum extent of a video series in time is extended by trading a reduction in temporal resolution for an increase in temporal range. Memory cells are overwritten in an ‘interleaved’ fashion to effect a reduced frame rate for certain periods in relation to an event moment. In time periods furthest from the event moment, an effective frame rate is minimized while at time periods closest to the event moment, an effective frame rate is maximized.

Description

BACKGROUND OF THE INVENTIONS

1. Field
The following invention disclosure is generally concerned with electronic data storage and specifically concerned with memory allocation and writing schemes in vehicle event recording devices.
2. Prior Art
Video recording systems are commonly used to monitor places where activity sometimes includes that which a record is desirable. For example, in a security zone criminal activity may be recorded if video monitors are arranged about space to be monitored. When an incident occurs, a video record of the criminal activity is available from the recording system. During periods when no cruel activity occurs, a considerable amount of data is generated by the video recording system. However, this data has little or no value. Thus, it can be readily discarded without loss. The act of ‘discarding’ data amounts to merely rewriting new data over old recorded data. Indeed, most video security systems are arranged with a recording medium that is reused continuously. When a video camera generates enough image data to fully consume available memory, additional collected data is recorded at the beginning of the memory. The act of writing newly acquired data is also an act which discards the old the data; i.e. the old data is lost to the ‘write’ operation with respect to the newly collected data.
In old videotape systems, this is sometimes called a ‘round robin’ arrangement. A memory medium fashioned as a tape in a continuous loop provides the data storage for these video security systems. In such systems the tape has no end, and no beginning—but rather the tape continuously passes a recording head where new images are written to the tape at the same time old images are discarded.
When an incident of interest occurs, the tape may be stopped to prevent loss of data which relates to the important incident. View images may be recovered from the tape and transferred to a permanent medium, while the tape is returned to the video system for further recording. Such re-use of memory is well known in these arts.
In a round-robin scheme, the data which is being overwritten (discarded) is the data which came into the system earliest—or was ‘first in’ the system. Sometimes recall the system's “first-in, first overwritten”—which is analogous to its close cousin, “first-in, first-out” well known from electronic buffer systems. In both cases, we refer to this method as FIFO.
A FIFO base system is generally a very good system for buffer management. This is due to the fact that the oldest data in a buffer is typically the least valuable. Therefore the oldest data, or the least valuable data, may be discarded without regard for its loss. It is not always the case in buffer systems that the earliest received data, the ‘first-in’ data, is the least valuable data in some cases is better not to overwrite the oldest data, but rather to provide an overwrite scheme which preserves certain special portions of the oldest data.
Vehicle event recorders are video recorder systems mounted within the vehicle to provide a video record relating to the environment surrounding a vehicle during its operation. These systems are well known and have been made famous for their use in conjunction with police activity. Many police departments in the United States are equipped with vehicle event recorders, which capture activity, sometimes criminal, which occurs in the presence of a police vehicle.
Of course, the vehicle event recorders use is not limited to police vehicles. More and more commercial vehicles are being equipped with systems to record activity associated with the use of the vehicle and within the environments in which the vehicle is used. The systems are particularly advantageous to fleet vehicles which are occasioned by heavy professional use and frequent incidents. These incidents may include traffic type accidents, theft, vandalism, among others. With a video record, vehicle fleet managers are better equipped to manage and control costs associated with operations of large vehicle fleets. Safety is improved, driver of performance is improved, confidence is gained in an understanding of accidents which do occur, and other benefits are associated with use of vehicle recorder systems.
Some of the more advanced vehicle recorder systems are described in U.S. Pat. Nos. 6,389,340; 6,405,112; 6,449,540; and 6,718,239. These inventions are all presented by inventor Rayner of San Diego, Calif. Primarily these inventions relate to a small device, which is mounted to vehicle rearview mirror to capture video images of traffic incidents ahead of the vehicle.
The '112 claims in combination of vehicle which includes a vehicle operator performance monitor. This monitor presumably records video of the vehicle operator, which may be used to determine how the operators actions affect use of the vehicle.
The '540 patent is an event recorder mounted in a vehicle which includes one or more wave pattern detectors for detection and recognition of the presence of the predetermined wave produced external the vehicle and for producing a trigger signal denoting predetermined wave presence. A detective wave is of the type which is produced by a police or fire department emergency vehicle. Detection of this wave triggers a capture function which stores video images a long-term storage memory.
The '329 patent includes a one-way hash function to perform a validation function. In this way the integrity of the video data which is recorded can be protected.
Finally, Rayner teaches in the '340 patent a very important relationship between two different types of memory. A first memory is arranged to store video for a short-term, and to transfer some of that stored video in response to a trigger event. Data from this short term memory is transferred to a more durable and long-term memory. Did in the short term memory is continuously overwritten in a scheme which is described by Rayner has “first-in, first-overwritten”. In this way Rayner couples a high-speed, high-performance volatile semiconductor memory, with a flashlight memory good for long-term storage of large amounts of data even when power is removed. As will be described in detail later, Rayner's first in first overwritten scheme necessarily creates a loss of important and valuable data.
While systems and inventions of the art are designed to achieve particular goals and objectives, some of those being no less than remarkable, these inventions have limitations which prevent their use in new ways now possible. Inventions of the art are not used and cannot be used to realize the advantages and objectives of the inventions taught herefollowing.

SUMMARY OF THESE INVENTIONS

Comes now, James Plante with inventions of memory allocation schemes in vehicle event recording systems including devices and methods. It is a primary function of these systems to provide improved memory allocation and writing schemes to preserve data over extended time periods. It is a contrast to prior art methods and devices that those known systems do not record over a large time range with respect to a prescribed size-limited memory. A fundamental difference between memory allocation of these newly presented inventions and those of the art can be found when considering its time dilation, which may be applied on the extremities of discrete capture periods. The invention thus stands in contrast to methods and devices known previously.
A memory of limited size is subject to an advanced managed loop memory allocation scheme. The storage frame rate is adjusted throughout a prescribed capture time period. At the time period extremities, a frame capture process is subject to reduced frame rate. Nearer to some instant of interest, an ‘event moment’, the frame rate is increased to improve detail (temporal) around that particular time. The scheme permits one to accommodate a greater temporal range at the expense of temporal resolution. Losses in temporal resolution are judiciously pushed away from the event moment and allocated to the capture time period extremities.
An overwrite scheme selects which frames are expired and subject to discard. At any random moment, video is continuously captured at a maximum frame rate. However, these frames are not put into memory in a conventional first-in, first overwritten manner, but rather, these frames are added to the memory locations which are determined to be available in accordance with the overwrite scheme. It is strictly not the case that the oldest frame is overwritten. Quite contrarily, an overwrite action is generally applied to a frame which is newer than at least one other frame stored in the memory.
As a result, newly acquired frames are placed into memory in processes which necessarily cause older frames to be discarded. However, the newly captured frames are written to memory positions in an ‘interleaved’ fashion whereby some of older frames are preserved. When a capture event occurs, data in memory may be transferred to a more permanent storage. When data is transferred, a timeline is reconstructed. The recorded timeline is unique in that it contains various frame rates over the capture period. At both the beginning and end of the capture period, the frame rate is modest. At the point of greatest interest in the capture period, the frame rate is maximum. This throttling of frame rate, permits a memory of given size to accommodate a timeline of greater temporal extent.

OBJECTIVES OF THESE INVENTIONS

It is a primary object of these inventions to provide high-performance memory allocation systems for vehicle event recorders.
It is an object of these inventions to provide for set memory allowances, and extended record period.
It is a further object to provide memory overwrite schemes which provide a time dilation on either end of a record period.
A better understanding can be had with reference to detailed description of preferred embodiments and with reference to appended drawings. Embodiments presented are particular ways to realize these inventions and are not inclusive of all ways possible. Therefore, there may exist embodiments that do not deviate from the spirit and scope of this disclosure as set forth by appended claims, but do not appear here as specific examples. It will be appreciated that a great plurality of alternative versions are possible.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

These and other features, aspects, and advantages of the present inventions will become better understood with regard to the following description, appended claims and drawings where:
FIG. 1 is a timeline illustration in proximity with a graphic to illustrate plurality of memory bin units;
FIG. 2 is a similar timeline illustration and memory graphic illustrating an overwrite operation;
FIG. 3 further illustrates the overwrite operation in conjunction with a trigger event;
FIG. 4 illustrates particular memory bins with various importance associated therewith;
FIG. 5 illustrates an advanced overwrite scheme to protect certain ‘high-value’ video frames;
FIG. 6 further illustrates this overwrite scheme near the end of memory space;
FIG. 7 illustrates in detail, memory allocation with pointers to memory bins which must be saved and pointers to those bins which may be erased;
FIG. 8 illustrates four alternative versions of these inventions of importance which work equally well within the spirit of the overall teaching;
FIG. 9 is a block diagram illustrating important elements of the apparatus of these inventions;.
FIG. 10 is a block diagram directed to the most general methods of these memory management systems; and
FIG. 11 is a more detailed-block diagram of these methods with greater specificity.

GLOSSARY OF SPECIAL TERMS

Throughout this disclosure, reference is made to some terms which may or may not be exactly defined in popular dictionaries as they are defined here. To provide a more precise disclosure, the following terms are presented with a view to clarity so that the true breadth and scope may be more readily appreciated. Although every attempt is made to be precise and thorough, it is a necessary condition that not all meanings associated with each term can be completely set forth. Accordingly, each term is intended to also include its common meaning which may be derived from general usage within the pertinent arts or by dictionary meaning. Where the presented definition is in conflict with a dictionary or arts definition, one must use the context of use and liberal discretion to arrive at an intended meaning. One will be well advised to error on the side of attaching broader meanings to terms used in order to fully appreciate the depth of the teaching and to understand all the intended variations.
Vehicle Event Recorders
Vehicle event recorders are video image recording systems which are responsive to triggers indicative of some event of interest.
Time Dilation
For purposes of this invention time dilation refers to an expansion of a video sequence timeline by way of frame rate manipulation.
Trigger
A trigger is electronic means for setting some instant in time associated with a particular event of interest and further for causing initiation of some associative processes.
Expanded Timeline Definition
An ‘expanded timeline definition’ is a prescribed rules set which sets forth and defines a timeline associated with a video frames sequence having more than one frame rate associated with any particular portion of the timeline.
Overwrite Manager
An ‘overwrite manager’ is a computer module, which determines in accordance with an expanded timeline definition which data recorded in memory and associated with a particular video frame is subject to discard and may be overwritten with data from a newly collected video frame.

PREFERRED EMBODIMENTS OF THESE INVENTIONS

In accordance with each of preferred embodiments of these inventions, there is provided apparatus for and methods of memory overwrite schemes in vehicle event recorder systems. It will be appreciated that each of the embodiments described include both an apparatus and method and that the apparatus and method of one preferred embodiment may be different than the apparatus and method of another embodiment.
Video event recorder systems are typically built around and deployed with memories of limited size. This is due to the fact that, vehicle event recorder systems may now be built very cheaply. While it is possible to include in such devices mass storage and mass storage management, for example a typical computer-type disk hard drive, this component remains quite expensive. Overall systems may be double in cost, if such memories were included. Instead, a ‘lightweight’ memory solution is embraced. An abbreviated memory or memory buffer is used to temporarily store information collected during the service day of a vehicle equipped with such video event recorders. Upon return to a base, a vehicle may transfer collected information to a more permanent memory for management and analysis. In this way it is possible to equip vehicles with video event recorders having very inexpensive cameras and memory.
Accordingly, memories of such video event recording systems may be preferably handled in the following manner. A memory system is divided into two portions: a fast, managed loop memory buffer, and a temporary mass storage memory. Video continuously received from a video camera may be put into the fast memory buffer. However, the amount of data generated by a video system is quite extensive and most of the time totally uninteresting. However certain select portions of video captured may become of great interest. For example, when a vehicle is involved in a traffic accident, captured video may yield important clues as to fault, cause, identity, response, among others. In this case, it is important to preserve video data associated with these select video capture periods.
To effect this, a trigger is arranged whereby the occurrence of some incident of interest, such as an automobile accident, causes data stored in the memory buffer to be transferred to a more permanent memory facility. Old data in the memory buffer is continuously overwritten by new data received from the video camera in real-time. In common and simplistic versions, this step is performed in a “first-in, first-overwritten” manner. However, there are great disadvantages to doing this. As the memory buffer is limited in its capacity to store video frames, “first in first overwritten” schemes provide a timeline of correspondingly limited extent. For example, at a frame rate of four frames per'second, a given memory buffer may be suitable for storing 30 seconds of video frame data—or a 30 second video timeline. In ‘first-in, first overwritten’ schemes, this timeline may be arranged as 15 seconds of continuous video before a trigger event and 15 seconds of continuous video after a trigger event.
However it is not necessary nor advantageous to maintain a continuous frame rate throughout the entire event capture period. It is possible to have a modest frame rate at times associated with the capture period extremities, and a high frame rate during periods around an event trigger. It is for suggested here that the storage frame rate be adjusted throughout a prescribed capture time period. Such a system allows an extended temporal range instead of a 30 second timeline, is entirely possible to have a 48 second timeline for the same memory. Such a timeline may be embodied as 12 seconds of video at a frame rate of one frame per second for the periods of time furthest from the event trigger, both before and after. In addition, the video sequence may include video for 24 continuous seconds, 12 seconds before and 12 seconds after an event trigger, at a video frame rate of four frames per second. In this way, the temporal range is extended but the temporal resolution is compromised in the time periods furthest from the trigger event.
To bring about this managed loop memory buffer management system, an overwrite scheme is provided to select which frames are ‘expired’ and no longer part of the particular extended timeline scheme. What is reminded that video is continuously captured at all times; and further this video is captured at the maximum frame rate. This is necessary because it is not possible to know whether or not an event trigger will be coming in the next instant. Accordingly, the system always captures video at the maximum frame rate as the capture frame rate cannot be adjusted in view of any event trigger which may come in the future. Video captured at the maximum frame rate is put into the memory and as it is put in the memory it displaces previously recorded video frames. These frames are added to the memory locations determined to be available in accordance with a prescribed overwrite scheme such as the one mentioned. However, this step is provided in striking contrast to a first in first overwritten scheme. In these novel systems, most frames being overwritten are actually newer than at least one other frame stored somewhere in the memory buffer. Newly captured frames are written to memory positions in a pseudo—‘interleaved’ fashion where some of olderframes are preserved.
When a trigger event occurs, data in memory is transferred from the memory buffer to a memory of more permanent nature. When this data is transferred, an expanded timeline is reconstructed as a timeline having at least two frame rates. At the extremities of the capture period timeline, the frame rate is reduced. At and about the point of greatest interest (trigger event) in the capture period the frame rate is maximized. This ‘throttling’ of frame rate, permits a memory of preset size to accommodate a timeline of greater temporal extent; albeit with reduced resolution in some places.
A better understanding of these inventions may be had with reference to drawing FIGS. 1 through 11 and the reference numerals set forth therein. In particular, FIG. 1 illustrates a simple example timeline associated with a memory system which has been divided into a plurality of memory bins. To develop the example, some arbitrary numbers for memory size, number of bins, video frame rates, et cetera, are selected. It is to be understood that these are not necessarily preferred values, but rather they are sensible and realistic values which promote a more clear understanding of the illustrative example.
The memory in question, the high-speed, high-performance memory, of limited extent. This memory is arranged as a buffer. It is put into communication with incoming video data recorded by a video camera. It's output, is directed to another data storage means, but one perhaps a memory system having far greater capacity but less speed. A memory suitable for use in this fashion includes in example a semiconductor DRAM type memory. Alternatively a non-volatile high-performance memory for example those arranged about ferromagnetic principles. This memory is primarily characterized in that it is quite fast and responds in real-time to video collected by a video camera, however it is of limited size and not suitable for saving the mass amounts of data generated by video image systems.
We can say for purposes of a useful example that this memory size is limited to the few megabytes. The memory is arranged to temporarily hold a limited number of video frames which may or may not be transferred to a more permanent memory in a transfer operation. It is convenient and useful for a clear illustration to say that the memory is divided into 120 bins; each been be sufficient for storing the data associated with a single video frame.
We also associated timeline 1 with this memory. This timeline is comprised of a 30 second time interval. The timeline is marked in the Figure from 0 to 30. A one second interval 2 is illustrated at the beginning of the timeline. Further, that one second interval is divided into quadrants, representing a quarter of a second interval 3. For the video systems of immediate interest, this quarter of a second interval nicely accommodates a single video frame (implicitly setting a frame rate of four frames per second). While most modern video systems have far higher performance than recording four frames per second, four frames per second is a useful rate for vehicle recorder systems which tend to have limited memories in the interest of maintaining low-cost. Further, the kinds of events being recorded in vehicle recorder systems are well served by frame rates of a few frames per second.
When video images are captured by a camera, frame-by-frame, each frame images can be recorded into a memory bin 4. A first frame is recorded and put into a first memory bin. Thereafter, a quarter second later, a second frame is recorded and put into an another memory bin—perhaps an adjacent bin. This frame-by-frame recording scheme may continue for up to 30 seconds before all memory bins becomes full and the supply of empty bins is exhausted. In the figure, the first 116 memory bins are shaded to indicate that one frame each of video data has been written to those bins. This is equivalent to recording of a video signal 5 of four frames per second for 29 seconds. The Figure illustrates four empty memory bins 6, which would be filled in the next second of video recording. Recording video images in this manner, is well-known and commonly found in the arts; as such, the Figure is labeled (prior art).
FIG. 2 illustrates a similar timeline 21 in conjunction with a graphical illustration of a memory having 120 memory bins. As presented in FIG. 1, a time interval equivalent to one second 22 as well as a time interval of one quarter second 23 is illustrated for reference. The graphical depiction of the memory includes lightly shaded areas 24 and 25. The memory bins presented as 24 represent those bins having data written thereto from video which was collected from a time t=14 up to a time t=30. The demarcation indicated as dotted line 26 reminds us where time t=14 is. At time t=30, the memory is completely full. Video data collected for 30 seconds at four frames per second fills 120 memory bins. The video data collected at time t=31 cannot be saved to memory unless a portion of the memory already allocated and consumed in a previous data write step is overwritten. Thus in the graphic, we present memory bins indicated by 25 on a second line to represent that video frame data is recorded in these memory bins at the expense of data captured 30 seconds prior. Accordingly, for the time period indicated, i.e. video data collected from t=0 to t=14, the data is lost to an overwrite step. In the figure, those bins shaded dark are indicated as 27, represent the ‘over written bins. This effectively illustrates the so-called ‘round-robin’ or ‘first-in, first-overwritten’ FIFO memory management schemes. Since these schemes are very well-known, this figure is also labeled (prior art). The FIFO memory management scheme is very useful. When a new video frame is collected by the video camera it is placed into memory at the same location as the oldest frame in the memory which is discarded in the overwrite step. The FIFO memory management scheme implies the oldest video information in the memory is the least valuable.
The memory described is a buffer memory. That is, this memory temporarily holds the data of a video series for some specified time, but also continuously discards previously recorded information. When the buffer contains a data set associated with an important event, that data is transferred from the buffer memory to a more permanent memory before becoming subject to lost by overwrite actions. A video series becomes ‘important’ when a detectable event occurs which implicitly indicates video is valuable. For example, if a vehicle is involved in a traffic accident, accelerometers can detect the accident and trigger a transfer of data from the buffer memory to a permanent memory.
In those vehicle event recorder systems, a trigger is sometimes arranged to indicate that such an event has occurred; an event for which the video images associated therewith may be of extreme importance. In this case, the short term buffer memory of 120 video frames, should be transferred to a more permanent long-term memory for example, a durable flash type memory.
FIG. 3 is directed to illustrate a timeline which includes an event moment. FIG. 3 includes a timeline 31, and the dashed line 32 to indicate the 29th second along with a marker ‘X’ 33 to indicate a trigger event has occurred at the 29th second. When a trigger event occurs, it is important to preserve the video data which occurred after the accident as well as the video data which occurred before the accident. Video images collected during a time period starting 15 seconds before the accident are in the bins indicated by 34; i.e. those video image frames collected between t=14 and t=29. Memory bins at the end of the time line indicated by 35 include four video frames collected during the first second after the accident. Video image frames collected between t=30 and t=44 are placed in the memory bins indicated by 36. Thus the memory buffer contains video images for 15 seconds prior to the accident and 15 seconds after the accident; the memory of limited size can only hold video image data which represents 30 seconds of video recording.
At this point in time, no new frames are recorded to memory; overwrite is prevented, and the memory buffer is “locked”. Rather, the system pauses to transfer data in the buffer memory to a permanent flash memory. After data is successfully transferred to flash, the buffer is “unlocked” and may be used again in the fashion described. It is useful to note that as video data which was placed into buffer memory bins between time t=30 and time t=44, it caused older data to be displaced, overwrote and forever destroyed. Data which was recorded between t=0 and t=14 is completely lost and we have no access whatever to this information which at one time resided in those memory bins—that information was destroyed in the overwrite step. It is first suggested here that some of this information may be very valuable and it is quite undesirable to lose it entirely. Indeed, it is suggested that some of this data is more important than data which is being saved in its place.
Since the moments which led up to a vehicle accident sometimes can explain a great deal about the complete story, it is highly desirable to have at least have some limited information relating to the scene at t=1 for example. If we can just see one frame at t=1, it may prove to be extremely valuable in explaining what happened in the accident. Thus one can argue, that the oldest frame in the memory at a time t=30; one second after the accident, is not in fact the ‘least’ valuable frame in the memory. In this case at t=31, the FIFO scheme may be actually destroying critically useful data.
This is more readily understood, in consideration of FIG. 4 which explicitly shows certain bins A-F associated with various points of the timeline 41 and with reference to trigger event 42 time at time t=29. The following discussion further illustrates the importance of those bins A-F.
In a FIFO system, all memory bins, indicated by 44 and 45 are preserved in the memory buffer. Amongst the oldest recorded video frames remaining are those which lie in memory bins A and B. These represent two adjacent frames; frames which are captured within a quarter of a second from each other. Since these frames represent images very close in time, these frames are expected to be quite similar to each other. While it is sometimes desirable in video systems to have high temporal resolution, i.e. as many frames per second as possible, one will appreciate that at higher frame rates, adjacent frames will contain much the same information as those closest thereto. Accordingly, where memory is limited these adjacent frames loose their importance as most of the information contained in each frame is similarly contained in the adjacent frame. Thus, if we keep frame A and discard frame B, most of the information of frame B can be known by examining frame A.
On the other hand, frames D, E and F, which are discarded in a FIFO system, may actually contain extremely important information. Frame D is separated from frame E in time by one second. In a video scene, there may be considerable differences between one frame captured an entire second later than another frame. Further, frame D occurs a full 29 seconds before the trigger event. In a traffic accident, it can be quite useful to know about what was happening a time periods further, both before and after, from a trigger moment. Thus it may be possible in a memory having a finite number of memory bins to trade some of the bins associated with less important time slots for bins associated with time slots having a greater importance. If we discard frame B, and preserve frame D, we may gain a greater overall understanding of the incident being recorded. In effect, we can trade some time resolution (frame rate) at t=15, for improved overall temporal range to realize an extended timeline.
The careful observer will notice if we preserve video data associated with a frame rate of one frame per second, in seconds 1-12, then we will still have available to us 36 memory bins into which will accommodate newly captured video data. Thus, rather than completely overriding the oldest video data in memory, one can perform an overwrite action on 3 of every 4 memory bins in the overwrite portion of the timeline, thereby maintaining ¼^thof the oldest video data in those memory bins. That is to say, for the oldest video data in memory it may be useful to save one frame per second. To effect this, when the overwrite operation is executed, new data is written to three memory bins, before one bin is skipped. This is repeated.
Timeline 51, includes a trigger event 52 at time t=29. In a particular overwrite scheme of interest, it is prescribed that a timeline be comprised of 12 seconds of low temporal resolution, 24 seconds full temporal resolution and a further 12 seconds of low temporal resolution. This is further defined in detail as: a 12 second period of one frame per second video, a 24 second period of four frames per second video, and finally a 12 second period of one frame per second; for a total video sequence of 48 seconds. Since it cannot be known at what time in the future an event trigger will occur, a data overwrite scheme must preserve data associated with various frames from which a prescribed timeline is comprised. In the example being developed here, continuous video data at a frame rate of four frames per second is preserved for a period of 12 seconds 54 before the trigger event; that data is in memory bins indicated by 53. It is acknowledged that in the FIFO system one can preserve data at four frames per second for up to 15 seconds before and after the trigger event. The newly proposed system, it is suggested that only 12 seconds of four frames per second data be kept. However, it will be shown for this small price, we can dilate the total timeline of the video sequence to 48 seconds in contrast to the 30 second timeline of the FIFO system.
In the 31st second, the first overwrite operation begins. Whether or not a trigger has occurred, newly captured video data is written to every three out of four memory bins leading the fourth memory been undisturbed. That is to say old data is preserved, albeit at one quarter of the frame rate, from which it was originally recorded. Video data after the trigger event is recorded in the memory bins 55 a frame rate of four frames per second. Just because some bins are skipped, it is not necessary reduced the frame rate of video data collected after the trigger event. This is readily understood in consideration of the time point indicated by 57 which indicates the time t=41 seconds. Without skipping bins, this point in memory would be time t=45. Careful observation will prove that the bins indicated by 55 will accommodate data at four frames per second for the entire 12 seconds after the event trigger. After time point indicated by 57, there remains several memory bins available for further overwrite operation before reaching the memory bins which contain data which must be preserved in agreement with the timeline definition 12/24/12. At least some of those memory bins up to the position indicated by 54 are available for overwrite. After the full 12 seconds of four frames per second video is recorded, it is desirable to continue recording video data at one frame per second or additional 12 seconds. Data captured in this period can be stored in memory bins, which are scattered in various locations about the memory buffer. FIG. 6 will help illustrate where these positions lie.
FIG. 6 illustrates memory bin locations which are available for overwrite as the memory approaches its full capacity for the particular schemes presented here. Once a trigger event occurs, i.e. is set in time, it is possible to compute which video frames must be saved in accordance with the particular timeline definition, and which ones are suitable for discard. To further our illustration, we suggest that 48 frames at four frames per second are preserved immediately before the trigger event. In addition, 12 frames at a video rate of one frame per second are preserved for the time t=5 up to t=17. These frames must be protected from any further overwrite operation. i.e. these frames are marked “must be saved” in the figure. These frames are saved as they are included in the timeline definition. All frames which precede t=5 are in condition for discard. That is, they lie outside the time range which is to be preserved. Accordingly, frames indicated as 69 for example have aged sufficiently and are okay to erase. These are frames which originally were preserved in the overwrite operation as skipped frames.
Video frames captured after the trigger event are also saved in the memory. For 12 seconds after the trigger event, t=29 to t=41, video is captured at a rate of four frames per second. That video data 65 is put into memory in accordance with the need to save particular frames of the oldest video data. When all video frames from the period t=29 to t=41 are properly recorded, the system continues to record data a frame rate of one frame per second. This is distinct, from the earlier operation where the overwrite action resulted in preservation of one frame per second. For the time period 12 seconds after the event trigger up to 24 seconds after the event trigger, data is put into memory at a reduced frame rate of one frame per second. Other frames may be captured by the camera, but they are discarded before entering the memory or instantly thereupon. Thus frames represented by 67 are put into memory bins which are available in accordance with the “OK to erase” label in the drawing. A careful observer will note that after three of these frames are placed in the memory, the fourth frame 68 cannot be placed into the memory in the same repeating geometric position. That is to say, those memory bins are not available for overwrite. As such video captured after that time, must be carefully managed and fit into memory bins which are available.
FIG. 7 illustrates the steps taken in the final filling of the remaining memory bins. The reader will be reminded of the timeline 71 and the event trigger 72 at time t=29. In agreement with the example timeline definition, video captured at a frame rate of four frames per second from t=17 to t=29 is stored in memory, as indicated by 73. Similarly, video captured for a 12 second period at a frame rate of four frames per second from t=29 to t=41, is stored in memory as shown 74. Finally, video frames captured during a 12 second period from t=42 to t=54 at a frame rate of one frame per second include those particular frames represented as 75. Those frames must be inserted into the memory bins which remain available for overwrite. Arrows 76 indicate that these frames may be placed in locations near the memory beginning where data had once been stored but is now expired due to the fact that the trigger event occurs at t=29. Once a trigger event is established, the bins which may be overwritten is determinable in an agreement with the particular rules which define the timeline.
The specific example presented clearly illustrates that careful management of an overwrite scheme permits a memory buffer to dilate a timeline by manipulation of which video frames are preserved and which are overwritten. In effect, temporal resolution is sacrificed to extend temporal range. That is to say, the frame rate of “saved data” is altered in order to make more space for video frames captured further in time from the event trigger. In this way, the greatest amount of information can be preserved in a memory buffer of the limited size. While the example illustrates in exhaustive detail where data is written in memory for illustrative purposes, experts will note there is really nothing sacred about the physical positions of memory bins. Therefore, it is explicitly set forth here that after a timeline definition is set, it is a simple matter to prepare an algorithm which defines at any time the bins which contain data that has expired and thus implicitly defines a bin available for overwrite.
While the specific example presented in FIGS. 5-7 nicely illustrates one elegant solution, it should be well understood that many very useful arrangements will permit a time dilation in accordance with the spirit of these inventions. Particular values used may be different than those presented in the example timeline definition. In another example timeline definition, one might arrange a system whereby two periods of eight seconds are used to capture video of high frame rate, and two periods of 28 seconds are used to capture data at a low frame rate—thus achieving a total expanded timeline of 72 seconds. The usefulness of these inventions does not depend upon the particular values chosen in these examples. Further, one should recognize that since capturing/saving video at two different frame rates allows one to expand the timeline, that similarly capturing/saving video at three different frame rates also allows one to expand the timeline with more flexibility. That is, it is possible to manage the memory such that for some time periods frames are preserved at a rate of four frames per second, in other time periods, frames are preserved at a rate of two frames per second, and still further in other time periods frames are preserved at a rate of one frame per second. In this way, we can maintain very high temporal resolution for the periods immediately surrounding an accident (trigger event), to have a medium level resolution for periods further away from the trigger event, and finally at the furthest extremities of the time range we can have very low temporal resolution. Such extensions should not be considered new invention, but rather a subset of inventions first taught in this disclosure.
In addition, we explicitly anticipate asymmetric timeline definitions. That is, the time periods on either side of the event trigger may not be equal in extent or number. It is possible to devise a timeline definition having a long, high resolution period before the event trigger and a short high resolution period after the event trigger. To illustrate various timeline definitions of interest, the FIG. 8 has been prepared with several examples each working equally well within the common concept of timeline dilation. FIG. 8 graphically illustrates a first memory buffer 81 example presented in detail above. In this example, there are two frame rates, a high video frame rate of four frames per second and a low video frame rate of one frame per second. A trigger event 82 which occurs at some instant in time implicitly sets the time periods for any particular example. In the example presented, time periods 83 starts immediately after the trigger event and extends for 12 seconds. A second time period 84 extends from the trigger event to 12 seconds prior to the trigger event. In both of these time periods, video captured and put into the memory buffer at a rate of four frames per second. The number of shaded memory bins reflect a frame rate of 4 frames per second. Time periods at the extremities of the timeline, periods 85 and 86, are each also configured to be 12 seconds in length. However, since only one frame per second is collected in those time periods, the number of memory bins consumed is considerably smaller; i.e. ¼ those consumed in the other time periods. This arrangement provides a total timeline of 48 seconds. In memory buffers which do not overwrite/store data at variable rates, the same memory size would only be able to accommodate a timeline of 30 seconds. FIFO memories of the same size are restricted to 30 seconds.
Example 2 presented as 87 in the drawing figure, suggests two high temporal resolution periods of 10 seconds each. In addition, there are two low temporal resolution periods each of 20 seconds. While there is a reduced overall period of high-resolution video data, the total timeline is extended to 60 seconds.
Example 3 is presented in the memory buffer of graphic 88. This important example illustrates that is possible to configure an asymmetric timeline definition. It is not necessary that the two periods in which high video rate recording occurs are the same in extent. Indeed it is possible to record video at a high frame rate for a longer period after a trigger event than that period immediately before the trigger event. In this example, video is recorded in the memory buffer for 16 seconds after the trigger event, but only for four seconds prior to the trigger event. In this way, the total high-resolution time period is the same as the example above, 20 seconds, but greatly favors preserving information after the trigger event, at the expense of information prior to the trigger event.
In a final example, there are six distinct time periods from which the timeline is comprised. Two 9 second periods occur symmetrically about an event trigger. In these time periods video may be captured a rate of four frames per second. Two additional periods each of eight seconds may be used record/overwrite data at a frame rate of two frames per second. In two additional 8 second periods are provided to store data at a frame rate of one frame per second. The reader will note that in the timeline of this example, two of the 8 second periods are of different sizes with respect to memory capacity, i.e. greater number of bins, than the other two 8 second periods. This is consistent with the higher frame rate used in two of the eight second periods.
It is now very easy to appreciate the great latitude one has when managing a memory buffer of limited capacity to expand a timeline. One further appreciates that where memory buffers deploy FIFO or ‘round-robin’ strategies for overwrite operations, very important data may be lost. FIFO and ‘round-robin’ strategies discriminate against the oldest data in a memory buffer. In cases where the oldest data is not the least valuable, FIFO and round-robin strategies are inferior.
With attention directed to FIG. 9, one will appreciate more clearly fundamental elements of apparatus of these inventions. These systems are always comprised of a video camera 91 operable for collecting optical energy and converting into an electrical signal which represents the image of a scene. The camera produces an electrical signal, suitable for processing by common electronic means such as digital semiconductor memories and processors. In addition, these systems include a trigger mechanism 92. In preferred versions, a trigger mechanism is the device arranged to provide an electrical signal to indicate that a particular video series should be transferred to permanent memory for long-term storage. A trigger may be an accelerometer operable for detecting abrupt changes in speed, for example that which may accompany a traffic accident. A trigger may be activated by other events such as heavy braking or swerving maneuvers. Triggers may be activated by means other than accelerometers. For example, a user panic button can be used to activate a trigger event. When the user comes to his own conclusion that a video series should be saved, he can hit a panic button to activate one type of trigger. It is not of particular concern for purposes of these inventions what precisely causes a trigger to be activated, but rather the response in memory handling once a trigger event has occurred. An overwrite manager 93 is a control module which interfaces with the trigger and a video camera, and also a buffer memory 94. An overwrite manager includes means where a timeline definition may be set and further means for executing overwrite operations in agreement with the stored timeline definitions. Further an overwrite manager may additionally integrate with a flush module 95. When a trigger event occurs, the overwrite manager continues to overwrite data to the buffer memory in accordance with the timeline definition by way of an overwrite pointer which is associated with a cell subject to an impending overwrite action. The overwrite manager sends a signal 96 to the flush module 95 to cause the flush module to copy the buffer memory and transfer the video data set with the prescribed expanded timeline to a high-capacity long-term storage 97. It is the overwrite manager which controls the algorithms and necessary processing facility which decides how to write to the buffer memory in order that select data is saved and redundant data is purged in accordance with a particular expanded timeline definition.
Preferred methods of these inventions are more easily understood in view of the block diagrams of FIGS. 10 and 11 which illustrate the primary steps of these methods. In particular, FIG. 10 suggests the most general methods which contains two steps including: a step 101 whereby frame data is received from a video camera, and second a step 102 whereby that newly received data is written over old data stored in the memory buffer in agreement with an expanded timeline definition.
FIG. 11 illustrates these methods in greater detail. Frame data is received 111 from a video camera and a first step. The buffer memory data write step 113 includes three distinct processes including: a step 114 where the frame is written to of been marked open. It is important part of this invention that data be written in the buffer memory in an organized fashion. So as not to disturb particular data frames, which are necessary to fill the prescribed expanded timeline definition. Therefore, a bin is marked ‘open’ when it no longer contains frame data necessary for the expanded timeline definition. In a second sub-step 115, a determination is made with regard to which memory been contains frame data, which is no longer needed in agreement with the timeline definition. This determination must be taken up on each cycle. For every new frame which enters the buffer memory another frame becomes no longer necessary at the same instant. Finally, in the third step, the bin which contained the no longer needed data is marked ‘open’. In the following cycle 112, the next incoming frame is written to the appropriate bin. It is useful to set a buffer memory pointer in agreement with this determination to effect a bin being marked ‘open’.
One will now fully appreciate how advanced memory management schemes may be deployed to expand a recorded timeline in memory buffers having limited capacity. Although the present inventions have been described in considerable detail with clear and concise language and with reference to certain preferred versions thereof including best modes anticipated by the inventors, other versions are possible. Therefore, the spirit and scope of the invention should not be limited by the description of the preferred versions contained therein, but rather by the claims appended hereto.

Claims

1) Vehicle event recorders comprising.

a camera; a managed loop buffer memory; and an overwrite mechanism,

said camera is operable for converting optical images to electronic signals suitable for recording in said managed loop buffer memory,

said managed loop buffer memory is a random access, re-writ able, electronic storage medium of finite size suitable for storing data, and in particular data arranged as discrete video frame data,

said overwrite mechanism electronically coupled to said managed loop buffer memory and arranged to manage data overwrite operations whereby a first video frame older than a second video frame is saved in the memory while the second frame is overwritten.

2) Vehicle event recorders of claim 1, said overwrite mechanism being arranged to overwrite data associated with a frame not the oldest frame in memory.

3) Vehicle event recorders of claim 1, said overwrite mechanism being arranged to overwrite data in an interleaved fashion whereby an effective frame rate is preserved by performing overwrite operations on alternate frames or frame sets.

4) Vehicle event recorders of claim 1, said overwrite mechanism further is comprised of an overwrite pointer arranged to point to a cell to be overwritten in a future overwrite action.

5) Vehicle event recorders of claim 1, said overwrite mechanism further comprising a timeline definition which defines an expanded timeline and virtual frame rates and periods for those frame rates.

6) Vehicle event recorders of claim 5, said timeline definition includes one having three distinct periods and an event moment at the timeline midpoint whereby the frame rates associated with the periods at the timeline extremities is lower than the frame rate associated with the period including the event moment.

7) Vehicle event recorders of claim 5, said timeline definition includes one having five distinct periods and an event moment at the timeline midpoint whereby the frame rates associated with the periods at the timeline extremities is lowest, and the frame rate associated with the period including the event moment is the highest.

8) Vehicle event recorders of claim 5, said timeline definition includes an asymmetry whereby the frame rate is different for the periods after an event moment than the periods before an event moment.

9) Vehicle event recorders of claim 1, further comprising a second memory, a high capacity, long-term memory coupled to the managed loop buffer memory.

10) Vehicle event recorders of claim 9, further comprising an event trigger arranged to cause a transfer of data from said managed loop buffer memory and said high capacity long-term memory in an expanded timeline format.

11) Vehicle event recorders of claim 9, said transfer of data step resets the managed loop memory into an initial state and a new timeline is initiated.

12) Vehicle event recorders of claim 9, data in long term memory is not overwritten in successive data transfer steps as it includes capacity sufficient for a plurality of events.

13) Methods of dilating a timeline of a video series in a vehicle event recorder video managed loop buffer memory comprising the steps:

receiving electronic signal data associated with video frames from a video camera; and

overwriting previously written data with newly received data, whereby previously written data is comprised of that belonging to a frame which is not the oldest frame in memory.

14) Methods of dilating a timeline of claim 13, said overwriting previously written data step includes overwriting data in an interleaved fashion whereby successive frames are skipped with reference to their capture times to effect an effective reduced frame rate.

15) Methods of dilating a timeline of claim 14, further comprising the step of moving an overwrite pointer to a cell not a member of a timeline definition.

16) Methods of dilating a timeline of claim 15, further comprising applying an overwrite operation to the cell associated with the pointer and further moving the pointer to another cell not part of the timeline definition.

17) In a vehicle event recorder system, a managed loop memory comprising:

a coupling to a video camera output whereby images from the camera are received at said managed loop memory;

a plurality of memory cells each being associated with a single video frame;

an overwrite pointer associated with precisely one memory cell; and

a timeline definition, whereby said timeline definition directs the overwrite pointer to be associated with a cell which no longer remains a member of the timeline definition thus subjecting that cell to an impending overwrite action.

18) Managed loop memory of claim 17, said timeline definition comprising two 24 second periods of 1 frame per second and one 48 second period of 4 frames per second, either of said 12 second periods falling before and after the 48 second period.