DE102011107603A1 - Computer-implemented method for generation, transmission, storage, management and processing of image features on display device, involves setting server system to arrange user-added graphical elements corresponding to original image - Google Patents

Abstract

An image (123) is created on the screen (106) or a portion of the screen of the client device, along with the incorporation of an already created image. A user interface (105) is provided between the user and device. Bundling of the message (118) consisting of the image and optionally further additives is carried out. The message is transmitted to the server system (112) which arranges the user-added graphical elements in such a way that the arrangement corresponds to the original image provided by the client device. An independent claim is included for a device for generation, transmission, storage, management and processing of image features on display device of client device which communicates with server system.

Description

Field of the invention:

The invention provides a novel method and apparatus for dealing with the need for detailed user-feedback to errors in displays of computer-aided devices and dealing with these feedback.

Underlying problem:

Computer-aided office and home appliances are routinely implemented to run programs and display data on an image output device to the user. The output of content is increasingly graphical (as opposed to 'in the form of text output'); The ability to influence this screen output in detail by the user in detail, for example by means of user-specific program configuration, is also steadily increasing. Because of this and because of the large number of possible combinations of hardware and software, it frequently happens that the user sees on his own picture output device a picture which differs from that desired by the programmer or operator of the program. This variance can have a myriad of causes, from data entry errors to calculation or program execution errors, to problems in the operating system or hardware of the user's device.

If a user notices this problem occurring with him, it is so far difficult for him to submit the present on his screen with all his comments to the responsible body, especially if the user has no expertise of other programs and the graphical output (image information ) constitutes an important part of the notification. A message in text form is technically easy to create on the computer, but the description of the graphical conditions is very complex for the user and for the operator or programmer, however, often incomprehensible. The user is also regularly the responsible contact person unknown and a shipment of possibly confidential information by mail per se very uncertain.

The remote sitting programmer or operator, it is impossible without a direct online connection to the user to view the image output of the user device in exactly the form as the user sees them, because the local device configuration and operating history rarely or never exactly represents the compilation, the occurred to the user when he had a problem with the image on his image output device.

In short, therefore, there is the problem for the user of the feedback of a false or unsatisfactory display on the screen of his computer, for the operator and programmer, the problem is that given feedback without graphic share important issues can not identify directly, are often incomprehensible and these problems can not be adjusted.

Summary of the invention:

The invention presents a method which solves the problem in several steps. Initially, an image of the user's screen image is produced, which precisely depicts the screen view visible to the user. This image will be transmitted to the operator during the procedure. The user can comment on the image before submitting it and highlight the problematic content in his or her eyes graphically. The original picture is not changed. The operator can take from her the important information for him, because of which he can narrow down problems. User information in the message and other meta-information are used to control the flow of messages on the operator side.

The inventive step consists in the combination of the individual steps "image of the screen of the user's device", "user's comments" and "graphic elements for illustration", each of which relates to a specific region of the image and associated with this image, without them and the procedure of keeping these pieces of information separate during transmission and storage and then reassembling them at the operator's premises via a suitable interface, thus making them accessible in an arrangement that clarifies the arrangement that the user has seen and determined. In addition, the invention utilizes the separate information and allows for rational processing, as the contents of the structure are used to control the data streams and display and arrangement and assignment of the messages in the operator's system and on the operator's workstations.

The process according to the invention can be reacted in a one-part or multi-part device.

Accordingly, the present invention relates to a method and a one-part or multi-part device, which allows a simple transmission of this image along with additional descriptions by the user and allows their view by the remote sitting programmer or operator and thus one or eliminates several problems of the prior art.

The present invention makes it possible to make the given to the user representation of the image display device to the programmer or operator, and in the course of the message creation to give the user the ability to add their own comments and comments, without the user know other programs or their use must be familiar. As part of the process, the available information is retained and are already used to control the process of processing.

To implement this, the invention realizes a method in which, with a part of the device made available on the user's device, the image visible on the image display device of the user's computer is detected at the user's initiative. This captured image can be commented on by the user by means of a user interface displayed for this purpose and, if necessary, provided with further comment elements. Illustration and comments are kept separate and linked via a logical structure; Comments also kept separate and partially logically linked. The image and these additional comments can then be sent via a possibly secure connection to the server system, the contents of the message thus created also remain unchanged during the transmission and on the server system in its structure and in its various parts get saved. There they can then be viewed and edited by the operator / programmer / competent expert via an interface and a part of the device on the server system makes the components of the message accessible again in the form and arranges how the user arranged them and on his image display device has seen. In addition, structurally determined parts of the message can be used to control the processes of editing and the presentation and arrangement of content on the operator side.

Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and in part will become apparent to those familiar with the art.

The term "interface" is used in the definition of meaning: "Interface or the interface (English for 'interface') is the part of a system that serves communication" and may refer to both human-computer interfaces and computer-computer interfaces ,

The term "user interface" is used in the definition of meaning "interface between computer system and human", specifically with reference to the described user interface on the computer of the user.

Short introduction of the achieved advantages:

The invention significantly facilitates dealing with display problems of programs installed on computers or computer-aided devices, as well as content displayed by other programs, e.g. B. Web sites represented by browsers.

With the presented invention, a simple message display problems is possible. By means of the device, even a computer user (which is 90% of the computer users) can transmit messages with complex content, and the operator can view the message in all details and peculiarities without the user having further knowledge of other programs and without Access to the computer of the user must be taken or the user further information such as the recipient of the message must be known. In addition, through the processing of the transmitted information, automated processes can be initiated which influence the processing of the message and can support the operator.

With the invention, the creation of a message for the user is simplified to such an extent that significantly more messages are delivered than was expected. This is to be regarded as very positive in the sense of the task.

Brief description of further embodiments:

The invention describes the method which includes device parts which can be executed as stand-alone programs on the side of the client and on the side of the operator, and which contains a server for data storage.

However, the method can also be executed as a server-centric solution, whereby the part of the device which is necessary on the client side is only transmitted to the computer and executed there on the basis of the client's request. The part of the device used for the operator-side presentation of the contents accesses the data-storing server and can be executed on this server or on another computer.

The method described can also be used in other applications, for example for commenting and evaluation the comments of any kind on image display devices or community applications that are based on the annotation of graphic image output.

Brief description of the drawings:

1 : Overview: Above is the user-operated system,
an example is the representation of the image output ( 106 ) by means of a screen ( 104 ). Visible on the screen is the output of the user interface ( 105 ), which the user uses to create a message with his comments on the image already created and displayed to him ( 123 ) of his screen view ( 106 ) can make. By means of this user interface ( 105 ), the user can also initiate the sending of the message, for example by means of a "Send" button ( 107 ).
The message ( 118 ), which describes the individual contents ( 122 ) is connected via a connection to the server ( 112 ), exemplarily it is a multi-level connection.
The server ( 112 ) takes over the interface ( 113 ) the message ( 118 ) stores them, separated according to their structure in non-volatile memories ( 115 ). Via an interface ( 116 ), the operator-side part of the device ( 119 ) the contents of the message on the image display device of the local workstation (A) - shown here as an example also as a screen ( 117 ) - rearrange as the user saw them at the time he sent the message. Components of the message can be used to process ( 128 . 120 ), for example the automatic assignment of messages to specific processing stations (A, B).

• a) diese Abbildung (123) zu beschreiben und allgemeine Anmerkungen hinzuzufügen, exemplarisch mittels eines Eingabefeldes für allgemeine Meldungen (201),
• b) diese Abbildung (123) detailliert zu kommentieren mittels weiterer Kommentarelemente, exemplarisch dargestellt durch Kreis (203) und Pfeil (204). Dem Kommentarelemente können sichtbare grafischen Elemente (bspw. Pfeil) zugeordnet sein sowie jeweils ein Texteingabeelement (205), das optional Kommentare (206) aufnimmt. Die Lage der grafischen Elemente und der ihnen zugeordneten Kommentare werden ebenfalls dem Kommentarelement zugeordnet, dieses wiederum der Meldung. Die Abbildung (123) bleibt dabei unverändert bestehen.
• c) weitere Kontaktdaten (207) anzugeben, exemplarisch Emailadresse und z. B. Name sowie die Einverständniserklärung zur Kontaktaufnahme (nicht detailliert abgebildet); Das NutzerInterface (105) kann auch das Auslösen des Versandes der Meldung ermöglichen, exemplarisch mittels eines Buttons (107); hierfür sind auch andere Mechanismen denkbar, etwa Gesten, Sprache, Zeitablauf oder detektierbare Ereignisse.

2 : Interface with the user:
Shown is an exemplary implementation of the user interfaces used within the method for generating the message at the client ( 105 ), which shows the view to start the procedure ( 106 ) of the image display device of the user and displays as an image ( 123 ). This user interface allows the user to

• a) this figure ( 123 ) and to add general comments, exemplarily by means of an input field for general messages ( 201 )
• b) this figure ( 123 ) to comment in detail by means of further comment elements, exemplified by circle ( 203 ) and arrow ( 204 ). The comment elements can be assigned visible graphic elements (eg arrow) as well as one text input element each ( 205 ), the optional comments ( 206 ). The location of the graphical elements and their associated comments are also assigned to the comment element, this in turn the message. The illustration ( 123 ) remains unchanged.
• c) further contact details ( 207 ), exemplary email address and z. Name and consent to contact (not shown in detail); The user interface ( 105 ) can also enable the triggering of the dispatch of the message, by way of example by means of a button ( 107 ); Other mechanisms are also conceivable for this, such as gestures, language, timing or detectable events.

3 : Interface with the operator:
Shown here is an exemplary implementation of the interface used within the method at the workplace of the operator (A) ( 301 ), which contains the components of the notification ( 118 ) in a way and arranges ( 302 ) that the operator sees the message in the order in which the user uses its user interface ( 105 ) created and looked at itself. For this purpose, the figure which remained unchanged in the course of the procedure ( 123 ) and the associated graphic elements ( 203 . 204 ) and arranged in the manner of a composite image ( 302 ) and the comments on the graphic elements ( 306 , Content like 206 ) as well as the general comments ( 305 ),
exemplified by a listing ( 303 ) of the individual comments, whereby the connection between the comment and the graphic element given by the structure is made clear, for example by a highlighting by means of a stronger border ( 304 ).
Due to the defined structure, the data contents in the invention are already used to control the flow of information distribution and presentation, for example to assign the message from the outset by way of example to the processor (A) and display it in his management view (FIG. 307 ) on his screen ( 117 ). By controlling the data streams, each responsible engineer receives only the relevant part of the message in his administrative area ( 307 ).

4 : Representation of the message ( 118 ) summarized individual elements ( 122 ), exemplified by the elements 123 . 201 . 203 . 204 . 206 . 401 . 402 . 404 ,

5 : Presentation of the process as a flowchart.
The start of the part of the procedure which takes place on the client side is initiated by the user, for example by pressing the "error message button" (FIG. 208 ). The framed part "User / Client" is the part of the process that takes place on the client computer that is part of the process for editing and managing used in the sphere of influence of the operator.

Detailed description of the invention:

The invention provides a novel method and apparatus that facilitates dealing with the need for detailed user feedback on errors in computer-aided display screens and how to handle those responses.

Today computers are installed in many devices and computer devices as such are used in many places. Periodically, office and household computer-based devices are implemented to execute programs and display data on an image output device to the user to provide feedback on the results of calculations performed or the like.

The output of content was formerly mainly as a text output, now increasingly in a graphic manner, which is sometimes elaborately processed and leads to graphic elements often enriched screen displays. The possibilities of influencing this screen output in detail by the user, for example by means of user-specific program configuration, are becoming more widespread. Because of this and because of the large number of possible combinations of hardware and software, it frequently happens that the user sees on his own picture output device a picture which differs from that desired by the programmer or operator of the program. This variance can have a myriad of causes, from data entry errors to calculation or program execution errors, to problems in the operating system or the hardware of the user's device.

If a user notices this problem occurring with him, it is so far difficult for him to submit the present on his screen output and his instructions to the appropriate body, especially if the graphic output (image information) constitutes an important part of the message and the User is a "normally educated computer user", so little knowledge of other programs, for example, has to image editing. A message in text form is technically easy to create on the computer, but the description of the graphical conditions is very complex for the user and for the operator or programmer, however, often incomprehensible.

This approach can be even more difficult, especially in complex usage situations - for example, if the user of a program with screen display can largely set (configure) the characteristics of the representation such as location and color of screen elements, or if the incorrect presentation of the contents only after a long series of work steps. Then the user would have to describe his system environment as well as all his work steps in the error message in order to make the error reproducible (otherwise) - otherwise the error message is worthless for the operator.

The remote sitting programmer or operator, it is almost impossible without a direct online connection to the user to see the image output of the user device in exactly the form as it presents itself to the user. A subsequent readjustment of the device and program configuration at a remote location is usually not possible because the local device configuration and operating history rarely or never exactly reflects the compilation that occurred at the user when he had a problem with the image on his image output device.

A typical practice example is the presentation of a website on the browsers of various computers: very often the configuration on its own workstation is completely different from the presentation on another computer system due to a configuration setting or software differences that are unimportant to the user. This problem is experienced by clients and agency employees creating websites every day. But of course, actual programming errors or errors in merging, editing, or entering the data can be the source of the display problems.

Discussion of the Related Art:

As described, the problem of reporting problems occurs in areas where screen representations are graphics-heavy or can only be explained by a graphical representation. There have been various ways to deal with it.

The easiest way for the "normal educated" user is to give feedback by phone. One way to make written messages is for the normal educated user in the form of emails or input forms that describe the display problems. These descriptions must be quite detailed, as indicated above. The more complex the graphical output and the more different the possible outcomes, the more detailed the description must be.

The approach that has hitherto been available for problem reports with a graphical focus is the use of a screenshot program, which was either offered by the computer's operating system or installed separately. This is the manual Creation of a screenshot (screenshot) possible. However, this is only possible for the expert user, so it will be eliminated for most users - and thus in most cases.

This screenshot can then be used by the person skilled in the art, for example, in a document of a word processing program and described there in terms of content and sent via e-mail to the contact person responsible for these errors. Also in the mail further commenting and description of the problem is possible. However, the existing problem of the need for a comprehensive description is still not resolved - when sending via email is possibly torn even the very important problem of data security: Emails can be viewed during the transport of many places.

Some separate special screenshot program also allows editing the screenshot, with z. For example, areas of the screenshot can be labeled, so there texts can be written into it, which are taken over into the image data. The screenshot is permanently changed in such editing, data is lost - and only a specialist has a special extensive screenshot program installed on his computer and can handle it.

• a) zb es auf seinem Computer im Moment technisch nicht möglich ist, einen Screenshot zu erstellen und/oder
• b) er keine Zeit dafür hat, den erstellten Screenshot ausführlich zu beschreiben und kommentieren und die umfangreichen, zum Verständnis der Nutzungssituation notwendigen Angaben hinzuzufügen,
• c) er beispielsweise den Ansprechpartner nicht kennt, der für die Behebung genau dieses aufgetretenen Fehlers beim Anbieter der Daten/Betreiber der Website zuständig ist und sensible Daten nicht an die allgemeine Kontaktadresse geleitet werden sollen.

Often, however, it is too time-consuming for the expert to go these ways because

• a) eg it is technically not possible on his computer at the moment to take a screenshot and / or
• b) he does not have time to describe and comment in detail on the created screenshot and to add the extensive information needed to understand the usage situation,
• c) he does not know, for example, the contact person who is responsible for correcting exactly this error that occurred with the provider of the data / operator of the website and sensitive data should not be routed to the general contact address.

Unreported errors and display problems cause economic damage, insufficient reporting mechanisms as well.

In contrast, there is a real need of some users to report bugs and also positive development ideas that would bring the operator profit and technical optimization.

If data problems are now reported despite existing barriers, it should be noted that the transmission of sensitive data by email is not considered secure; E-mails are easy to see and manipulate during their transmission, causing sensitive or precious data to be lost in transit or, uncheckable, being seen by strangers. This can cause enormous economic damage.

In rare cases, so-called bug-tracking systems are used to process problem reports. These are message management systems, but are also designed only for the text messages described above. Although the incoming messages can be managed there, the existing problems with the presentation and creation of messages with graphic emphases and the usability of computer laypersons do not do justice to the existing technology.

The application of Philips known to the inventor (EU patent WO2010 / 070490 A1 ), which actually deals with this topic, describes a part of a locally installed on the user's computer software that allows the steps to report a problem in a workflow, but should only be used in medical-technical contexts. In this solution texts are rendered into a screenshot of the program interface, so the original screenshot is changed, partly overwritten. The permanently changed screenshot will then be sent along with other data by e-mail. Thus, the requirements presented in the problem can not be solved.

Registered procedures like WO 2010/030914 A3 automate the process of creating the screenshot and pasting this screenshot into another program. Again, the comment is written without reference to the image in a separate document or an email; in comments on the screenshot they are sent into it, so the screen image is changed permanently and sometimes several times, the original data is overwritten and lost. The method described there is only applicable in closed user groups, that is not suitable for 'publicly available' applications and also uses a larger number of programs to generate the message.

The only known to the inventor implementation of user feedback on the Internet, which goes beyond the offer of text boxes as problem reporting mechanism, is on the website of YouTube.de realized. Here, however, the screen display of the user is not taken as the basis of the message, but it is in the course of the Message Creation regenerates a representation that looks similar to the original screen image. However, this newly generated view deviates significantly from the original user view (for example, videos are not displayed at all) - and thus does not solve the existing application problem. Just the lack of transmission of the unchanged, visible to the user at this moment screen display as the basis (and reason) of the message is fundamental part of the problem.

In summary, therefore, there is the problem for the user of the feedback of false or unsatisfactory screen content, for the operator and its programmers, the problem is that the feedback given by the user feedback without graphic share important problem areas of the graphical representation of the screen display directly and these problems can not be adjusted. The average user will not be able to provide detailed and meaningful feedback on display issues and errors, especially in areas that are very graphic-heavy.

The invention presents a further development of the current state of the art in many respects in order to achieve the solution of the described problem.

Solution to the problem by the invention

The invention presents a method which solves the problem in several steps. The procedure is in the picture 5 illustrated pictorially in its sequence.

Initially, the production of an image ( 123 ) of the screen display ( 106 ) Of the screen ( 104 ) of the user who exactly the user visible screen view ( 106 ). This image will be transmitted to the operator during the procedure. The user can comment on the image before the transfer, so that the operator can better classify the usage situation; In addition, the user can give special hints to graphically emphasize the problematic content and content areas in his eyes and comment on them by means of graphical comment elements and texts assigned to them. An important feature of the procedure is that the original figure ( 123 ) is not changed, all information is thus completely preserved, so that the operator can take the important information for him, such as picture details, on the basis of which he can narrow down problems, which the user evt not understandable or accessible.

The inventive step consists of the interface-based ( 105 ) Creation of the message ( 118 ) as a combination of the items "Screen view of the user's device" ( 106 ) as picture ( 123 ), "User's Comments" ( 206 . 201 ) and "graphic pointers" ( 203 . 204 ), each referring to a specific region of the map ( 123 ) and this figure ( 123 ), without changing them, and by the procedure, this bundled individual information ( 122 ) in a message ( 118 ) on the transmission path and when storing separately and only at the operator via a suitable interface ( 301 ) and so in an arrangement ( 302 ), which clarifies the arrangement that the user has seen and met ( 105 ).

In addition, the separated information ( 122 . 121 . 114 ), as its structural assignment is known by the method and uses the contents of the structure to control the data flows and display and arrangement and assignment of the messages in the system of the operator and on the processing stations of the operator (A, B) ( 120 . 126 ).

The software developed for implementing the invention in a computer-implemented method has been designed using the client-server model. The system consists of at least two parts, a server and a client component, which usually run on different computers.

Accordingly, the present invention relates to a method and apparatus that allows the user to easily transmit this image along with a description by the user, as well as to view it by the remote sitting programmer or operator, and thus the problems of the prior art gets out of the way.

The present invention makes it possible to make the given to the user representation of the image display device to the programmer or operator, and in the course of the message creation to give the user the ability to add their own comments and comments, without the user have more programs, know or with whose use must be familiar. The available information is evaluated in the best possible way and also used to control the process of processing.

Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and in part will be understood by those of ordinary skill in the art.

Description of the invention:

The invention makes it easier to deal with display problems of programs installed on computers or computer-aided devices, as well as contents represented by other programs, e.g. B. Web sites represented by browsers.

In order to achieve this object, the invention includes a method in which, with a part of the device made available on the device of the user ( 125 ) the representation visible on the image display device of the user's computer ( 106 ) is detected at the user's initiative to obtain an image ( 123 ) of it. Detection means in this context that a copy of the picture actually displayed is created or used; this is done by addressing a program present in the computer's operating system, by running its own program in or through the device, or by accessing a screenshot already present at that time, or optionally another suitable method.

This captured image ( 123 ) can be used by the user by means of an interface subsequently displayed for this purpose ( 105 ) and with the necessary additional visible comment elements ( 201 . 203 . 204 . 206 ). This interface appears on the user's image display device and shows, on the one hand, the reduced display of the display ( 123 ), as well as input options for comments ( 201 ), and adding additional graphical comment elements ( 202 ). These elements consist of a representative whose position can be displayed graphically, eg by a cross ( 209 ), associated visible graphical elements such as circle ( 203 ) or arrow ( 204 ) as well as text fields ( 205 ), the comment texts ( 206 ) be able to record. In the text fields ( 205 ), the user can use the user interface ( 105 ) More comments ( 206 ) to those with graphic elements ( 203 . 204 ) Visibly marked areas of the image ( 123 ) enter.

The illustration ( 123 ) and the comment elements ( 201 . 203 . 204 . 205 . 206 ) are kept separate and linked via a logical structure. The logical structure is implemented in such a way that a comment element with a text ( 206 ) and an optional graphical element ( 203 . 204 ) of the message ( 108 ), the location of the graphical element, the representative ( 209 ) and optionally additional additional information such as a visibility status of the text of the structure of the message ( 108 ). The illustration ( 123 ) and these additional comments with their elements are part of the message ( 118 ), as in 4 shown.

The message is then sent via interfaces ( 124 ) and a connection to a server system ( 112 ), the contents ( 122 ) of the message thus created ( 118 ) remain unchanged during the transmission. The data exchange takes place via suitable interfaces ( 124 and 113 ), which is also the technically flawless transfer of content over the arbitrarily complicated transmission path ( 108 - 112 ) controlled. The interfaces are designed to check and ensure the correctness of the transmission.

The transmission can, but need not, be made via a transmission connection which is particularly secure against the use of eg SSL (Secure Sockets Layer) in order, for example, to protect company secrets. This possibility offers the encryption of the data and the use of, for example, the SSL protocol, which in each case without user intervention via the interfaces ( 124 . 113 ) of the client-side and the server-side device part.

The message is always standardized, whereby the number of elements can vary. For example, as in 4 shown, the entire message consists of the figure ( 123 ), the data for specifying the graphic element or graphic elements (eg circle 203 and arrow 204 ), the text comments on the graphic elements ( 206 . 403 ), the positional arrangement of the graphical elements ( 401 . 402 ) and additional information ( 404 ) entered by the user (eg, his name) and that the client-side device part has inserted to clarify the client's system environment, such as the operating system, browser version numbers, and plug-ins.

The data is stored on the server ( 112 ) in non-volatile memories ( 115 ), according to their content type in database ( 114 ) and / or file storage ( 121 ) filed.

The individual comments contained in a message, together with their graphic elements, their location information, texts, etc., are stored separately in a structurally secure manner. The data can be encrypted and saved against falsification, so that a legally secure documentation system for the received messages is created in this way.

The stored data are connected to each other via one or more links, such links can be, for example, categories such as belonging to a specific Message, to a specific "notification button" or to a website or to be edited by a specific user.

The message is displayed on the server system ( 112 ) according to their structure and in their various parts ( 114 . 121 ) saved ( 115 ). For example, the image of the screen display ( 123 ) stored separately from textual information ( 201 . 206 . 403 ) and reference points of the comment elements ( 401 . 402 ).

In the area of the operator, the contents can then be accessed by the operator / programmer / competent expert via an interface ( 301 ) on his workstation (A, B) via an image output device ( 117 ) can be viewed and edited by a part of the device ( 119 ) via an interface ( 116 ) on the stored parts ( 122 ) and make the message accessible to the operator, together with its components, in the form that the user uses in their user interface ( 105 ) and on its image display device ( 104 ) had seen.

The device uses ( 119 ) the information stored in the structure during the creation of the message, e.g. B. location information ( 401 . 402 ) to the image ( 123 ) associated graphical elements ( 203 . 204 ) at the appropriate place. The separately managed text content of the comments ( 201 . 206 . 403 ) can, for example, be output as a clear list ( 303 ). By virtue of the inventively separate storage, these components can be passed through the device ( 119 ) so that the user (in this case the operator, or his programmers or editors) can each derive the most benefit from the arrangement.

The operator-side parts of the device ( 119 . 127 ) may, for example, be implemented as running on the server and / or a workstation computer software programs or parts of a software program. The devices use the hardware of a computer in order to process the information of the messages in specially adapted calculation steps, to sort them, and to prepare the data parts for the workstation-specific output to the processors, to assemble them, etc.

Structurally determined parts of the message can be used to control the processes of editing and the presentation and arrangement of content on the operator side ( 120 . 126 ).

Due to the defined structure, the data contents in the invention are already used at the time of reception and storage in order to determine the sequence ( 126 ) control the information distribution and presentation. For example, information contained in the message about the origin of the message is used to treat the message from the outset in a suitable manner determined by the operator, assign it to the processor (A) by way of example and access it via the operator-side device ( 119 ) in its management view ( 307 ) on his screen ( 117 ). If the operator were responsible for part of the message in the specialist area of the editor A and for other parts in another field, each responsible editor (A, B) would only relevant parts of the message in his management view ( 307 ) are listed.

This arrangement can be configured differently on different workstations (A, B), depending on the requirement (eg content editor, programmer, supervisor).

These processes ( 120 . 126 ) can also by adjusting the possibly multi-part device ( 127 . 119 ) are influenced on the part of the operator, for. For example, for notifications from certain sources, one or more recipients responsible for these notifications may be designated, and they may be assigned a part of the device ( 119 ).

By means of the various rights assignments, an access-protected editorial system has been set up for the interests of the operator, which serves the processing and also the administration of the notifications.

Users of this editorial system may have various rights to display content and / or links to the data; accordingly, the contents of a message displayed by the system may differ individually depending on the user.

From the separately available data (image, texts, features of the comment elements, etc.), in the operating part of the device ( 119 ) creates different display types by using information in the message and in the memory ( 115 ) stored data by the arithmetic units ( 127 . 119 ) and the stored algorithms via the interface ( 116 ) and these are evaluated and sorted. This arrangement of the contents according to the arrangement of the received messages is possible by the structure of the data described in the invention. For example, in the overall view of the message ( 302 ) the visible graphical components of the comment elements such as circle ( 203 ) and arrow ( 204 ) are arranged at the places specified by the user and texts are again assigned to the given features of the image. As a result, the information in a meaningful way again merged and depict the sentiment sent by the reporter.

In addition, the information can be evaluated and output separately, for example in textual lists ( 303 ) or text fields ( 305 ). The content-logical link (message → note → drawing element with position and eg text information) provides an interactive representation of the drawing elements ( 204 , Emphasis: 310 ) linked text elements ( 306 , Emphasis 304 ) possible. An interactive link can also consist of the graphic comment elements to the text list arranged in the figure to make the orientation and assignment easily comprehensible. This is done, for example, as a colored marking of the text element and of the linked drawing element when the mouse is touched or marked. Does the engineer touch the text of the listing with the mouse ( 306 ), this line is marked ( 304 ) and also the assigned symbol arrow ( 204 ) ( 310 ).

The invention presents a related system, which includes the creation of the messages, the transmission and the operator-side handling of the messages and their processing and storage.

This system allows different views and uses of the data transmitted during the message. The structure and procedures of the method are designed to meet this requirement.

The procedure is in the picture 5 illustrated pictorially in its sequence.

Achieved benefits:

With the presented invention, a simple message display problems is also possible by Computerlaien. The message can convey complex content and make the operator aware of all the details and peculiarities without the user having further knowledge of other programs and without having to access the user's computer.

With the invention, the creation of the message is simplified to such an extent that significantly more messages are delivered, which is an unexpected effect, which is to be regarded as very positive, both in terms of task, as more messages are created and the initially described maladministration is remedied, as well as in the sense of users of computer systems that can now work with more enjoyment and send with joy and ease error messages and instructions, as well as for the operator of the offer, who now more feedback regarding his offer and thereby develop innovations faster.

Further embodiments

The invention describes a method and a device. This method can be implemented as a multi-part software system in client-server architecture that includes multiple pieces of equipment that can be run as stand-alone programs on the client side and also on the operator side, and that includes a data storage server.

However, the method can also be executed as a server-centered solution, wherein the part of the device required on the client side is only transmitted to the computer on the basis of the request of the client and executed there in accordance with the circumstances of the client computer.

The part of the device used for the operator-side presentation of the contents may comprise the data-storing server or else only the access to this server, and may be executed on this server or on another computer, in which case the hardware components (CPU, main memory) of this computer system on which it is carried out with. According to the invention, the program to be executed on the client computer uses the hardware of the client computer, namely the CPU and the main memory, and carries out data movements there.

The method described can also be used in other applications, for example for commenting on and evaluating the comments of any kind on image display devices or in community applications that are based on the annotation of graphic image output.

Exemplary practical implementation:

The process notified here can be implemented in practice in the following way:
Use case is a website on the Internet, the complex database content presented graphically processed. Here, as described, a variety of errors occur. The provider and operator of the website is referred to below as the operator. For the editorial staff of various content areas and for programming, the operator has different contact persons who are not known to the outside.

In the example database, complex searches may be made by the user whose results may be influenced by the interim results of previously performed queries. The possible concatenation of intermediate results can lead to a large number of choices, which can also contain errors. In addition, the presentation of the results (eg formatting) on the screen of the user can be faulty. In addition, the entered and underlying data may also be incorrect.

The invention also enables computer laypersons to deal with these problems. The computerlaie that uses the offer is referred to below as the "user".

In order to implement the method, the part of the device running on the device of the user (client) is used as software ( 125 ) implemented on the client computer ( 101 ) and can run there if necessary.

This call can be exemplified by an easy to install "error message" button on every website ( 208 ). This is visible to the user of the offer (for example, the website). The user starts this process if he noticed a faulty entry or a display error. He presses the "error message" button ( 208 ) and then, from the user's point of view, the message creation interface ( 105 ). Technically speaking, starting the procedure makes the client-side part of the device ( 125 ) and started a program on the user's computer, which in this case may be loaded by the server. This part of the device ( 125 ) completes the first steps of the procedure: he creates an image ( 123 ) of the currently displayed screen ( 106 ), and opens the presentation of the user interface ( 105 ) on the screen of the user. On this interface is the picture ( 123 ) of the problematic screen image ( 106 ) visible, noticeable. The user now has intuitive possibilities to extensively document, graphically comment on the errors, problems or suggestions identified by him on the basis of the screen display visible to him and in a short time send them directly to the operator of the website or provider of the data , In our example, the user sees two errors, which he marks with different graphical elements: once a superfluous line, which disturbs the screen layout, and once, for example, a wrongly entered record. These problems affect two areas: first, the graphic implementation, and then the editorial content; The operator is responsible for two different contact persons. However, the recipient of the message (A, B) does not have to determine the user, this is determined by the operator by setting in the operator side part. The procedure ensures transmission to the appropriate recipients (A, B), who can contact the user as needed.

Since, in our example, this is confidential data, the operator has chosen to send via a secure connection. The communication of the interfaces ( 124 . 113 ) of the client-side and the server-side device part thus takes place securely over the SSL (Secure Sockets Layer) protocol. The user does not need to worry about the need and implementation of encryption.

The user can carry out his message quickly and comfortably. He does not have to spend expensive working hours, for example, to research the forgotten knowledge about the use of various programs for screenshots, drawing programs, etc., or to search for the responsible addressee.

For the user appears as the first action after 'pressing' the message button ( 208 ) the user interface ( 105 ), which animates him to enter his message. All necessary input elements are immediately available for him. So he can give a general description ( 201 ), but also special areas of the figure ( 123 ), for example, with an arrow ( 204 ) and highlight the malady as a comment ( 206 ) in an arrow ( 204 ) assigned input field ( 205 ) enter. When he is done with the input, he sends with a simple push of a button on the "send" button ( 107 ) the message ( 118 ); it safely reaches the server of the operator.

The operator of the website may be in the operator-side device ( 127 ) integrated part of the device ( 128 ) upon receipt of a message ( 118 ), for example by sending an e-mail with a reference to the received message.

The individual comments contained in a message together with their graphical elements, their location information, texts etc are structurally separated and stored securely. The data can be encrypted and saved against falsification, so that a legally secure documentation system for the received messages is created in this way.

The stored data are as described via one or more links connected to each other, such links can also be, for example, categories such as belonging to a specific message to a specific "Report button" or to a website or editing by a specific user.

The message will be processed according to the built-in information and by a part suitable for accessing the data ( 119 ) of the operator-side device by data-controlling operations ( 120 ) in accordance with the available information to the processor assigned to the error-triggering website at his workstation (A, B). The operator-side parts of the device ( 119 . 127 ) may, for example, be implemented as running on the server and / or a workstation computer software programs or parts of a software program.

The engineer looks at his workstation (A) on his interface ( 301 ) the receipt of the notification in its administrative area ( 307 ) and can see the message.

The editor responsible for editing the graphical and program problems can now edit the part of the message that deals with the graphical problem.

This engineer can use an interface ( 301 ) the individual components ( 122 ) of the message ( 118 ). The editor can thereby see the original error message as 'in the original' and understand the contents and conditions well. He can access them individually, but also the content elements ( 122 ) by the operator-side device part ( 119 ) on the screen ( 117 ) of his workplace (A, B), as the user 104 ) has seen. Thus he easily recognizes which areas of the screen view are meant by the comment. Does the engineer touch the text of the listing with the mouse ( 306 ), this line is marked ( 304 ) and also the assigned symbol arrow ( 204 ) ( 310 ).

In order to view details of that position, he can enlarge the image accordingly and take from the unadulterated and pixel-exact figure the information necessary for the solution of the representation problem.

By means of the system parameters of the web browser transmitted with the program, the processor has an overview of technical details which even an adept user could only ascertain in a time-consuming search, for example the version number of the browser used, the plugins etc.

The invention makes it easy to create messages from the user's point of view as well as operator access to and management of the messages, and the precious information can be directly and securely accessed by the right person, documented and regulated.

The entire process of preparation until the response to the message is covered by the procedure.

Reports about problems, even if the offer is more computer-aided or generally non-academic customer segments, can be reliably and usefully provided by the operator by enriching automatically inserted information and the simple operation of the NuterInterfaces to be used for technical optimization.

Description of the drawings

Fig. 1: Overall view:

At the top you can see the user-operated system, exemplarily as a computer ( 101 ) with keyboard ( 102 ), Mouse ( 103 ) and image output device (here as a screen) ( 104 ). The computer shows the client-side part of the device ( 125 ), which has an interface ( 126 ) communicates with the user. On the screen ( 104 ) the output of the user interface ( 105 ) via which the user receives a message ( 118 ) with his comments on the image ( 123 ) of his screen view ( 106 ) can make. The comments are shown here by way of example as circle and arrow symbol. By means of this user interface ( 105 ), the user can also initiate the sending of the message, for example by means of a button ( 107 ). From the content ( 122 ) is removed from the part of the device ( 125 ) on the user's computer the message ( 118 ) assembled and shipped.

The message is sent via an interface ( 124 ) via a connection to the server ( 112 By way of example, it is a multi-level connection that can be used as a wired connection ( 108 ) to an access provider ( 109 ), from there via the Internet ( 110 ) to the access node of the operator ( 111 ) and from there to its server ( 112 ) runs.

The server takes over the interface ( 113 ) receives the message, stores it according to its structure in non-volatile memories ( 115 ), for example in databases ( 114 ) and file storage ( 121 ). The contents of the message components are stored by the server-side part of the device ( 127 ) for controlling ( 128 ) of the processes used. Via an interface ( 116 ), the part of the device ( 119 ) on the part of the operator, the contents of the message on the local image display device, here also exemplified as a screen ( 117 ), so again arrange how the user saw them at the time he sent the message. Components of the message are used to process ( 128 ) on the server and on the part of the device at the operator ( 119 ) ( 120 ), for example the automatic assignment of messages to certain processing stations (A, B).

Fig. 2: Interface at the user:

• a) diese Abbildung (123) zu beschreiben und allgemeine Anmerkungen hinzuzufügen, exemplarisch mittels eines Eingabefeldes für allgemeine Meldungen (201),
• b) diese Abbildung (123) detailliert zu kommentieren mittels weiterer Kommentarelemente. Der Nutzer kann Kommentarelemente durch grafische Markierungen anbringen, die er auf der Abbildung (123) platziert. Er trift dafür eine Wahl des gewünschten grafischen Zeichnugselementes, exemplarisch dargestellt durch Wahlfelder zum Erstellen von Kreisen, Rechtecken oder Pfeilen (202). Hier können auch Freiformlinien möglich sein. Das Ergebnis des Platzierens ist exemplarisch verdeutlicht mit einem Kreis (203) sowie einem Pfeil (204) sichtbare grafischen Elemente (bspw. Pfeil) zugeordnet sein sowie jeweils ein Texteingabeelement (205), das optional Kommentare (206) aufnimmt. Die räumliche Lage der Kommentarelemente auf der Abbildung wird optional durch ein Symbol, hier ein Kreuz (209), gezeigt, wobei optional über dieses Kreuz auch das dem Kommentarelement zugeordnete Textfelder (205) sichtbar und unsichtbar geschaltet wird. Die Lage der grafischen Elemente und der zugeordneten Kommentare werden ebenfalls dem Kommentarelement zugeordnet, dieses wiederum der Meldung. Die Kommentarelemente werden mitsamt ihrer Lageposition wiederum der Abbildung (123) zugeordnet, die dabei unverändert bestehen bleibt;
• c) weitere Kontaktdaten (207) anzugeben, exemplarisch Emailadresse sowie (nicht abgebildet) Name und die Einverständniserklärung zur Kontaktaufnahme; wobei alle diese in a–c beschriebenen Bestandteile, sofern sie vorhanden sind, in ihrer Gesamtheit (122) in die Meldung (118) verpackt werden. Die Meldung kann auch weitere, auch nicht-sichtbare Bestandteile aufweisen.

Shown is an exemplary implementation of a user interface used within the method ( 105 ), which shows the view visible at the start of the procedure ( 106 ) of the image display device of the user ( 104 ) in the form of a figure ( 123 ) holds and represents. The user interface allows the user to

• a) this figure ( 123 ) and to add general comments, exemplarily by means of an input field for general messages ( 201 )
• b) this figure ( 123 ) to comment in detail by means of further comment elements. The user can attach comment elements by means of graphic markings, which are shown on the picture ( 123 ). He makes a choice of the desired graphic drawing element, exemplified by selection fields for creating circles, rectangles or arrows ( 202 ). Here also freeform lines can be possible. The result of placing is exemplified by a circle ( 203 ) and an arrow ( 204 ) are assigned visible graphical elements (eg arrow) and in each case a text input element ( 205 ), the optional comments ( 206 ). The spatial position of the comment elements on the image is optionally represented by a symbol, here a cross ( 209 ), whereby optionally via this cross also the text fields ( 205 ) is switched visible and invisible. The location of the graphical elements and the associated comments are also assigned to the comment element, this in turn the message. The comment elements, together with their position, turn the figure ( 123 ), which remains unchanged;
• c) further contact details ( 207 ), exemplary email address and (not shown) name and consent to contact; all of the constituents described in a-c, if present, in their entirety ( 122 ) in the message ( 118 ). The message can also have other, even non-visible components.

The user interface ( 105 ) can also enable the sending of the message, by way of example by means of a 'send' button ( 107 ); Other mechanisms are also conceivable for this, such as gestures, language, timing or detectable events.

Fig. 3: Interface at the operator:

Shown is an exemplary implementation of an interface used within the method on the workstation (A) of the operator ( 301 ) containing the constituents ( 122 ) of the message ( 118 ). The operator sees the figure together with the assigned visible comment elements as an arrangement ( 302 ) on the image display device ( 117 ) at his workplace (A) in the form in which the user uses his user interfaces (A) 105 ) and at the image display device ( 104 ) during the message creation at the user interface ( 105 ) has seen.

For this purpose, the ingredients ( 122 ) of the message ( 118 ), including the figure which remained unchanged in the course of the procedure ( 123 ) and the associated graphic elements ( 203 . 204 ) in a composite image ( 302 ). The composite view ( 302 ) of the message shows the graphical elements ( 203 . 204 ) without the text elements assigned to them ( 306 . 403 ), which are arranged elsewhere, by way of example in a list ( 303 ). The comments on the graphical elements ( 306 , Content like 206 ) as well as the general comments ( 305 , Content like 201 ) are made accessible again in a suitable form, this is done here by way of example by a listing ( 303 ) of the individual comments below the composite figure ( 302 ), whereby the connection between the comment and the graphic element given by the structure is made clear, for example by a suitable highlighting on the graphic element (FIG. 310 ) and the text entry ( 304 ).

Due to the defined structure, the part of the device ( 119 ) that contain information within the calculation ( 120 ) to link the graphical elements ( 203 . 204 ) in the composite image ( 302 ) to connect to them and to allow a mutual emphasis in marking each one element.

The use of the information in the operator side part of the device ( 119 ) in calculation processes ( 120 ) also allows the message to be handled in a suitable manner, assigned to the person responsible for the special subject area (A, B) by way of example, and to be displayed in the administration view ( 307 ). Here are features and contents of the message shown to allow identification of the message, exemplifying the date of creation.

In the area of the operator's interface ( 301 ) the currently viewed message is distinguished by features and contents of the message, exemplarily with the creation date ( 308 ) as well as further administrative information, exemplarily the sequential number of the management system of the operator. Further informatioenen from the message, exemplarily the email address ( 309 ) are arranged to support the operator's editing process.

Fig. 4: message components

Representation of the message ( 118 ) summarized individual elements ( 122 ), exemplified by the elements 123 . 201 . 203 . 204 . 206 . 401 . 402 . 404 ,

The message includes at least all the elements that the user uses via the user interface ( 105 ) has put together. In particular, the message always includes the figure ( 123 ). The illustration ( 123 ) is not changed in the process by the compilation of the message. Other elements such. B. graphical pointers such as the arrow, the user can use his user interface ( 105 ) add to the message.

The position of the graphic element (eg arrow (eg 204 )) by means of the location information ( 401 ) relative to the figure ( 123 ), textual information ( 206 ) are also the graphical comment element ( 204 ) and thus with the visible arrow ( 204 ) connected.

Additional Information ( 404 ) may be added to the message, such as creation time information, user information, user information such as the name or email address, or system information to the client.

Fig. 5: representation of the method as a flowchart.

The start of the part of the procedure which takes place on the client side is initiated by the user, for example by pressing the "error message button" (FIG. 208 ).

The bordered part "user / client" is the part of the process that takes place on the client computer and from the device part active there ( 125 ) and the user is served. Server and client part can be accessed via the interfaces ( 113 and 124 ) communicate and exchange data. Through the transport, the message with its information about the server comes on the side of the operator. The message is processed and stored on the server.

The operator can edit, view and manage the messages. In this case, a part of the device ( 119 ) via an interface ( 116 ) the components of the message from the memories ( 115 . 114 and 121 ) and the items in an interface ( 301 ) so that the figure ( 123 ) with the user-added graphical elements in a composite view ( 302 ) is displayed as the user himself in his user interface ( 105 ) had in mind. In order to keep the representation clear, the comments associated with the graphical commentary elements can be displayed separately, as shown in FIG 3 is shown as an example.

pictures:

1 : Overview

2 : Interface of the user (Detailed view of 105 )

3 : Interface of the operator (Detailed view of 117 )

4 : Components of the message (Detailed presentation of 122 )

5 : Flowchart of the procedure (user actions, client, server, operator actions)

LIST OF REFERENCE NUMBERS

101

Computer on the client side

102

Input device for text input

103

Input device for pointing

104

Image output device

105

user interface

106

Image output from the client computer

107

Submit button

108

Data Connection

109

Access provider to the data network

110

data network

111

Access provider to the data network

112

Server computer on the operator side

113

Data interface on the operator side

114

Database storage

115

Memory for permanent data storage

116

Interface to access the unit on the server

117

Image display device at the workplace (A) on the operator side

118

Message in assembled form

119

Part of the device on the operator side

120

Processes of information distribution and storage

121

Memory for data in alphanumeric form

122

Components of the message ( 118 )

123

Image of client-side image output ( 106 )

124

Interface for data communication with the server ( 112 )

125

Client-side part of the device

126

Processes of information distribution and access

127

Part of the device on the server

128

Processes of information distribution and access

A, B

Processor (symbolic) and engineer jobs on the operator side

201

text commentary

202

Controls for inserting graphic comment elements

203

Graphical comment element circle

204

Graphical comment element arrow

205

Text input field for comments that follow the arrow ( 204 ) should be assigned

206

Comment with the arrow ( 204 ) indicated content of the figure ( 123 ) concerning

207

Input field for further information, here: Email adress

208

Error message button on the exemplary website

209

Graphical representative of the placement of the graphical comment element

301

Operator interface for viewing and processing of messages

302

composite illustration, by arrangement of elements 203 and 204 on the picture 123 created from the message

303

List of contents of the comment elements ( 203 . 204 ) associated comments

304

Highlighting an entry (here by thicker border)

305

Content of the text of the message

306

Content of the comment element 204 (Arrow) associated comment

307

Listing of additional messages assigned to the workstation (administration area)

308

further arranged information from the message (here creation date of the message)

309

further arranged information from the message (here email address from 207 )

310

Highlighting a drawing element (here by thicker border)

401

Location data for specifying the position of the arrow in the comment element 204 relative to the picture 123

402

Location data indicating the location of the circle in the comment element 203 relative to the picture 123

403

Text content of the comment element 203 associated comment

404

Further information, eg system parameters, address information, name, address.

405

Logically linked components of the graphical commentary element arrow

406

Logically linked components of the graphical comment element Circle

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• WO 2010/070490 A1 [0043]
• WO 2010/030914 A3 [0044]

Cited non-patent literature

• YouTube.de [0045]

Claims (13)

A computer-implemented method for generating, transmitting, storing, managing and processing messages on image features on the user's display device at least one device on a client communicating with a server system, the method comprising a plurality of steps as follows Marked are: The device makes it possible to create at least one image of the screen view of the user's device and / or a part of this screen view and / or to incorporate an already created image of the screen view, The device provides at least one user interface for providing at least one interface between the device and the user, The device allows the compilation of at least one message consisting of at least the figure and optionally further additives, The device makes it possible to send at least the message to the server system, - The server system stores at least the message separately in their components permanently and structured, so that the composition and arrangement of the individual components is possible again in a manner that corresponds to the original given on the client image and arrangement. The method of claim 1, further determined by the server system having at least one device for, via an interface, displaying the map and the additions sent by the device of the client to match the screen view of the user at the time of launch of the device of the client may be wholly or partially, optionally with all, some or without the added by the user additions. The method of claim 1 or 2, further comprising the step The device allows additional comment elements to be assigned to the image used in the message without altering the image. The method of claim 1, 2 or 3, further comprising the step - The device allows the addition of information that can be assigned to the message. A method according to claim 4, characterized in that marked on the image by means of one or more visible graphic element conditions and the position of the graphical element relative to at least one feature of the position of the image is defined. A method according to claim 5, characterized in that the visible graphic elements do not change the data structure of the image. A one or more-part device that computer-implemented a method of reporting presentation types of the image output of the client computer, and characterized by: - At least a part of the device that does as client part on the client computer, the creation of an image of the screen view of the device of the user and / or a part of this screen view or inserts an existing in the message At least a part of the device which, as a client part on the client computer, provides the user with an interface with which a message is created and / or edited in which further information can be assigned to the image, - At least a part of the device on the server, which receives the message according to the predetermined structure in their parts - At least a part of the device on the server, which is adapted to store the information and permanently stores the message according to the predetermined structure in their parts - At least one interface between the parts of the device on the server and the client. An apparatus according to claim 7, further comprising: At least part of the device for arranging at least part of the components of the message in the manner corresponding to the arrangement of the components on the user interface providing the message at the time the message is generated. An apparatus according to claim 7 or 8, further comprising: - At least a part of the device for editing the components of the message. A device which consists of at least one client part which communicates with at least one server system, comprising - at least one memory for storing information which is supplied as a message, - at least one interface for providing at least one interface between server System and the operator; At least one interface section for providing at least one interface between the server system and the device; The device comprises at least a part for displaying information in the arrangement as well as they were delivered by the client part as a message to the server. The device of claim 8, further comprising: - At least a part of the device for editing content and / or the arrangement of the parts of the message. A method or apparatus according to any one of the preceding claims, further allowing, by evaluating the contents of the message known in its structure, to determine management, presentation and arrangement options of the contents of the message. A method or apparatus according to any one of the preceding claims, additionally comprising the management of the message.

Images