CN101091148A - Systems and methods for teaching a person to interact with a computer program having a graphical user interface - Google Patents

Abstract

Systems and methods are disclosed for teaching a person how to perform a task within a computer program with at least one graphical user interface object. A script is formulated that contains step-by-step instructions indicating to a person how to perform a task within a computer program through interacting with at least one graphical user interface object of the computer program. The script is formulated by reference to an entity map, which contains programming instructions for accessing at least one graphical user interface object of the program to perform a step of the task. The script is executed by displaying a partial graphical overlay on top of an actual, non-simulated graphical user interface of the program. Each graphical user overlay instructs a user how to provide pertinent user input to graphical user interface objects of the computer program. The partial graphical overlay is positioned adjacent to the pertinent graphical user interface object of the program regardless of the position of that object without any reprogramming of the underlying computer program.

Description

Guiding operation person and system and method with computer program interaction of graphic user interface

Technical field

The invention relates to a kind of guiding operation person and system and method, especially, the invention relates to the Software tool of the software application that moves in a kind of guiding operation person's operating equipment with computer program interaction of graphic user interface.

Background technology

Software application all contacts with millions of people's life phase every day.Software application is applied to many great commerce and plays a role in individual business activity.The ubiquity of software with and the complicacy that increases day by day make software be more suitable for operation, allow the user can learn and use their software fast.This demand is more and more by online and next satisfied based on the training and the indication of computer.

Be used to train the traditional online of user function software and be based on one of defective of the teaching material of computer: it only provides passive learning experience usually, promotes higher level study and can not make the user participate in enough interactions.High-end teaching software can comprise diagram, video, animation and sound wait create the specialty and attractive presenting, but this content had both lacked desired user interaction usually, also lacked the effect of practical operation.

This training content is normally given the user by browser or other external application passes, and the user need be between the application program (Live Application) in training form and the activity forward or switching backward, make this applications active (application program that the user accepts to train) for the user may become tedium, obscure easily, expend time in and efficient not high.

More the training teaching material of advanced form has adopted analogue technique, and this analogue technique is simulated a software application and come execution work to allow the user in a kind of training environment, just as the user uses this product in reality.Though this technology has overcome the many defectives that lack the interactive teaching study course, its exploitation and carrying cost are very high.In case initial simulation is to make up according to a certain specific interface, so whenever the interface of this application software is modified (being revised by user or software developer) afterwards, perhaps at every turn after the software development of a redaction is come out, this study course must be upgraded equally.In addition, owing to the cost of each function of making this software application in simulated environment again and ability is quite expensive, therefore this simulation study course is restricted to usually and only has most basic function and program.The result causes the investment of new edition software to tail off, the company of only more remaining employee's lacks of skill, and cause exploration and experiment on most advanced and sophisticated marginal technology to reduce probably.

In addition, because the restriction of simulation, the user can not understand the repertoire of this software product.Because exploitation and keep simulation softward and had cost and degree of difficulty problem, make simulation teching Products Development person suffer obstacle equally aspect this restriction overcoming with wide variety of functions like this and effect.

Therefore, need provide a kind of mode to develop and keep and simultaneously in conjunction with the advantage of multimedia communication and the software teaching of interactive learning with greater efficiency and worthwhile cost.

Summary of the invention

The present invention be directed to that the state of prior art develops, fail still at existing educational aid particularly that the technical matters that solves fully and demand develop.Therefore exploitation of the present invention is for a kind of educational aid that can overcome many defectives in the prior art is provided.

Particularly, the invention provides a kind of system and method and be used for coming and the interaction of graphic user interface (GUI) object, how to operate a computer program with the indication user by partial simulation user's input.These system and methods can be implemented by many different modes.For example, in one embodiment, software may be distributed in the system and method for implementing above announcement on the medium (for example, be stored in a CD goes up or interior by the file of network download) of a computer-readable.

In one embodiment, software is used to make the developer can design a script, and how the indication that this script comprises step by step carries out a task to be shown to the user in a computer program.This design the Script section by with the graphical user interface object interaction of computer program to realize this function.The present invention can be widely used in various computer program, for example by the Microsoft Word program of MS (MicrosoftCorp.) exploitation etc.

The execution of this design script is by showing that a part coverage diagram layer position is above the graphic user interface of the computer program of an actual non-simulation.Each part coverage diagram layer can comprise how literal, figure, animation or video are imported to carry out a step of this task to the graphical user interface object of relevant computer program to show to the user.This part coverage diagram layer only covers the part of the user interface of this computer program, and the user can see that still the relational graph user interface object that needs input is to carry out a step of this task so like this.

Each part coverage diagram layer is arranged on the position near the graphical user interface object of relative program, no matter whether this lower floor's computer program has any process variations to influence the position of this object.In one embodiment, the position of the operating system of this computer program operation graphical user interface object that is required to judge that this is relevant.By this information, this part coverage diagram layer is set at the position near this relational graph user interface object.Owing to do not need to revise this computer program, the present invention can implement and be widely used in various computer program easily.Therefore, its program that does not need to advise the manufacturer of computer program to go to revise them adapts to the agreement and the standard of trouble.Both can understand the interaction of the present invention and graphical user interface object by the present invention.

In addition, because this part coverage diagram layer is to be shown on the computer program of actual and non-analog version, teaching provided by the present invention is higher than the efficient of existing teaching method.When undergoing training on the computer program of user in analog version, the user must do the difference of correspondingly adjusting with the software of understanding this simulation softward and its use, causes the reduction gradually with efficient obscured of teaching process potentially.System and method of the present invention is exactly this task of carrying out on the software that the user originally will be used to execute the task, so can improve the efficient of teaching effectively.

In one embodiment, this design script can be carried out several steps in this task for the user, and alternative is that this script also can only indicate the user to carry out a step of this task, and before the indication of demonstration, wait for user's correct input about the next step of this task.

This script designs by reflecting figure with reference to an entity, and this entity reflects figure and comprises that several programming instructions are used to visit at least one graphical user interface object of this program.This entity reflects figure can further comprise the title related with each programming instruction, to simplify the visit to each programming instruction.In one embodiment, individual other programming instruction can reflect figure (hard coded) from this entity and directly is embodied as this script.In another embodiment, entity reflects figure and can do dynamic or static reference by specific script.When creating an entity and reflect figure and carry out script with the computer program of a corresponding redaction, the dynamic reference that this entity reflects figure just can make this script reorganization.Therefore, when issuing, the computer program of redaction do not need to redesign again script.

In the script executing process, user's input is monitored at any time.On the adept degree of judging a specific user, the supervision that the user is imported may be very useful.If the input that is provided by a specific user often is incorrect, this information can be stored in the daily record need take the training of which kind of form to improve its skill level to the user with decision.In one embodiment, in the process of carrying out script, incorrect input will be forbidden accepting by this computer program.In another embodiment, can provide correct indication to the user who submits incorrect user's input to.This correct indication can comprise figure, literal, sound, video or animation.

In one embodiment, the present invention can use with learning management system (Learning ManagementSystem (LMS)) or Learning Content Management System (Learning Content ManagementSystem (LCMS)).Learning management system and Learning Content Management System all are used to digital studying (e-learning) industry with to user's display digit learning content and follow the trail of their use, execution and skilled state.In such system, content of the present invention comprises this system of permission and shows its content to the user, in the content of this computer program, provide which relevant indication with indication user (for example, the user will see that several tasks are performed in the application program that he is learning).

For example, a learning management system (LSM) may comprise that a class journey is to create several with a character program to transport sign.This learning management system will utilize this invention to transmit content, and the operation that character program of this content explicit user use oneself and rolling mouse, the input of transmission keyboard etc. need is transported sign to create this practically.The content of multimedia that this invention can show any video or animation at any time and carry out any sound or other needs is to pass on necessary information or indication.Any will be in available true feasible data in future by the data that script executing of the present invention produced.The Any user input can be stored in this learning management system or the Learning Content Management System, and can be used for keeper's analysis or tracking.

In another embodiment, the present invention comprises about user's expansion of learning management system or Learning Content Management System, computer program or computer program or the task assigned work flow process that defines voluntarily.These workflows will be guided the user by a given step that is used to execute the task.The present invention will only allow the object interaction of user and this workflow appointment.For example, implement software of the present invention and can carry out a voice files, press certain specific button in this computer program with the indication user.This script also can be used for demonstrating an arrow, the button that this arrow points will activate.This invention can allow the user to move the mouse of this computer then, and carries out correct user's input of this step except being used for, and indicates this system to ignore any click.

In another embodiment of the present invention, can test the ability that the user executes the task in a computer program.The present invention can track user input also/or confirm whether this user has correctly carried out these specific tasks.This affirmation result can be recorded and be stored in the database, and this database combines with learning management system or Learning Content Management System.For example, a learning management system can show that to the user a test is with the degree of understanding of evaluation user to showing material after a study course finishes.One of this test problem can be that the requirement user carries out a task.This learning management system will utilize one embodiment of the present of invention to come the interactive of track user and computer program and confirm whether this user completes successfully this operation.The present invention will exchange with learning management system with the result of these tests, and learning management system in regular turn this outcome record in this database.

In a further embodiment, the invention provides a kind of system, in this system, the task that operable automatically is carried out by the user usually.The example of this embodiment can be an application program of carrying out the quality guarantee test on computer program.Example hereto, but the present invention's Run Script is with a specific function of test computer program correct execution whether.Another example is that the task of automatically performing is for efficient that strengthens a certain particular task or auxiliary impotentia or has the individuality of obstacle to carry out particular task.

For the feature of being mentioned in the whole instructions, advantage and similar term and do not mean that the attainable all feature and advantage of the present invention must all embody in any one independent embodiment.More properly, terms such as these feature and advantage can be regarded as, and are included at least one embodiment of the present invention in conjunction with embodiment described concrete feature, advantage or a characteristic.That is to say, whole instructions for feature and advantage and similarly term argumentation can but do not limit and refer to same embodiment.

In addition, feature of the present invention, advantage and characteristic can be combined with various suitable manner in one or more embodiment.Being familiar with this operator can know, the present invention also can implement during one or more concrete feature in lacking a specific embodiment.In other examples, other feature and advantage of not mentioning in all embodiment of the present invention also can be added among some embodiment.

The feature and advantage that the features and advantages of the present invention and other are not enumerated will become clearer in ensuing instructions and claim, perhaps can be by understanding in the specific embodiments of the invention of hereinafter mentioning.

Description of drawings

For being more readily understood advantage of the present invention, the embodiment during aforementioned the present invention who simply mentions incites somebody to action in conjunction with the accompanying drawings describes in detail.

Be understandable that these accompanying drawings only are preferred embodiments of the present invention, and are not limiting the scope of the invention, the present invention will be in conjunction with the accompanying drawings in addition extra concrete and detailed explanation.

Fig. 1 is the block scheme of an embodiment of enforceable system of the present invention and environment thereof.

Fig. 2 is an embodiment of a kind of graphic user interface that can use of the present invention.

Fig. 3 A is the block scheme of a system according to an embodiment of the invention, and wherein the operation of this system is to be relevant to the present invention.

Fig. 3 B uses a notion entity to reflect the embodiment of figure according to one embodiment of present invention.

Fig. 4 is a kind of process flow diagram of an embodiment that the script that comprises instruction comes the method for analog subscriber input that is used to handle.

Fig. 5 is the process flow diagram of an embodiment of the processing entities method of reflecting figure.

Fig. 6 is the process flow diagram of an embodiment of creating the method for an entity.

Fig. 7 is the process flow diagram of an embodiment of creating the method for dissimilar entities.

Fig. 8 is the process flow diagram of an embodiment that obtains the method for suitable system resource for an entity.

Fig. 9 is the process flow diagram of an embodiment of carrying out the process of script command.

Figure 10 carries out script with the process flow diagram of rolling mouse to an embodiment of the method for a concrete graphical user interface object of this computer program.

Figure 11 is the process flow diagram that filters or stop an embodiment of method of user's input in carrying out scenario process.

Figure 12 A-12E is a screenshot capture of carrying out the step of the task in the example computer program with one embodiment of the present of invention.

Figure 13 is the screenshot capture of an embodiment who catches step of a script that is used to carry out the task in the computer program.

Figure 14 is the screenshot capture by an embodiment who provides video output to help to teach the method that the user carries out the task in the computer program.

Figure 15 is the flow diagram of an embodiment of religion user a kind of method of carrying out the task in the computer system.

Figure 16 is the logic block-diagram of an embodiment of religion user a kind of system of carrying out the task in the computer system.

Embodiment

Various embodiment of the present invention will be illustrated in conjunction with Fig. 1 to Figure 16 at this.Common description of the present invention and be shown in element among the figure can be implemented by various structures widely.Like this, the next detailed description of the embodiment of system and method for the present invention as shown in FIG., is not to be used to limit scope of the presently claimed invention, and only is the example of preferred embodiment of the present invention.

Many functional units of being mentioned in this instructions are denoted as module or element, implement to highlight their independent.For example, module can be by comprising client VLSI circuit or gate array, turn-off certainly semiconductor for example the hardware circuit of logic chip, transistor or other discrete elements implement.Module or element also can be by programmable hardware device field programmable gate arrays (fieldprogrammable gate arrays) for example, programmable logic array, programmable logic device or similarly device implement.

Module or element also can be by the software implementations of carrying out by various programs.The module of sign or the element of executable code can comprise the one or more physics or the logical block of the computer instruction that can consist of object, program or a function.But, an enforceability that indicates module or element does not need being positioned at together of entity, can comprise the different kenels instruction that is stored in diverse location,, can form this module and realize the different instruction of certain function of this module as when combining in logic.

In fact, the module of an executable code or element can be an independent instruction or many instructions, even are divided into several sections different code snippets, are positioned at different programs, and visit several memory body equipment.Similarly, discernible execution data can specifically be implemented with the data structure that various forms is organized in various adequate types in the illustrational here module.These execution data can be collected as an independent data acquisition or can be dispersed to the diverse location of different storage devices, and can, at least in part, as just the electronic signal existence of a system or network.Below the step of described each method or each method can implement by variety of way equally, for example by hardware, software or both are in conjunction with enforcement.

" embodiment " who mentions in the whole instructions or similarly term refer to concrete feature, structure or the characteristic of describing that combines with this embodiment and be contained at least one embodiment of the present invention.That is to say that " in one embodiment " that is occurred and similar term are passable, but do not limit, and all refer to same embodiment.

In addition, described feature, structure or characteristic can combinations mutually in an embodiment or a plurality of embodiment in any suitable manner.In ensuing explanation, many concrete details will be provided, for example program, software module, element, user's selection, network interaction, request of data, data structure or the like, coming provides more thorough understanding to embodiments of the invention.Be familiar with this operator and should be known in that the present invention can perhaps implement under the situation with additive method, element, material or the like lacking wherein one or more details.In other examples, some structure of knowing, material or operation do not show or describe in detail, in order to avoid cover flesh and blood of the present invention.

Fig. 1 to Figure 16 has shown the system that carries out these Software tool specific tasks and the embodiment of flow process.Though the present invention describes in conjunction with exemplary embodiment shown in this Fig, being familiar with this operator and can knowing that the present invention can implement by multiple different application program or embodiment, is not that it is implemented specifically to be defined in this specific embodiment as described herein.

What Fig. 1 showed is according to one embodiment of present invention, the system 100 that the present invention can implement and the block scheme of environment thereof.This system 100 comprises a central processor unit (CPU) 110 and a bus 111 that can be connected other elements of this processor 110 and this system 100.Display converter 116 and display device 118, for example CRT monitor or LCD screen are also used typically.This system 100 adopts memory bodys 112 of some forms usually, for example stores memory body (for example RAM) immediately, and storer 114 (for example hard disk drive) keeps, storage and deal with data.

This system 100 comprises a user interface converter 124 typically, and it can make user input device (for example keyboard 126, mouse 127 or other input equipments 128) provide the user to import to this system 100.This user input device 126,127,128 makes that the user can give an order, deal with data or contact with other of system 100.This various input equipments 126,127,128 and be used for by the user can be by mouse, keyboard, trace ball, touch-screen to the device (being commonly referred to as " input equipment ") of system input, speech recognition equipment or other allow the user to form to the equipment or the system of these system's 100 inputs.

This system 100 also can pass through communication facilities 130, and for example wireless or cable network converter or modulator-demodular unit are communicated by letter with other equipment or resource, with accesses network or online resource 132.This system 100 may be connected by network in LAN (Local Area Network) or wide area network with other computers or equipment.The technology that this this system 100 is connected with internet or other communication systems by communication facilities 130 also is that to be familiar with this art person known.This system 100 can be PC, workstation, kneetop computer or a notebook computer, personal digital assistant, portable phone, the perhaps equipment of other various permission user inputs.In fact, any have utilize the equipment and the system of the graphic user interface that the user imports all can use system and method for the present invention.

Many modern computers system 100 all adopts graphic user interface, and these systems 100 are equipped with a display device 118 and one or more input equipment 126,127,128 to handle data and the information that is stored in this system 100 to allow the user usually.In such system, the data and the program mode with figure on display device 118 that store and run in this equipment 100 show.The user by operate various input equipments 126,127,128 and change and with the object interaction of this graphic presentation.These are converted into signal by the action that the user carries out in these systems 100, and are received by corresponding user interfaces converter 124.The signal that this user interface converter 124 is received it converts in order can be by the corresponding system information of computer system 100 processing.The signal of this reception makes this computer system 100 go to revise data and the program that it is preserved, and allows the user to come interaction and revise this abstract data by the mode of vision and sense of touch like this.

The enforcement that can combine with the computer system 100 of various multi-form use user interfaces of disclosed system and method.For example, this computer system 100 can be PDA(Personal Digital Assistant), notebook computer, desktop computer, server, panel computer or a wireless telephone etc.

That Fig. 2 shows is an embodiment of graphic user interface 200, and it can be shown on the display device 118.The graphic user interface 200 of this demonstration comprises a desktop 201, and it is a zone that comprises the visual Drawing Object of various users.This desktop 201 comprises a form 202 with pattern exhibiting software, program or function.This desktop 201 and form 202 optionally comprise several graphical user interface objects (" GUI objects "), for example button 203, menu 204, menu item 205, character area 206, bulleted list 207 and image 208 are familiar with this operator and are all known, graphical user interface object can also have other various forms except shown in Figure 2 these.

Several objects that are shown in graphic user interface 200 only are the Visual Display that is stored or created software, data, function and the program etc. that form by computer system 100.The link that this computer system 100 keeps between graphical user interface object and real data and their relative program.Way is normally preserved the tabulation of all animated graphs user interface object at system or application layer like this, and is linked by handle, resource identification number or other parameters graphical user interface object and real data that this is relevant.For being familiar with this operator, obviously, system and method described here can with current available various graph user interface systems, for example by the Microsoft's Window operating system of Microsoft exploitation etc., enforcement combines.

Next will more fully disclose an embodiment of a kind of mechanism provided by the invention, the developer can create and transmit the interaction content how the indication user operates a computer program by this mechanism.This interaction content can and be coordinated contents such as literal, sound, video, animation, image by tissue and provide indication to the user.This interaction content can with by with the user learning actual be not interactive generation of graphical user interface object of the computer program of simulation, perhaps user's mutual-action behavior that can cause generation combines." interaction content " and " mutual-action behavior " can comprise the instruction that the simulation user interaction in computer program that is sent to computer program or indication user provide such interaction.Use by interaction content and mutual-action behavior, the mouse pointer that this content developer can drive the user moves on screen, make a choice from menu and tabulation, input keyboard inputs to a certain specific zone, and simulates any behavior that can be used the input equipment execution by the user.One embodiment of the present of invention indications user provides a step of the concrete input task to carry out this computer program, and before the indication that shows about the next step of this task, waits for user's correct input.The present invention also can carry out the desired system command of this content developer, program and function and assist the indication that is just taking place.For example, this content developer specific computer program that the present invention is called be not written into.For simplicity, the content developer can select not by this computer program of same routine call, calls this application program but simplify order of processing.

Fig. 3 A be one with the system 301 of one embodiment of the present of invention binding operation or the block scheme of computer equipment.Shown system 301 comprises that a graphic user interface 310 and an input queue 320 are used to accept the input of user by various input equipments 330.In one embodiment, the present invention includes an input monitor 343 and monitor the input of receiving from input equipment 330.

The present invention utilizes script 344, and analog subscriber is input in the graphic user interface 310 of computer program 346 of system 301 operation.To the user's input that should simulate, this computer program 346 is carried out a step of the task in the computer program 346.By carry out task and/or the interaction content that uses analog input on computer program 346, the present invention has clearly showed the task of how to carry out this computer program 346 to the user.

In alternate embodiments, itself does not execute the task the present invention, but by using analog subscriber input and/or interaction content to show how to execute the task to the user.In such embodiments, this input monitor 343 judges whether the right user input is received.When the right user input had been received, the user was instructed to carry out the next step of this task.In one embodiment, if incorrect input is received, then provide correct indication to the user.

How one group of one or more entity (be referred to as entity and reflect Figure 34 5) of this script 344 visits when carrying out simultaneously (in this script design, operation or) provides the analog subscriber input to specific graphic user interface to judge.This entity reflects Figure 34 5, shown in Fig. 3 A and Fig. 3 B, comprises that several programming instructions 347 are used to visit at least one graphical user interface object of specific computer program 346.

Shown in Fig. 3 B, each project of reflecting Figure 34 5 at entity optionally comprises a title 349, and with reference to several programming instructions 347, these several programming instructions 347 are used to visit the graphical user interface object of demonstration with convenient.For example, shown in Fig. 3 B,, reflect among Figure 34 5 and can discern by the program at title " App.Buttons.OK " and this button place or the parameter of form and the explanation or the order ID of this button at entity for " determining " button of certain menu or program.

This entity reflects programming instructions 347 in Figure 34 5 to be needed in detail enough, allows system 301 that parameter or the handle that the position of this object and the method that provides by system 301 and function obtain this object can be set and realizes.In one embodiment, the present invention relies on and this graphical user interface object to be abbreviated to one to can be used for locating the system's handle of this object or the digital coding of other parameters.By obtaining this handle or parameter, this system 301 can require to give for change the relevant information of this graphical user interface object, when for example working as script 344 execution, and the position that this graphical user interface object is current.

The use of title 349 is selectable, and and failed call in entity reflects each embodiment of Figure 34 5, all use.Yet, in one embodiment, use title 349 can simplify the step that one group of script command of use visits specific graphical user interface object.In other words, this entity can pass through with reference to title 349, rather than visits by programming instruction 347.

In one embodiment, to reflect that Figure 34 5 is stored be a special file to this entity.Yet, being familiar with this operator can know, this entity reflects Figure 34 5 and needs not to be an independent file, can be compiled in the computer program of implementing 346, can provide by a data stream of online or Internet resources, perhaps even can dynamically generate (on the fly) and only be stored in the random access memory by the enforcement of graphical user's list object in the check system.In fact, any known program and method all can be used for implementing a suitable entity and reflect Figure 34 5.

Fig. 4 carries out the present invention to comprise the step of the script 344 of instruction with the process flow diagram 400 of an embodiment of the method for analog subscriber input.As shown in the figure, step 401 is beginnings, and in one embodiment, step 405 is the resolved one-tenth indication of script, order and explains.Order is in the process of implementation by the function of script executing, and indication is to prepare to carry out script 344 (for example entity reflects the dynamic link of Figure 34 5 to script 344).In one embodiment, this script 344 comprises a title.This title is one and shows which entity reflects the indication that Figure 34 5 should be used in combination with this script 344.Note is the readable literal of people, can ignore in script 344 is handled or need not.

After resolving the step 405 of script 344, it is processed and be written into memory body (this content will be explained in detail in conjunction with Fig. 5-7) that step 410 is that this entity reflects Figure 34 5, and following step 415 is that this script command is performed.Followed by the execution of script command, step 420 is that this script 344 finishes.

Carrying out script 345 can realize by variety of way.For example, in one embodiment, this script 344 can read line by line.A certain particular row can show that visit which entity reflects Figure 34 5.Each row of this script is carried out 415 in proper order until the ending that arrives script.

In another embodiment, to be these scripts 344 reflect Figure 34 5 and create necessary code and carry out order listed in the script 344 and handle by resolving title, processing entities step 405.This script is compiled executable format (for example .NET compilation) then and is saved or execution in step 415.

Being familiar with this operator can know, these script 344 parsings of step 405, the processing of step 410 and the execution of step 415 can realize by the numerous different modes in the scope of the present invention.

Fig. 5 provides among Fig. 4 the process flow diagram that is described in detail of an embodiment that the only simple processing entities of describing of the present invention is reflected the method for Figure 34 5.At first, step 501 is that this processing procedure begins, and step 505 is one and comprises the file that entity reflects Figure 34 5 and be opened then, and this next line of step 510 (being first row in first embodiment) is read.

Step 515 judges whether to arrive the ending of this script then, and when arriving the script ending, step 520 is that this processing finishes.

If the ending of no show this document, step 525 are to judge whether associated row/order is an entity statement.If an entity statement, step 530 will be created an entity.This processing will repeat each row of this script 344.After entity set-up, it can be used to visit or obtain information about the relational graph user interface object by script 344.

In one embodiment, the establishment of entity can take when script executing the form of the class declaration that shows in the future some point to create.Selectable, this processing can comprise the demonstration and the storage of this related object in a kind of mode that is fit to use in the future when the command execution, perhaps also can simply comprise the establishment of an object will using immediately.In another embodiment, the demonstration of object is by implementing a standard interface and this object stores being realized in a collection.Each object is created to set the position of the related graphical user interface object of this related entities together with the information that it needs.The collection of this entity in one embodiment, can come cross-section according to the title of entity.

In another embodiment, script 344 is converted into and can carries out set.In such embodiments, it is self defined class statements that entity reflects each entity 345 among the figure, and increase by is about to show the code of the object of corresponding name when this set is carried out.

Explanation once more is familiar with this operator and can be known and of the present inventionly can realize by other technologies outside the said method and mode in conjunction with the described handling procedure of Fig. 5.

Fig. 6 is the process flow diagram 600 of detailed description of an embodiment of the present invention of simply mentioning among Fig. 5 method of creating 530 1 entities.

As mentioned above, the entity entity that reflects 5 li of Figure 34 comprises that instruction 347 is used for calling party graphical interfaces object.This instruction can link with the title 349 that can be used for visiting the relational graph user interface object.Step 601 is when processing entities is reflected Figure 34 5, and this each entity that reflects Figure 34 5 is converted into an available formats.In one embodiment, step 605 is the fundamental type (for example entity of form, button, tool bar or menu-style) of this entity of decision and the parameter that is used to locate this entity (for example title, parent object, classification or the like).In one embodiment, step 610 is suitable classification that this information is used to select the implementation criteria interface, and should classification and specifying information to this entity illustration.Then, step 620 is these entity set-up EOP (end of program).

Explanation once more is familiar with this operator and is will be appreciated that above only is a example in numerous modes of the interior establishment entity of the scope of the invention.

Fig. 7 shows provides the process flow diagram in greater detail 700 that the present invention that Fig. 6 simply mentions creates an embodiment of an entity.After the entity set-up program began step 701, step 710 was to judge whether the entity that will create is a form of window.If, step 715 form entity of establishment.

If the entity that will create is not a form entity, step 720 judges that whether the entity that will create is a button form.If step 725 is created a button.

If the entity that will create is neither form neither the button form, step 730 judges whether it is the entity of menu-style, if step 735 is created a menu entity.

For embodiment in, entity set-up EOP (end of program) 740 after the above-mentioned steps.Be familiar with this operator and should be known in that this method can be implemented by numerous different modes.For example, in other embodiments,, can create and be different from various types of entities shown in Figure 7 if suitable.

Fig. 8 is the process flow diagram for an embodiment of the method for relevant system resource handle of graphical interface of user object acquisition or parameter.The beginning of this program of step 801 is along with step 805 finds related entities in entity reflects Figure 34 5.Then, can pass through, for example location and access entity are reflected the associated title of dependent instruction of the entity in Figure 34 5 (shown in Fig. 3 B), the location of carrying out entity.

Again obtain the instruction 347 of the graphical user interface object that will locate then.Provide rules and procedures to carry out according to system 301 then, this system 301 (as shown in Figure 3A) of step 815 inquiry (for example operating system of using by this system 301 of inquiry) is to obtain resource handle or other parameters of this graphical user interface object that system 301 preserved.Step 820 finishes this program and the parameter of being obtained is returned to the program of asking it then, and this parameter can be used for carrying out the related task of the graphical user interface object relevant with this like this.

Fig. 9 is according to one embodiment of present invention, carries out the process flow diagram 900 of the method for the independent order in the script 344.Step 901 program begins, at first step 905 recognition command.Usually the order of being discerned is that in the script 344 next ordered in proper order.

Each order can comprise program object, the function of this object that will visit and carry out and the correlation parameter of attribute and this function/attribute that a program is carried out.Accordingly, the first step of step 910 is to obtain this program object.Next, step 915 is obtained the function/attribute of this program.Then, if be present in parameter in this order, step 920 also will be acquired.

Step 925 judges whether entity of this parameter sensing then.If step 930 is that this entity is used to obtain a handle, resource and indicates number, perhaps other parameters of other these related entities.After these steps is this order of execution in step 940.Certainly, if do not point to entity, this order can not obtain that handle, resource indicate number or other parameters of this related entities and directly being performed.

Step 945 then, the execution of individual command finish also to carry out another order according to the above-mentioned program of mentioning, if perhaps arrive the ending of this script 344, then finish script.Explanation is familiar with this operator and can be known once more, and this script 344 can be carried out by other the whole bag of tricks except that aforementioned described method in the scope of the invention.

Figure 10 is according to embodiments of the invention, mouse pointer is carried out the process flow diagram of the method for specific instructions (for example rolling mouse) on graphic user interface.What this process flow diagram shown in Figure 10 showed is the more recapitulative concrete enforcement that instruction is carried out shown in Fig. 9.For Figure 10, the entity title (for example app.Buttons.OK) that a concrete graphical user interface object is accepted in the order of shown rolling mouse then rolling mouse to this object.

Step 1001 is that this program begins, along with step 1005 is that the order of this " rolling mouse (App.Buttons.OK) " must at first be identified.Then, step 1010 is that this object " mouse " is acquired.Next, the function/attribute that obtains this object " moves " and is acquired.Then, step 1020 is that this parameter " App.Buttons.OK " is acquired.

Step 1025 is that this parameter of decision is pointed to a concrete entity then.Because parameter is pointed to a concrete entity, step 1030 is to obtain the handle of this related entities, resource identification number or other parameters.

Step 1035 is that this entity handle that can be used to obtain this relational graph user interface object (for example " OK " button), resource indicate number or other parameters and decide the screen adjustment of this graphical user interface object by inquiry system 301 (as shown in Figure 3A).Then, fill order moves to this entity concrete graphical user interface object pointed with this mouse pointer.Then, step 1045 is that this command execution finishes.

Above program is no matter active user's graphical interfaces to liking what state, all allows script behavior work.For example, point to the above-mentioned script of mentioning button with the rolling mouse pointer to this button, and no matter the position of this button in this graphic user interface.This is only to be a parameter of reflecting this button of Figure 34 5 by this entity because script 344 does not comprise the hardware identification code coordinate of this button.This content on the establishment interaction content, provides multiple dirigibility and feasibility like this, because can be described the concrete graphical user interface object fixed position of this graphic user interface frequently.This is very crucial, because if the user moves a graphical user interface object or different resolution of screen moves this computer program 346 (as shown in Figure 3A) down, a graphical user interface object is activated in the fixed position in the identification graphic user interface may be unreliable when selection or access graphics user interface object.

This entity reflects Figure 34 5 also can make script 344 correct execution on the computer program 346 of different editions.Use the present invention, an independent script 344 can all be effective for the computer program of two kinds of complete different editions.For example, the menu option of a visit printing function may be positioned at file pull down menu in the version 1 of word processing computer program 346, and in the version 2 of same computer program 346, this menu item may be positioned at the operation drop-down menu.Provide if this script 344 is the correct entities with respective version reflects figure, then rolling mouse is to this print menu graphical user interface object and to click it may all be effective to the computer program 346 of two kinds of versions with the script 344 that activates the corresponding command.When this script 344 is carried out in version 1, will use the entity of this version computer program 346 to reflect Figure 34 5 and under File menu, find the print menu option.When same script 344 is carried out in version 2, the entity that uses the computer program 346 of redaction reflected Figure 34 5 and under the operation menu to find this print menu graphical user interface object.

In one embodiment, the present invention also can force the input that hook stops, filter or other forms are come monitoring user on system input device 330.This can be used for preventing the task that the user interrupts carrying out or stops user's unallowed input under the parameter that system is sent in current script or the workflow just handled is set.The establishment of these hooks is by the inventor, for example uses input monitor (shown in Fig. 3 A) oneself to impose on lower floor of system input queue (also as shown in Figure 3A).This forces is according to being taken place by the function and the program that set for the system that finishes a such input hook.

These hooks can be installed and unload to shown embodiments of the invention as required, to stop or to allow all user's inputs, also is used to set up the input strobe utility that only allows some input.For example, the user who guides by the present invention in a workflow can be instructed to a concrete option of selection tool bar.The present invention can allow the user to move this mouse pointer everywhere on graphic user interface 200 (shown in Figure 2), but ignores except clicking all clicks the click on the tool bar option.As shown in Figure 3, because this input monitor 343 obtained user's input before system's input queue 320, the present invention can confirm whether allow this input under current environment.

Figure 11 shows is the process flow diagram 1100 of an embodiment of the process of filter user input.This program of step 1101 begins, and along with step 1105 is that an entity is registered as a filtration, activates a filtration then, and this filtration is according to the type customization of a graphical user interface object and desired user input.Step 1110 is to judge whether user's input is accepted then.When input is accepted, step 1120 is that this entity reflects Figure 34 5 and is used to obtain handle or the parameter that is registered as filtering object.Step 1125 is that this system 301 of inquiry (shown in Fig. 3 A) is with coordinate and the border of judging this graphical user interface object and coordinate and the border of therefore judging this acceptable user's input then.

Step 1130 judges whether this user who receives input drops in the parameter area of this entity setting then.If user's input is incorrect, then step 1135 is ignored this input and step 1110 monitoring user input once more.If user's input is correct, step 1140 is sent to input queue 320 (shown in Fig. 3 A), this EOP (end of program) of step 1150 then by the signal that this input produces.

In case the user clicks this relevant range or other associated user's inputs is provided, and then carries out the Next Command of this script 344.The filtrator that these bunchiness are arranged makes the present invention can guide the concrete steps that the user finishes the specific tasks in this computer program 346 (shown in Fig. 3 A) jointly in whole workflow.

Being familiar with this operator can know, can monitor and filter input by variety of way, comprises record and confirms user's input.One embodiment of the present of invention can be tested understanding and skill level with assesses user by the frequency of judging incorrect input and correct input.Alternatively, the input of going track user to be confirming that the user is carrying out the relevant function of task of being assigned with them, or on principle or standard in execution.

Figure 12 A to Figure 12 C is the screenshot capture of carrying out according to the embodiment of a script 344 of the present invention (shown in Fig. 3 A).In this embodiment, the computer program 346 that this user learnt (shown in Fig. 3 A) is the text software by Microsoft's exploitation.Should be understood that this illustrated embodiment only is an example of the present invention and be not limitation of the invention.The present invention can be used for creating script 344 with various computer programs with graphic user interface and corresponding operating system, and is used to create and carry out script 344 to carry out nearly all function of computer program 346.

See also Figure 12 A, it shows the first screenshot capture 1200a, and in this screenshot capture 1200a, this associated script 344 has started.This script 344 provides step by step indication to come guides user how to carry out task (being the framework that founds an organization in this embodiment) in the computer program 346.This script 344 makes that first's coverage diagram layer 1201 is shown in this computer program 346 actual be not simulation graphic user interface 1203 above.This first's coverage diagram layer 1201 only covers the part of the graphic user interface 1203 of this computer program 346, and the user still can see the remaining graphic user interface 1203 of this computer program 346 and according to the demonstration and 346 interactions of this computer program of this part coverage diagram layer 1201 like this.

Part coverage diagram layer 1201 shown in this Figure 12 A shows " click and insert menu " 1205 to the user.In one embodiment, clicking insertion menu 1205 can carry out or selectively be carried out by the user for the user.If this step is to be carried out by the user, before clicking insertion menu 1205, the user can not show the next step of this task.

As mentioned above, in one embodiment, this user's skill level is judged in the daily record of preserving user's input.For example, often be incorrect if the user who is provided imports, this shows the training that these user's needs are extra.These daily records can be stored in a storage facilities 114 or the memory body 112 (shown in Figure 1) to judge after a while whether this user's skill level and this user's skill level are improved, stagnate or fall back.In one embodiment, incorrect user imports the state that is under an embargo and conveys to this computer program 346 and can not change or influence this computer program 346.

This first's coverage diagram layer 1201 is set at the position near relevant graphical user interface object 1205 (inserting menu 1205), no matter have or not the position of this graphical user interface object of influence of change of any these computer program 346 source codes.As shown in the figure, this part coverage diagram layer 1201 is set at the position of close this insertion menu 1205 and points to this insertion menu 1205.In one embodiment, the present invention is reflected the position that Figure 34 5 judges the graphical user interface object 1205 of being inquired to the system queries of these computer program 346 operations to utilize entity.In case after the position of this graphical user interface object 1205 was judged, this part coverage diagram layer 1021 can be arranged on the position near this relational graph user object 1205.

See also shown in Figure 12 B, screenshot capture 1200b shows be shown in second step of task, as mentioned above, the first step of this task has been carried out for the user or by the user.Accordingly, can demonstrate the get off menu 1207 relevant with this insertion menu 1205.A second portion coverage diagram layer 1209 (corresponding to second step of this task) be shown and be arranged near drop-down menu " chart ... " the position of menu option 1211, this menu option 1211 activates by clicking insertion menu 1205.The part coverage diagram layer 1209 of this step shows " clicking this chart ... menu option " to the user.

See also shown in Figure 12 C, that screenshot capture 1200c shows is the embodiment that correct indication 1208 is shown.Example hereto, the demonstration of the correct indication 1208 shown in this Figure 12 C are owing to incorrect input occurred for second portion coverage diagram layer 1209 (shown in Figure 12 B).For example, if the user has clicked " picture " menu option 1213 (rather than chart menu option 1211), the correct indication 1208 shown in will occurring.Should shown in correct indication 1208 are figures of a mouse input equipment, its show left mouse button highlighted with show to the user correct selection " chart ... " the method of menu option 1211.Certainly, also can provide the correct indication of other types, for example the alarm song in the scope of the invention, animation indication, video indication or audio description.

See also shown in Figure 12 D, demonstration be shown in the screenshot capture 1200d of third step of task.A third part coverage diagram layer 1215 appears in the position at the graphical user interface object relevant with organizational structure 1217, selects this graphical user interface object 1217 to show to the user.

See also shown in Figure 12 E, demonstration be shown in the screenshot capture 1200e of the 4th step of task.The 4th part coverage diagram layer 1218 be arranged near with shown in the position of the relevant graphical user interface object of confirming button 1219, indicate the user to select this button 1219 to finish the task of the framework that in this computer program 346, founds an organization.

In one embodiment, when script 344 was carried out, this entity reflects Figure 34 5 (shown in Fig. 3 A) and/or script 344 (shown in Fig. 3 A) dynamically associates the literal 1220 that one or more graphical user interface objects will show with decision in part coverage diagram layer.The significant advantage of this program is that an independent script 344 can be with the employed language of this computer program (for example English, Spanish, French) display text 1220 in this part coverage diagram layer 1218.Correspondingly, script 344 does not need to adopt and writes the different language of language with this script and write literal 1220 with computer program 346 related uses again.

See also shown in Figure 13ly, show that the user of a script 344 exports the screenshot capture 1300 of the graphic user interface 1301 of journal.In one embodiment, when recording element of the present invention was activated, script 344 can design by the step of carrying out this script in the computer program 346.User's output is presented at the step of the script of creating in such recording process 344.More specifically, the user exports the step of the script 344 shown in the displayed map 12A-12E.As shown in the figure, the step of this task is not performed by script 344.On the contrary, this script 344 as shown in the figure, for example, " wait for that the user clicked the insertion menu " by script step 1303 before the next step of this task of processing, wait for user's correct input.

See also shown in Figure 14ly, it shows that the video be used in combination with the execution of script 344 exports 1401 screenshot capture 1400.This script 344 of the present invention is multi-functional, not only comprises part coverage diagram layer, and can comprise how video and audio frequency output use concrete computer program 346 (shown in Fig. 3 A) to help user learning.Multifunctionality of the present invention makes, and is very simple for the establishment of the interaction content of a specific user or customer group customization.

See also shown in Figure 15, its demonstration be to instruct the operator to carry out the process flow diagram 1500 of an embodiment of the method for the task in the computer program 346.This method can be implemented by several different modes.For example, this method can be distributed in for example software in CD or the DVD of computer readable media by use, or the concrete data-signal that internet usage or LAN (Local Area Network) are downloaded in the communications media of this software is implemented.The software of implementing this method also can hard coded form distribute the part of embedded system or be stored in the hard disk drive 114 for example with a computer system.

At first step 1501 is to create an entity to reflect Figure 34 5.As mentioned above, this entity reflects the special pattern user interface object that Figure 34 5 comprises several programming instruction 347 visit computer programs 346.This entity reflects figure can further comprise the title 349 related with each programming instruction 347, to simplify the visit to programming instruction 347.This entity reflects Figure 34 5 can be by analyzing the relevant graphical user interface object of specific computer program 346, manually or automatically or use the mode of manual and automatic combination to create.This entity reflects Figure 34 5 can all be used in the design of script 344 and implementation.

As above mentioned, this entity reflects Figure 34 5 provides very big dirigibility on use system and method for the present invention.For example, in one embodiment, can reflect Figure 34 5 by script 344 being associated to the entity relevant, and make script 344 in the computer program 346 of early version and redaction, all can use with the computer program 346 of respective version.

Next step 1505 is reflected Figure 34 5 design scripts 344 by reference entity.In design process, entity reflects Figure 34 5 and can be used for determining how to visit and locating the step of concrete graphical user interface object to execute the task.In one embodiment, indivedual programming instructions 347 can reflect Figure 34 5 by this entity and realize that directly (hard coded) becomes this script 344.In alternate embodiments, specific script 344 can dynamically or statically reflect Figure 34 5 with reference to this entity.When carrying out in the computer program 346 of script 344 in redaction, reflect Figure 34 5 by the entity of creating according to 346 reorganizations of redaction computer program, the dynamic reference entity reflects Figure 34 5 and can impel this particular script to be adapted.In one embodiment, when script 344 was performed, the computer program 346 of which version of these script 344 decidables reflected Figure 34 5 with this script 344 of correct execution in operation and with reference to corresponding entity.

Be right after after the design of script 344, step 1510 is to carry out this script 344.This script 344 can be one and come from entity and reflect the special file of Figure 34 5 or directly be embodied as this entity and reflect Figure 34 5.In the process of carrying out script 344, as previously explained, this script 344 shows how carrying out in this computer program 346 of task to the user.It can be by carrying out this task and realize that this can comprise the indication of part coverage diagram layer (for example part coverage diagram layer 1201 shown in Figure 12 A) or its form for the user, perhaps how to carry out this task but do not carry out this task in person by showing to the user.As mentioned above, the instruction 347 of reflecting Figure 34 5 from entity can directly be embodied as this script 344 or selectively this script 344 can dynamically or statically reflect Figure 34 5 with reference to this entity.

In script 344 implementations, step 1515 is display part coverage diagram layers.This part coverage diagram layer directly be shown in this computer program 346 actual non-simulation graphic user interface above.Each part coverage diagram layer makes this computer program 346 carry out a step of this task from visually being shown to the user so that input to be provided.In one embodiment, if carry out this step for the user, each part coverage diagram layer will show that before next step is carried out, the user will have sufficient time to this part coverage diagram layer of preview like this with certain speed.Selectively, each part coverage diagram layer will be shown to this user always and execute the just end afterwards of this correlation step.

This part coverage diagram layer can comprise that how literal, image, animation or video show that to the user the relevant graphic user interface to computer program 346 provides input.This part coverage diagram layer only covers the part of the user interface of this computer program 346, and the user can see that still the relational graph user interface object that needs input is to carry out a step of this task like this.

In addition, in the process of carrying out script 344, part coverage diagram layer is arranged at the position near the relational graph user interface object.The setting of this position is no matter no matter have or not the position of this graphical user interface object of influence of change of any these computer program 346 source codes.As mentioned above, this can judge the position of this graphical user interface object when moving by inquiring about employed system 301, and the position of this part coverage diagram layer correctly is set then.The result of this feature is, user oneself can be directly and this program interactive and no matter which kind of configuration the graphical user interface object of this computer program is.Like this, the User Defined of graphical user interface object can not hinder the execution of this method.In addition because the user be with the configuration that will execute the task under computer program 346 interactions, the user will be more prone to association and carry out this task.Just the user can not obscured by the different situations of the graphical user interface object in the computer program 346.

In the implementation of script, step 1525 is supervisory user inputs at any time, and this can realize by using the input monitor of being explained in conjunction with Fig. 3 A 343.The supervision of this user's input can be used for judging this specific user's skill level.

In one embodiment, step 1530 is that incorrect input is forbidden receiving by this software 346.For example, this can realize by using the method in conjunction with the described filter user input of Figure 11.

In another embodiment, in script 344 implementations, when the user submitted incorrect input to, step 1535 was to provide correct indication to it.This correct indication can be implemented by variety of way, and for example correct indication 1208 shown in Figure 12 C is perhaps by modes such as sound, video or animations.

Figure 16 instructs the operator how to carry out the logic element block scheme of a kind of system 1600 of the step of the task in the computer program 346.Shown logic element can use Fig. 1 and one or more physical components shown in Figure 3 to implement.In addition, perhaps selectable, various logical elements can be by being stored in memory body 112 and/or storer 114 and being implemented by the software module that central processor unit 110 is carried out.Certainly, this logic element also can be combined by software and hardware and implement.If adopt the mode of software to implement, each element can comprise one or more dispersions, but independent operation or with the popular software module of other softwares or hardware compounding practice.This system 1600 does not need to comprise a single software backup, can comprise the software backup of numerous dispersions.

This system can comprise that one reflects the script design element 1605 that Figure 34 5a designs script 344 by reference entity.As mentioned above, this entity reflects Figure 34 5a and comprises that several programming instructions 347 are used to visit the graphical user interface object in the computer program 346.These script design element 1605 these entities of visit reflect Figure 34 5a and remove to comprise or be associated with this script 344 to determine which title 349 or programming instruction 347, with the graphical user interface object in the visit computer program.This title 349 or programming instruction 347 also can be used in the script 344, carry out a step of this task with the position of judging a concrete graphical user interface object.As previously mentioned, the entity of new edition reflects Run Script 344 on the computer software 346 that Figure 34 5b is used in redaction.

In one embodiment, script design element 1605 service recorder elements 1610 design script 344.When recording element 1610 was activated, the user input that is directly inputted into computer program 346 was hunted down and is recorded follow-up execution.The design of script has been simplified in the use of recording element 1610.Can only know a bit or not know the dissimilar script 344 of design under the situation of designing program of this computer program 346.Only by carrying out these steps and writing down them, these steps just can be caught into a script and be used for follow-up execution.In one embodiment, this part coverage diagram layer (for example part coverage diagram layer 1201 shown in Figure 12 A) comprises the indication in this figure layer, can generate automatically in recording process.

Be right after after this design script 344, can carry out this script 344 by a script executing element 1615.This script executing element 1615 comprises a display element 1620, a setting element 1625, and comprises that selectively an input monitors element 1630 and a correct indicator elment 1635.

In the process of implementation, this display element 1620 shows at least one part coverage diagram layer on the graphic user interface of the actual non-simulation of this computer program 346.Each part coverage diagram layer visually shows to provide input to carry out a step of this task to drive computer program 346 to the relational graph user interface object to the user.In one embodiment, this script executing element 1615 also can be carried out the step of this task for the user.

This setting element 1625 is arranged at the position of close relational graph user interface object with part coverage diagram layer, no matter have or not the position of this graphical user interface object of influence of change of any these computer program 346 source codes.As mentioned above, this is an important advantage, because disclosed system and method can be used for any computer program 346 with correlation function of graphical user interface object and these objects of visit, for example operate the computer program 346 of Microsoft's Window operating system.

In one embodiment, input monitors that element 1630 can be used for monitoring the input from the user of receiving.This element 1630 can comprise, for example, and the input monitor 343 shown in Fig. 3 A.This user input can arrive this computer program or even system's input queue 320 (shown in Fig. 3 A) just be hunted down before.This input monitors that element 1630 also can filter or stop those fronts in conjunction with the described incorrect user's input of the method for Figure 12.

One embodiment of the present of invention comprise a correct indicator elment 1635, and this correct indicator elment 1635 is communicated by letter with user's input of determining to receive with input supervision element 1630.If the user who receives input is incorrect, this correct indicator elment 1635 can provide correct indication (for example, the correct indication 1208 shown in Figure 12 C) to give the user to show the input of the correct form that receive.

As previously mentioned, program as described herein can be mixed with known display packing and associated multimedia object.Such object comprises, but is not limited to video, animation, audio files, hyperlink file and image etc.By multimedia object is combined with interaction function as described herein, can develop a kind ofly can provide the software of efficient interactive training for software product, and needn't be as paying a large amount of expenses and work in the prior art.

Be understandable that foregoing layout only is the application of principle of the present invention.Yet the present invention can implement with various concrete forms under the prerequisite that does not break away from essence of the present invention and spirit.Described embodiment is an example of the present invention and non-limiting.Scope of the present invention with claim but not above stated specification be as the criterion.The variation done of with good grounds spirit of the present invention and the extension that is equal to all be contained in the scope of claim of the present invention.

For example, for replenishing of aforementioned functional and effect, the present invention also can be applied to other various application programs by reorganization.What can reckon with in one embodiment of the invention, is that its function with the interaction of graphical user's object can be used to test and confirm the correct function of a software program before this software is distributed to terminal client fully.It is that impotentia or the user that obstacle is arranged automatically perform this task that another embodiment then can allow the developer.By further possible anticipation of the present invention is it in the application that provides to the software terminal user aspect technology and the workflow support, for example supports the supplier by software engineering or by the variety of ways such as script that automatic support program provided of internet.

As mentioned above, the present invention can pass through browser or other application programs, perhaps in its oneself program and memory body space, is used in combination with the LMS/LCMS system and carries out target program.This comprises by the internet, by file system, perhaps with study or help system, the present invention of transmission or its content.

Like this, below the present invention is described in detail in conjunction with most preferred embodiment of the present invention, the Yi Ban Ji Intraoperative people tool of Dui in this Ji Intraoperative Collar territory, for various combinations of the present invention, use or modification, include but not limited to the modification of various sizes, material, shape, form, function and executive mode, do not break away from the scope of claim of the present invention obviously.

Claims (27)

1. computer readable media, it has the computer executable program and be used to carry out the method how a kind of guiding operation person executes the task in computer program, this computer program has at least one graphical user interface object and carries out the input of required action with respective operations person, and it is characterized in that: this method comprises:

Design a script by reflecting figure with reference to an entity, this script comprises how the operator who is shown to this computer program carries out the indication step by step of this task, and wherein this entity reflects figure and comprises that several programming instructions are to visit at least one graphical user interface object of this computer program; And

Carry out script by following step:

Be not to show at least one part coverage diagram layer on the graphic user interface of simulation computer program actual, each part coverage diagram layer visually shows input, and this input is will offer the relational graph user interface object so that this computer program is carried out a step of this task; And

No matter have or not the position of this graphical user interface object of influence of change of any this computer program source code, all this part coverage diagram layer is arranged on position near this relational graph user interface object.

2. computer readable media as claimed in claim 1 is characterized in that: carry out script and comprise that this entity of dynamic reference reflects figure.

3. computer readable media as claimed in claim 2, it is characterized in that: it comprises that further the entity of a redaction reflects figure and creates for the computer program of redaction, and the step of wherein carrying out script comprises with reference to this entity reflects the redaction of figure and with the computer program correct execution of this redaction.

4. computer readable media as claimed in claim 1 is characterized in that: at least one comes from the programming instruction that this entity reflects figure and is directly incorporated into this script.

5. computer readable media as claimed in claim 1 is characterized in that: carry out script and comprise a handling procedure that separates from this computer program.

6. computer readable media as claimed in claim 1 is characterized in that: this method further is included in operator's input that supervision is received in the script executing process.

7. computer readable media as claimed in claim 6 is characterized in that: this method comprises that further stoping incorrect operator to import is received by this computer program.

8. computer readable media as claimed in claim 6 is characterized in that: this method further comprise the indication that provides correct to the operator in response to incorrect operator's input of being received in the script executing process.

9. computer readable media as claimed in claim 1 is characterized in that: the design script comprises the execution of logger task step.

10. computer readable media as claimed in claim 9 is characterized in that: it further comprises when the execution of recording step, sets script and produces the execution of part coverage diagram layer with corresponding task step, and create programming code automatically in this script.

11. computer readable media as claimed in claim 1 is characterized in that: the execution of script comprises that further the dynamic reference graphical user interface object is to determine to be shown in the word language on the part coverage diagram layer.

12. computer readable media, it has the computer executable program and be used to carry out the method how a kind of guiding operation person executes the task in computer program, this computer program has at least one graphical user interface object and carries out the input of required action with respective operations person, and it is characterized in that: this method comprises:

Design a script by reflecting figure with reference to an entity, this script comprises how the operator who is shown to this computer program carries out the indication step by step of this task, and wherein this entity reflects figure and comprises that several programming instructions are to visit at least one graphical user interface object of this computer program; And

Wherein this script comprises programming instruction, and this programming instruction is used for:

Be not to show at least one part coverage diagram layer on the graphic user interface of simulation computer program actual, each part coverage diagram layer visually shows input, and this input is will offer the relational graph user interface object so that this computer program is carried out a step of this task; And

No matter have or not the position of this graphical user interface object of influence of change of any this computer program source code, all this part coverage diagram layer is arranged on position near this relational graph user interface object.

13. computer readable media as claimed in claim 12 is characterized in that: this script is set this entity of dynamic reference and reflects figure in the script executing process.

14. computer readable media as claimed in claim 13, it is characterized in that: it comprises that further the entity of a redaction reflects figure and creates for the computer program of a redaction, wherein reflect figure, set this script and on the computer program of this redaction, suitably carry out by entity with reference to this redaction.

15. computer readable media as claimed in claim 12 is characterized in that: at least one comes from the programming instruction that this entity reflects figure and is directly incorporated into this script.

16. computer readable media as claimed in claim 12 is characterized in that: the execution of several steps by writing down this this task designs this script.

17. the system how guiding operation person executes the task in a computer program, this computer program has at least one graphical user interface object and carries out the input of required action with respective operations person, and it is characterized in that: this system comprises:

An entity reflects figure, and it comprises at least one graphical user interface object that several programming instructions are used to visit this computer program;

A script design element, it reflects figure with reference to this entity and produces script and shows how step by step indication carries out task in this computer program for the operator of computer program; And

A script executing element, it is carried out this script and comprises:

A display element, it is not to show at least one part coverage diagram layer on the graphic user interface of simulation computer program actual, each part coverage diagram layer visually shows input, and this input is will offer the relational graph user interface object so that this computer program is carried out a step of this task; And

A setting element no matter have or not the position of this graphical user interface object of influence of change of any this computer program source code, all is arranged on this part coverage diagram layer the position near this relational graph user interface object.

18. system as claimed in claim 17 is characterized in that: when script executing, this this entity of script executing element dynamic reference reflects figure.

19. system as claimed in claim 17 is characterized in that: it comprises that further an input monitors element, monitors operator's input of receiving in the script executing process.

20. system as claimed in claim 19 is characterized in that: it further comprises a correct indicator elment, according to incorrect operator's input of receiving, provides correct indication to the operator in the script executing process.

21. system as claimed in claim 17 is characterized in that: it further comprises a recording element, and wherein when this recording element was activated, this script design element designed this script by the execution of record script step.

22. system as claimed in claim 17 is characterized in that: this script executing element is to carry out on the type computer on the table.

23. system as claimed in claim 17 is characterized in that: this script executing element is to carry out on wireless telephone.

24. system as claimed in claim 17 is characterized in that: this script executing element is to carry out on personal digital assistant.

Import with respective user 25. the method how guiding operation person executes the task in computer program, this computer program have the required action of at least one graphical user interface object execution, it is characterized in that: this method comprises:

Design a script by reflecting figure with reference to an entity, this script comprises how the operator who is shown to this computer program carries out the indication step by step of this task, and wherein this entity reflects figure and comprises that several programming instructions are to visit at least one graphical user interface object of this computer program; And

Carry out script by following step:

Be not to show at least one part coverage diagram layer on the graphic user interface of simulation computer program actual, each part coverage diagram layer visually shows input, and this input is will offer the relational graph user interface object so that this computer program is carried out a step of this task; And

No matter have or not the position of this graphical user interface object of influence of change of any this computer program source code, all this part coverage diagram layer is arranged on position near this relational graph user interface object.

26. method as claimed in claim 25 is characterized in that: carry out this script and comprise that this entity of dynamic reference reflects figure.

27. method as claimed in claim 26, it is characterized in that: it comprises that further the entity of a redaction reflects figure and creates for the computer program of redaction, wherein carries out script and comprises with reference to this entity and reflect the redaction of figure and with the computer program correct execution of this redaction.

Images