WO2001075683A2 - Computerized system and method for soliciting, collecting and evaluating suggestions from structured and unstructured stimuli via a computer network - Google Patents

Abstract

A method and apparatus for suggestion solicitation, collection, and evaluation. The method and apparatus enables individuals and organizations to obtain and provide real-time, anonymous, and subject-specific feedback via a computer network, such as the Internet, to improve their decision-making capabilities. Bi-directional anonymous communication is established to allow for clarification on prior suggestion submissions and better targeted future submission. Indirect feedback is gathered among consenting users sharing anonymous responses to similar suggestion-solicitation stimuli. An anonymous incentive-awarding mechanism is provided to enhance the efficiency of the information flow. The method and apparatus further provides an intermediate processing layer between the suggestion or information sought and the suggestion or information delivered; such intermediate processing includes systematic evaluation, filtering, and knowledge extraction steps.

Description

COMPUTERIZED SYSTEM AND METHOD FOR SOLICITING,

COLLECTING AND EVALUATING SUGGESTIONS FROM STRUCTURED AND

UNSTRUCTURED STIMULI VIA A COMPUTER NETWORK

FIELD OF THE INVENTION

The present invention relates in general to methods of acquiring useful

information via a computer network, such as the Internet. More specifically,

the present invention relates to a method and apparatus enabling individuals

or institutional entities to obtain and provide real-time, anonymous, and

subject-specific feedback to improve their information gathering, information

providing, and decision-making capabilities. In particular, the present invention

relates to a method and apparatus for soliciting and organizing suggestions

(or feedback) in electronic format via a computer network, such as the

Internet, using electronic suggestion boxes; and processing such feedback

information by evaluation, filter, and knowledge extraction mechanisms

before delivering to the entities that seek the feedback. The present

invention further relates to an assembly of individual suggestion boxes, inter¬

connected on the World Wide Web, and thus allows sharing of anonymous

feedback information among consenting users.

BACKGROUND OF THE INVENTION

Suggestion boxes have been used by "organizations" (i.e., business,

government and non-profits) to solicit suggestions and feedback from their

members and constituents. Perhaps the most common type of conventional

suggestion collection system is the one associated with the purchase of

goods or services. A consumer can usually provide feedback to a merchant

by filling out a questionnaire and dropping it off in a physical suggestion box

at the store, or by sending a letter or completed questionnaire to the

company via the postal service, or electronically, by clicking on a

"suggestion" or "feedback" icon/link on the company's Web site.

It is noteworthy that the use of suggestion boxes associated with

"private individuals" (e.g., consumers, employees, members of organizations)

is rare, if at all present, in existing suggestion collection schemes. For

example, it is difficult for a given individual to gather feedback, in an

anonymous fashion, from other individual(s) with whom he or she may be

interacting with for business or other purposes, and whose opinion may be

of interest or importance to him or her in specific areas. In addition,

conventional suggestion collection systems are also limited with respect to

their accessibility, anonymity, privacy, and evaluation.

Great accessibility is desirable because it will increase the volume of

the data collected, as well as enhance the diversity, thus increase the

representativeness of the data. To receive a large and diverse quantity of data, physical suggestion boxes need to be situated in as many locations as

possible. In the example of individuals, this task is complex and burdensome

because one may need to interact with a myriad of people in a myriad of

places. Also, it may be impracticable for an individual to place physical

suggestion boxes over a large geographic area, due to various constraints

(costly to rent private property for such placement). The problem of

accessibility is less pronounced for organizations, since they generally

occupy the real property where they transact with customers or where

employees congregate to work, and the customers and employees are the

constituents they care to receive suggestions from. However, the lack of

flexibility in physical suggestion boxes remains a significant obstacle for their

accessibility.

Electronic implementations of suggestion boxes allow an individual or

institutional entity to use a designated site on the Web to gather suggestions

from a potentially large and diversified group of contributors. Typically, web

sites collect suggestions or feedback information electronically via dedicated

e-mail addresses (e.g., suggestions@xyzcorp.com) or html forms accessible

directly on the web site. Additionally, electronic forums (e.g., bulletin boards

and chat rooms) on the recipient's or a third-party's web site may be used

for the same purpose, where suggestions and feedback are submitted and

viewed by the general public. The seemingly improved accessibility,

however, provides little benefit to individual private users because most of them do not have the facility to create and maintain their own web site, and

to receive significant number of hits required to gain meaningful pools of

suggestions. Meanwhile, the available web-based suggestion-collection

systems do not offer mechanisms for organizing, evaluating, and filtering

information gathered. Therefore, a new paradigm of collecting and managing

suggestion information with consistent high accessibility is needed.

Anonymity is an important factor for any system that is designed to

effectively deal with information related to people. The e-mail or bulletin

board-based suggestion collection systems discussed above, for example, do

not afford desired anonymity, because a contributor's identity is easily

discoverable though various tracking techniques over the Internet, such as

using cookies. Current e-mail-based suggestion boxes also inevitably require

a return e-mail address, which tend to discourage contributors from

submitting suggestions in some situations when they are concerned about

revealing their identity. The lack of anonymity in these existing suggestion

collection systems also impairs the web site owners' ability to fashion

incentives and monetary rewards to effectively encourage suggestion

submission, since contributors may be reluctant to disclose who they are in

order to receive the reward. Therefore, better resolutions for the anonymity

issue are desired to liberate potential contributors, hence to secure a stable

volume and diversified resources for suggestion or information of interest. Inseparable from the anonymity issues are the concern of privacy. For

example, an electronic forum, one form of the current suggestion system,

may discourage potential suggestion recipients from seeking certain

feedback, because the posted data is of a public nature. This leads to

compromised volume and completeness of the collectable information. The

seeker of a suggestion is more likely to reveal important issues, often

sensitive and private, in constructing stimuli to obtain feedback, if he or she

is ensured that the information gathered can only be viewed by him or her

self in a secure and private manner. Similarly, in certain situations, a

provider of a suggestion is more likely to submit a suggestion on sensitive

issues when such submission can be carried out in a private manner.

Evaluation is helpful for a suggestion collection and managing system.

As data may grow exponentially through efficient stimuli and incentive-

induced collecting, the task of procedurally and intellectually evaluating and

processing the data becomes increasingly important. The traditional and

limited evaluation by the data-recipients themselves often results in biased

conclusions due to their preexisting dispositions.

The issue of evaluation also implicates the failure of current

suggestion collection systems to learn from common themes that transcend

various unaffiliated suggestion boxes, since these systems operate

independently of each other (e.g., through their respective and distinct web

sites). It is impossible for the existing systems to provide indirect feedback on a topic of broadly-shared interest, for example, to an organization or

individual, when no direct feedback to this organization or individual has

been received. The utility of such systems is therefore drastically limited.

It is clear that aforementioned drawbacks related to accessibility,

anonymity, privacy, and evaluation render the currently available suggestion

collection and managing systems less than desirable for the effective

acquiring and using of feedback information in the new Information Age. An

improved system with higher accessibility, better anonymity and privacy, and

more sophisticated evaluation mechanisms is needed by both institutional

and individual entities in their pursuance of useful feedback information.

SUMMARY OF THE INVENTION

To resolve the above problems, the present invention is directed to a

computerized suggestion solicitation, collection and evaluation system and

method, which uses a computer network, such as the Internet, to provide

real-time, anonymous feedback to individuals and organizations improve their

information gathering, information providing, and decision-making capabilities. A

"suggestion" is defined to be any commentary, opinion, idea, factual

statement, and the like, in various formats such as text, graphical, audio,

video, and the like. Suggestions may be obtained in response to a stimulus,

i.e., a questionnaire, a graphic or video image, an audio recording, and a

physical encounter. The recipient of a suggestion is referred to as a "suggestionee. " The provider of a suggestion is referred to as a

"suggestioner."

In accordance with one aspect of the invention, a system and method

are provided for collecting one or more suggestions from a first entity (a

suggestioner) and providing collected suggestions to a second entity (a

suggestionee) comprising a memory for storing one or more suggestions

submitted by the first entity in response to a stimulus selected by the second

entity and an interface permitting the second entity to access the stored

suggestions. The suggestions may be released in unedited ("raw") form to

the suggestionee, or they may be formatted and transformed to evaluated

feedback before being delivered to the suggestionee.

It is an object of the invention to provide a computerized system that

displays, controls and facilitates the submission and evaluation of

anonymous suggestions from a large universe of participants that are inter-

connected via a computer network, such as the Internet. It is a further

object to provide an integrated information collecting and processing system

that provides a middle layer between the information provider and the

information receiver. Passing the suggestions through a middle layer makes

it possible to optionally provide anonymity to the suggestioner and to

optionally analyze or format the suggestions before they are accessed by the

suggestionee. In accordance with another aspect of the invention, the system

achieves high level of accessibility by allowing both institutional and private

users to license the use of the system for their particularized needs. As

licensed users or subscribers, people can easily seek, obtain, or submit

suggestions through the system.

In accordance with yet another aspect of the invention, the anonymity

of the persons or organizations seeking or submitting suggestions is

ensured, because the system operates as a third-party trustee of their

identity information (e.g., name, e-mail address, Uniform Resource Locator,

Internet Protocol address, etc.) . Similarly, the participants may be

guaranteed privacy whether they submit suggestions or seek feedback

information in any particular area, since the present system affords them an

insulated or personalized scope. The suggestioners and suggestionees can

also disclose their identity at their own discretion thereby making the system

anonymous in both directions or one direction based on the users' decision.

In accordance with another aspect of the invention, the present

system provides an "evaluator," for evaluating collected information and

organizing it into meaningful suggestions to be delivered to the

suggestionee. An evaluator can be an automated process or agent

preconfigured in the system, or an individual or institutional entity registered

to participate as an evaluator, rather than a suggestioner or suggestionee. In accordance with another aspect of the invention, the basic building

block of the present system is a "suggestion box," which is implemented in

the form of a set of computer programs and related databases. A suggestion

box is assigned to a specific suggestionee; it is used to gather and process

suggestions he or she desires. A suggestion box comprises five major sub¬

components: 1 ) a "Suggestion Stimuli Input Tray"; 2) a "Raw Suggestion

Output Tray"; 3) an "Evaluated Suggestion Output Tray"; 4) a "Create

Suggestion-Stimuli Process"; and 5) a "Suggestion Evaluation Process". The

Suggestion Stimuli Input Tray is openly accessible to the public or a defined

group of participants according to the suggestionee. The other four sub¬

components are private to the suggestionee; a unique and secure computer

identification and password scheme is used.

In accordance with another aspect of the invention, a Network

Commonality Determination Process identifies the common themes among

suggestion-box stimuli and responses, and establish sharable suggestions to

participants having similar interests. Indirect feedback information can

therefore be provided among consenting participants, which is particularly

useful when stimuli-induced direct suggestions are lacking.

In accordance with the another aspect of the invention, a Redeem

Awards process allows the system to use incentives, such as monetary

awards, to encourage participation, maintain a steady information flow,

hence effectively generate and deliver useful suggestions. The present invention, therefore, brings significant improvements to

existing methods and facilities for obtaining meaningful feedback information

in a business setting, academic world, or private persons' everyday life.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features will now be described with reference to the

drawings summarized below. These drawings and the associated

descriptions are provided to illustrate preferred embodiments of the

invention, and not to limit the scope of the invention. Throughout the

drawings, reference numbers are reused to indicate correspondence between

referenced elements. In addition, the first digit of each reference number

indicates the figure in which the element first appears.

FIG. 1 is a diagrammatic illustration of a computerized system for

suggestion solicitation, collection, and evaluation, according to an

embodiment of the present invention;

FIG. 2 is a diagram illustrating a layered architecture of the

computerized system according to the present invention;

FIG. 3 is a diagram illustrating a set of processes or modules in the

computerized system according to the present invention;

FIG. 4 is a flowchart illustrating an example of a sequence of steps of

the Open Account 301 shown in FIG. 3; FIG. 5 is a flowchart illustrating an example of a sequence of steps of

the Create Suggestion-Stimuli Process 303 shown in FIG. 3;

FIG. 6 is a flowchart illustrating an example of a sequence of steps of

the Suggestion-Stimuli Input Tray 305 shown in FIG. 3;

FIG. 7 is a flowchart illustrating an example of a sequence of steps of

the Suggestion Evaluation Process 307 shown in FIG. 3;

FIG. 8 is a flowchart illustrating an example of a sequence of steps of

the Raw Suggestion Output Tray 309 shown in FIG. 3;

FIG . 9 is a flowchart illustrating an example of a sequence of steps of

the Network Commonality Determination Process 31 1 shown in FIG. 3;

FIG. 1 0 is a flowchart illustrating an example of a sequence of steps

of the Evaluated Suggestion Output Tray 31 3 shown in FIG. 3;

FIG. 1 1 is a flowchart illustrating an example of a sequence of steps

of the Redeem Awards 31 5 shown in FIG. 3.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to Fig. 1 , the computerized suggestion system in a preferred

embodiment of the invention operates in a client-server environment, where

users communicate via the Internet 1 1 1 . Users may be a plurality of persons

or organizations that either solicit suggestions (suggestionee 1 03), submit

suggestions (suggestioner 1 05), or evaluate suggestions (evaluator 1 07) .

Each user is a client when he or she connects to the system via a web browser interface (1 1 3 for suggestionees, 1 1 5 for suggestioners, and 1 1 7

for evaluators, respectively) .

The sever engine 1 01 manages all processes in the computerized

suggestion system, including producing and dispatching stimuli, gathering

and evaluating suggestions, and targeted suggestion delivering. Various

types of information processed by the system are stored in a suitable

database in one embodiment of the invention, or a set of interconnected

databases in another embodiment of the invention. For example, dedicated

suggestionee database 1 21 , suggestioner database 1 23, and evaluator

database 1 25 may be established. Particularly relevant in technical

applications of the present system, a separate database registering expert

evaluators for certain knowledge domains (expert database 1 29) is included.

A database maintaining rewards information (rewards database 1 27) and

another database for compiling sharable suggestion information

(commonality database 1 31 ) may also optionally be implemented in this

embodiment of the invention.

Referring to Fig. 2, the computerized suggestion system according to

the present invention contains an intermediate processing layer 205 between

the information receiver/suggestionee layer 201 and the information

provider/suggestioner layer 203. Suggestionee produces stimuli for potential

suggestions, which was integrated in the middle processing layer 205 and

was dispatched to the suggestioner layer 203. Induced by such stimuli, suggestioners submit suggestions to the system, which are sent to the

middle processing layer 205 to go through the evaluation process.

Evaluation strategies correspond to the type of stimuli used, and hence

ensure that the meaningful and desired suggestions, or advice and ideas, can

be extracted for delivering to the suggestionee. Multiple suggestioners may

submit various suggestions to a suggestionee via one or multiple suggestion

boxes. One suggestionee may host multiple suggestion boxes receiving

suggestions from a plurality of suggestionees. Between a specific

suggestioner and a specific suggestionee, multiple suggestion boxes

configured with different purposes and functions may be used for

information exchange.

The mid processing layer 205 may be configured to perform a

plurality of evaluation steps according to the present invention. An individual

or business entity registered with the system as an evaluator can manually

review, categorize, and summarize the submitted suggestion to produce

deliverable suggestion "knowledge." An expert evaluator, in addition,

focuses on evaluating the soundness of a suggestion based on a specific

technical or business area, and synthesizing from the raw suggestion

submissions the high-level "knowledge" or "theme," in response to the

suggestionee's stimuli. Thus, a suggestionee is provided with a spectrum of

suggestion products by the computerized suggestion system, ranging from

raw suggestion submissions to processed and integrated suggestion knowledge, according to the particular needs of the suggestionee. In other

embodiments of the present invention, automated evaluation steps are used

to perform statistical analysis, text parsing for words of interest, for

example. Sophisticated knowledge mining techniques such as machine

learning are used for analyzing complex data submissions. These processes

and steps are implemented in the form of computer programs, and operate

under the synchronized control of the present system.

Fig. 3 illustrates the processes and modules in the computerized

suggestion system according to the present invention. First, a user opens an

account with the system and enters the demographic information (301 ) . A

suggestionee that seeks suggestions then creates customized stimuli (303)

to induce suggestions and feedback from potential suggestioners, who will

subsequently be presented with the customized stimuli (305) . Suggestions

gathered by the system are delivered to the suggestionee in a raw form

(309) or evaluated form (31 3) . Suggestion evaluation is managed by the

system (307), as discussed above. In addition, the system also allows the

suggestionee to evaluate raw submissions him or herself using the evaluation

tools provided in the system; or to specifically designate a group of

evaluators. Network Commonality Determination Process 31 1 identifies

potential common themes, i.e., similarity and correlation between various

stimuli and corresponding responses. This allows a suggestionee to access

indirect yet relevant suggestion information, although there is no direct stimuli-induced submission available. Such mechanism of sharing is based

on participants' voluntary consent, it increases the applicability of the

information, as well as enhances the accessibility. The effectiveness of the

computerized suggestion system according to the present invention is

supported by an incentive awarding process, Redeem Awards (31 5) .

Suggestioners or evaluators earn rewards for the submissions and evaluation

they made. Each of the above processes and modules are described in more

detain in the following description.

(D OPEN ACCOUNT

The Open Account process registers a user on the system either as a

Suggestionee (the recipient of suggestions), a Suggestioner (the provider of

suggestions) or as an Evaluator (one that evaluates previously submitted

suggestions) . During the process of registering, the system captures

demographic information associated with the user. The true identity (name

or e-mail address) may be volunteered by the user, but is not required as part

of the registration process.

Referring to Fig. 4, the Opening Account process 301 in an

embodiment of the present invention includes a sequence of steps. The

system creates a user id and password 401 and records background

information, such as demographic data 403. The system then prompts the

user how he or she want to register and processes the input accordingly. If

registered as a suggestionee, the record is stored in suggsetionee database 1 21 ; if registered as a suggestioner, the record is stored in suggestioner

database 1 23; and if registered as an evaluator, the record is stored in

evaluator database 1 25.

(2) CREATE SUGGESTION-STIMULI PROCESS

Referring to Fig. 5, the Create Suggestion Stimuli Process 303

includes a sequence of steps to produce suggestion stimuli, in various

formats including textual, graphical, multimedia, etc. A suggsetionee

initiates the process by logging onto the system using the designated user id

and password. The choices of generating structural or non-structural

questionnaires are presented. Experts' input are needed to control the

format of structural questionnaires (501 ), whereas unstructured

questionnaires take a much more relaxed and flexible form (503) . To

produce structured questionnaires, the system conducts a computerized

interview with the suggestionee. A series of questions are presented to the

suggestionee, which are pre-compiled in the system based on the expert

knowledge in the suggestionee's field, e.g., from a psychologist, a

motivational coach, or a management consultant. Such expert's knowledge

on specific matters is previously categorized and stored in a linked computer

database, the expert database 1 29. The response from the suggestionee is

used to either retrieve a list of previously created questionnaires from which

the suggestionee can choose, or to formulate a new questionnaire. The

purpose is to create a best-suited document for eliciting suggestions that would effectively assist the suggestionee in achieving certain specific

objectives.

In one embodiment of the present invention, the examples of

questionnaires include the ones related to health, parenting or financial goals;

musical performances, meetings or presentations; business reports or

correspondence; and comments on efficiency improvement, working

conditions, new product, compensation, etc.

The questionnaires (or other similar stimuli) are placed in the

Suggestion Stimuli Input Tray 305 to allow simultaneously cross-reference in

a separate computer database, the commonality database 1 31 in subsequent

analyses, as described below (see Network Commonality Determination

Process) . Briefly, the suggestionee is presented with the option of allowing

others to be able to view this questionnaire, and any associated responses,

in an anonymous fashion. This feature makes indirect suggestions available

to other participating suggestionees, i.e., they may see responses to similar

questionnaires although no direct suggestions to their own and similar

questionnaire are received) . Setting the Commonality Flag to "TRUE" (505)

enables this feature for a given questionnaire; and a setting of "FALSE"

disables this feature. This setting is maintained in the suggestionee's record

within the Suggestionee Database 1 21 .

Furthermore, the Create Suggestion-Stimuli Process 303 allows a

suggestionee to establish incentive awards (e.g., monies, goods or services) to be subsequently displayed along with the suggestion stimuli in the

Suggestion Stimuli Input Tray 305. This mechanism is to promote and

encourage the submission of thoughtful anonymous suggestions; it works in

the following way. If a suggestionee chooses to offer incentives for a

particular stimuli, the system interacts with the suggestionee to determine

the amount, type, and conditions of the rewards (507) and sets up an

escrow account (509) to deposit any reward monies. The Rewards

Database 1 27 is subsequently updated (51 1 ) .

In one embodiment of the present invention, the final step of the

Create Suggestion Stimuli Process 303 gives a suggestionee the option of

creating an e-mail distribution list to inform potential suggestioners of the

newly created contents in his Suggestion Stimuli Input Tray 305. If choose

to use this option, the suggestionee is prompted to provide an e-mail

distribution list (51 3); this list is subsequently entered into the suggestionee

database 1 21 .

(3) SUGGESTION STIMULI INPUT TRAY

The goal of the Suggestion Stimuli Input Tray 305 is to provoke

structured and/or unstructured responses (i.e., suggestions) to subject

matter that is of interest to a suggestionee. The stimuli in the Suggestion

Stimuli Input Tray 305 include questionnaires, and any other textual, visual,

or audio and video presentations, that have been created by the Create

Suggestion Stimuli Process 303. In addition, the Suggestion Stimuli Input Tray 305 is used to display an incentive award (e.g., a given amount of

money, a piece of consumer goods, or the enjoyment of certain type of

services) to promote and encourage the submission of suggestions.

Referring to Fig. 6, the Suggestion Stimuli Input Tray 305 involves a

series of operations in one embodiment of the present invention. Once a

suggestioner logs onto the host web site of the computerized suggestion

system, he or she is prompted to enter a search term to retrieve an specific

suggestion box associated with a registered suggestionee (601 ) that is of

interest. The suggestioner may choose the intended suggestionee, and

download the contents of the suggestionee's Suggestion Stimuli Input Tray

305 to his own computer and view the contents (603) through the web

browser 1 1 5. The Suggestioner may submit suggestions in response to any

of the listed suggestion stimuli (e.g., various questionnaires) . If the intended

suggestionee is not found to have been registered on the system, the

suggestioner may create a temporary suggestion box (608) and submit a

private, anonymous suggestion to the unregistered suggestionee. The

system prompts the suggestioner for the e-mail address (or other contact

information, such as a postal address or telephone number) of the

suggestionee (609), creates a user id and password (61 0) and informs the

suggestionee that an unsolicited suggestion has been received by the

system. The email notification contains the user id and password, which

may be used to unlock the private, temporary suggestion box and allow the suggestionee to view the content via the Raw Suggestion Output Tray (See

Fig. 8). The temporary suggestion (content) and the user id and password of

the unregistered suggestionee is subsequently stored in the Suggestionee

Database (61 0) .

In an embodiment of the invention, the suggestioner is not required to

open an account on the system, in which case he or she may not engage in

anonymous bi-directional communication nor share any available rewards

offered in the system. If, instead, the suggetioner chooses to open an

account on the system, the Open Account Process 301 is executed. In

another embodiment of the invention, this allows a suggestioner to uses a

unique id and password when submitting a suggestion, and each such

submission to be identified by a unique suggestion number. An option of

enabling future bi-directional communications with the suggestionee (605) is

then presented to the suggestioner, which allows the suggestionee to seek

further clarification and feedback from the suggestioner. Furthermore, if the

suggestionee has established a reward for the stimuli to which the

suggestioner responded with a suggestion, the suggestioner is given the

reward based on established criteria (607) .

(4) RAW SUGGESTION OUTPUT TRAY

The Raw Suggestion Output Tray 309 presents suggestions gathered

from suggestioners in their original form. A suggestionee may view the

contents of his Raw Suggestion Output Tray 309 once he enters the unique user identification number and password. Further, a suggestionee may

communicate with an anonymous suggestioner if this privilege has been

given by the suggestionee at the time when the suggestion was submitted.

This bi-directional communication allows the suggestionee to seek

clarification or additional data in a secure and anonymous manner from the

suggestioner on a specific suggestion submission.

Referring to Fig. 8, the Raw Suggestion Output Tray 309 involves a

sequence of operations. At the host web site of the system, a suggestionee

enters his unique user identification and password (801 ) . The suggestionee

then selects a specific stimuli (803), e.g., a specific questionnaire, from a list

of presented stimuli that have been previously created in the Create

Suggestion-Stimuli Process 303. The system accordingly presents a list of

suggestions gathered in response to that stimulus, which the suggestionee

can browse through (805) . The suggestionee may select an individual

response and request that additional information/clarification be provided by

the suggestioner (807) . This is accomplished by querying the suggestioner

database 1 23 to determine if the necessary bi-directional communication has

been granted by the suggestioner. If such communication was enabled, the

suggestionee can enter a clarification request (807) that is then forwarded to

the suggestioner via e-mail (809) .

(5) SUGGESTION EVALUATION PROCESS The Suggestion Evaluation Process 307 allows suggestions to be

evaluated and organized before delivered to a suggestionee, by a group of

evaluators, experts, or automated system processes. In one embodiment of

the invention, a suggestionee identifies the set of suggestions to be

evaluated; and the system automatically recommends suitable evaluators,

e.g., a group of computer scientists, for the suggestions, based on

demographic or other type of information related to various evaluators. In

another embodiment, the suggestionee may set up a new group of

evaluators; and recommend each evaluator in the group by entering their

identity information to the system. Each evaluator is assigned a random

subset of the suggestions collected in response to a stimuli to process.

Referring to Fig. 7, the Suggestion Evaluation Process 307 involves a

sequence of detailed operations in one embodiment of the invention. A

suggestionee first enters his unique user identification and password, and

then selects a specific stimulus (701 ), such as a questionnaire, from a

presented list of stimuli that have been created in the Create Suggestion-

Stimuli Process 303. The system displays a list of collected suggestion

responses to the selected stimulus, along with a list of available evaluators

or automated evaluation processes that may be suitable for evaluating the

suggestions (703) . The suggestionee may either accept the recommendation

of evaluators by the system, or create a new group of evaluators, by

entering the e-mail addresses and other information of those evaluators to the system (705). Each evaluator is assigned a subset (the "Response

Subset Group") of the gathered suggestions (the "Response Population") in

a random fashion. As samples are drawn from the Response Population, the

suggestion number associated with each suggestion is used to cross-

reference a response (or suggestion), a stimulus (or questionnaire) in other

processes and operations in the system.

In this embodiment of the invention, it is possible for the same sample

(i.e., suggestion) to be assigned to different evaluators. The system only

ensures that each evaluator has been assigned a Response Subset Group of

equal size and that all suggestions in the Response Population have been

distributed (707) . The assignment of Response Subset Groups to the

evaluators is recorded in the evaluator database 1 25. The system then

transmits a Response Subset Group to each evaluator via e-mail. The e-mail

account may either be on the system or on a third party e-mail system as

determined by the evaluator when he registered with the system using the

Open Account process 301 . In the cases where automated system

evaluation process are used, such as statistical analysis, complex data

correlation analysis, and knowledge extraction using machine learning tools,

etc., the allocation of suggestions to various processes are made

automatically; and the system synchronizes these various operations to

optimize overall performance. An example of the type of evaluation is to rate the suggestions:

evaluators rank the suggestions contained in their respective Response

Subset Groups from "best" to "worst" based on a preset numbering system,

e.g., "best" equals "1 " and "worst" equals 1 0. Evaluators submit their

evaluation result to the system, which at a reasonable interval, gathers and

integrates evaluation results from the group of evaluators. In the case of

ranking data, a temporary array "Ranking Array" is used to organize the

result, which eventually holds the consensus ranking for all evaluated

suggestions. The Ranking Array is an associated array where the rate of

each suggestion is linked to its suggestion number, e.g., referenced by a

computer program as Ranking_Array[suggestion number][ranking].

In an embodiment where none of the suggestions is allowed to share

the same ranking, the system processes all suggestions to determine their

ranking by maintaining a running sum in the Ranking Array. This is

accomplished by: a) retrieving the current ranking for each entry in the

Ranking Array using the suggestion numbers; b) inserting the newly

processed ranking to the ranking list retrieved in the previous step, one at a

time; and c) placing the newly ranked entry into the enlarged Ranking Array,

one at a time. The resulted Ranking Array contains a running total of all

rankings of the evaluated suggestions, each identifiable by its suggestion

number. In another embodiment, a suggestion can have the same numeric

ranking value as some other suggestions; such suggestions are deemed to be

of equal value for the purpose of awarding any rewards. The reward monies

may be distributed among the suggestioners who submitted the highest

ranked suggestions. The evaluation results and the rewards data are

subsequently used to update the evaluator database 1 25 and the

suggestionee database 1 21 .

The system may choose to engage in additional rounds of evaluation

(709); and it may also choose to run a number of different evaluation steps,

such as statistical analysis, etc. outlined above. After completing all

evaluation steps, the system queries the suggestionee database 1 21 to

determine if the Commonality Flag was set TRUE by the suggestionee in the

Create Suggestion-Stimuli Process 303 to allow sharing of the suggestions.

If it was set TRUE, then the commonality database 1 31 is updated (71 1 ) and

the evaluated suggestions are made available for viewing by other

suggestionees who had also enabled their Commonality Flag for similar

suggestion stimuli.

(6) EVALUATED SUGGESTION OUTPUT TRAY

The Evaluated Suggestion Output Tray 31 3 displays the evaluated

suggestions output from the Suggestion Evaluation Process 307. The

evaluated suggestions may be rank-ordered . Referring to Fig. 1 0, the

Evaluated Suggestion Output Tray 31 3 involves a sequence of operations in the preferred embodiment of the invention. At the host web site of the

system, a suggestionee enters his user identification and password, and

selects a stimuli (e.g., a questionnaire) for which he intends to view

responses (1 001 ) . The suggestionee is then presented with a list of

evaluated responses generated from the Suggestion Evaluation Process 307

that are associated with the selected stimuli (1 003) . The suggestionee may

select an individual suggestion, and request that additional information or

clarification be provided by any of the various evaluators. This is

accomplished by the system querying the evaluator database to determine

which evaluators had enabled bi-directional communication with the

suggestionee (1 005). Once an evaluator or a group of evaluators are

identified who had enabled such communication, the suggestionee may enter

a clarification question (1 007), which is then forwarded to the evaluator(s)

via e-mail (1 009).

(7) NETWORK COMMONALITY DETERMINATION PROCESS

The system enables a suggestionee to receive indirect suggestions to

similar suggestion stimuli that are used by other suggestionees. This

privilege of sharing suggestion feedback is only extended on a reciprocal

basis to other suggestionees having set the Commonality Flag TRUE in the

Create Suggestion-Stimuli Process 303, in the preferred embodiment of this

invention. The system invokes the Network Commonality Determination

Process 31 1 automatically at a preset reasonable interval. Referring to Fig. 9, the Network Commonality Determination Process

31 1 includes a sequence of steps in one embodiment of this invention. The

system periodically and automatically parses the suggestionee database 1 21

to determine if the Commonality Flag was set by new and existing

suggestionees in the Create Suggestion-Stimuli Process 303 (901 ). Once

the system detects a TRUE Commonality Flag, it copies the suggestion

stimuli and the associated suggestion responses to the commonality

database 1 31 , from the Suggestion Stimuli Input Tray 305 and Raw

Suggestion Output Tray 309, respectively (903) . The system then again

parses the suggestionee database 1 21 , and compares all stimuli or

questionnaires found in the suggestionee database 1 21 to those in the

commonality database 1 31 . If a match is found, then those suggestionees

who had previously set the Commonality Flag TRUE are informed (e.g., via e-

mail) of available indirect suggestions (905) . These suggestions are then

automatically copied to the respective Raw Suggestion Output Trays 309 of

the participating suggestionees (907) .

(8) REDEEM AWARDS

The Redeem Awards process 31 5 allows a suggestioner or evaluator

to redeem their reward points which they earned by submitting a suggestion

via the Suggestion Stimuli Input Tray 305, and by evaluating suggestions in

the Suggestion Evaluation Process 307. A Reward Point may be defined to

1 /100^th of a US Dollar ($0.01 ), in an embodiment of this invention; in other embodiments of the invention, reward points may be converted to goods or

services at a fair market price.

Referring to Fig. 1 1 , the Redeem Awards process 31 5 includes a

sequence of steps. A registered suggestioner or evaluator first enters his

user identification and password at the host web site of the system and

chooses Redeem Awards by clicking on a labeled button. The system then

performs a query to the rewards database 1 27, and retrieves and displays

the balance of the user's rewards account (1 1 01 ) . The user then selects the

amount of Reward Points to redeem (1 1 03). The Rewards Points can be

redeemed to either cash or goods and services of equal value. If the user

elects to convert Reward Points to cash, then the system prompts the user

to enter the necessary information to process the transaction (e.g., bank

routing numbers, credit card accounts, name, address, etc.) (1 1 05). The

system then disburses the funds (e.g., prints a physical check and mails it

via the postal system, or transfers funds electronically to a bank or credits a

charge card account) (1 1 07). If the user elects to convert Reward Points to

good or services, then the system prompts the user for the information

necessary to complete the transaction. The system then updates the

suggestioner database 1 23, the evaluator database 1 25, the suggestionee

database 1 21 , and the rewards database 1 27. In one embodiment, the

system may post the amount of any award on the host web site. Although only a few exemplary embodiments of this invention have

been described in detail above, those skilled in the art will readily appreciate

that many modifications are possible in the exemplary embodiments without

materially departing from the novel teachings and advantages of this

invention. Accordingly, all such modifications are intended to be included

within the scope of this invention.

Claims

What is claimed is:

1 . A system for collecting one or more suggestions from a first entity and

providing access to collected suggestions to a second entity comprising a

memory for storing one or more suggestions submitted by the first entity in

response to a stimulus selected by the second entity and an interface

permitting the second entity to access the stored suggestions.

2. A method for obtaining suggestions, comprising selecting a stimulus

to solicit suggestions from a first party, said suggestions being stored in a

memory, and obtaining access to suggestions stored in the memory through

an interface displayed by a server.

3. A system of flow of suggestions, comprising:

(a) at least one solicitor providing one or more stimuli for soliciting

suggestions;

(b) at least one respondent providing one or more raw suggestions in

response to said one or more stimuli; and

(c) a processor that is capable of organizing, evaluating, and otherwise

processing said one or more raw suggestions, resulting in processed

suggestions which are then delivered to said solicitor.

4. The system of claim 3, wherein said processed suggestions are

substantially the same as said raw suggestions.

5. The system of claim 3, wherein said processed suggestions are

delivered to a second solicitor which provides one or more substantially

similar stimuli.

6. The system of claim 3, wherein said processed suggestions are

delivered to a receiver which does not provide any stimuli and which is

designated by said solicitor.

7. The system of claim 3, further comprising a channel whereby said

solicitor engage in direct communication with said respondent after receiving

said processed suggestions.

8. The system of claim 3, wherein said stimuli further comprises rewards

incentives.

9. The system of claim 3, wherein the flow of suggestions is via a

computer network, and wherein a single interface on said network is

provided to said each solicitor and to said each respondent to support said

flow of suggestions.

10. The system of claim 3, further comprising a database having stored

therein information of said solicitor, said respondent, said stimuli, and said

raw and processed suggestions.

1 1 . The system of claim 1 0, further comprising a user interface whereby

said solicitor and said respondent access said information.

1 2. A method for obtaining suggestions, comprising

(a) providing one or more stimuli by at least one solicitor for soliciting

suggestions;

(b) gathering one or more raw suggestions from at least one

respondent in response to said one or more stimuli;

(c) organizing, evaluating, and otherwise processing said one or more

raw suggestions, resulting in processed suggestions; and

(d) delivering said processed suggestions to said solicitor.

1 3. The method of claim 1 2, wherein said processed suggestions are

substantially the same as said raw suggestions.

1 4. The method of claim 1 2, further comprising delivering said processed

suggestions to a second solicitor which provides one or more substantially

similar stimuli.

1 5. The method of claim 1 2, further comprising delivering said processed

suggestions to a receiver which does not provide any stimuli and which is

designated by said solicitor.

1 6. The method of claim 1 2, further comprising providing rewards

incentives along with said stimuli.

1 7. A method for providing suggestions, comprising

(a) providing one or more stimuli by at least one solicitor for soliciting

suggestions;

(b) gathering one or more raw suggestions from at least one

respondent in response to said one or more stimuli;

(c) organizing, evaluating, and otherwise processing said one or more

raw suggestions, resulting in processed suggestions; and

(d) delivering said processed suggestions to a receiver designated by

said solicitor.

1 8. A system of flow of suggestions, comprising: (a) at least one solicitor providing one or more stimuli for soliciting

suggestions;

(b) at least one respondent providing one or more raw suggestions in

response to said one or more stimuli; and

(c) a processor that is capable of organizing, evaluating, and otherwise

processing said one or more raw suggestions, resulting in processed

suggestions which are then delivered to a receiver designated by said

solicitor.