US20140025690A1 - Content ranking system and method - Google Patents
Content ranking system and method Download PDFInfo
- Publication number
- US20140025690A1 US20140025690A1 US14/036,610 US201314036610A US2014025690A1 US 20140025690 A1 US20140025690 A1 US 20140025690A1 US 201314036610 A US201314036610 A US 201314036610A US 2014025690 A1 US2014025690 A1 US 2014025690A1
- Authority
- US
- United States
- Prior art keywords
- user
- content
- scoring
- content item
- score
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G06F17/3053—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2457—Query processing with adaptation to user needs
- G06F16/24578—Query processing with adaptation to user needs using ranking
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
Definitions
- the content items When displaying content items to a user, the content items are typically ranked so that a desired set of content items is displayed first.
- One problem is that, when a user rating is used to rank items, a content item with only a few ratings that are very high can be displayed within the desired set. However, this content item may not be appropriately ranked because the few very high ratings may not represent the content item's rating after more rating are aggregated.
- FIG. 1 is a graph illustrating a simulated history of the user interaction score, us(t), in one embodiment.
- FIG. 3A , 3 B, and 3 C are graphs illustrating embodiments of the distribution of the different scoring contributions.
- the scores are normalized to make it easier to compare the different scoring contributions, and the normalized frequency refers to relatively how many documents obtain a particular score.
- the invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor.
- these implementations, or any other form that the invention may take, may be referred to as techniques.
- the order of the steps of disclosed processes may be altered within the scope of the invention.
- a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task.
- the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.
- Ranking content items is disclosed.
- a user input is received from each of one or more users indicating an opinion of the user with respect to a content item included in a plurality of content items. Based at least in part on a number of users from whom user input has been received, a degree is determined to which a ranking of the content item relative to one or more other content items in the plurality of content items is determined by user input.
- the content item is associated with an event.
- the user input is weighted by a user reputation.
- the degree of contribution of user ratings to the overall ranking varies depending on how many user ratings there are; if there are very few, they are not counted much (e.g., to discount being a non-representative sample); if there are more, they are counted more; at some point though the contribution levels out with regard to the number of user ratings.
- a relevance of a content item to an event is determined.
- a user rating of the content item is received.
- a ranking of the content item is determined with respect to the event, relative to one or more other content items associated with the event based at least in part on the relevance and the user rating. In some embodiments, the ranking is determined based at least in part on a sentiment score associated with the content item.
- the ranking algorithm disclosed provides a novel method for retrieving, scoring, ranking, and presenting content related to an event based on a combination of: Geography, Time, Similarity, Sentiment, User rating and User preferences. It is aimed at providing users with the most relevant, engaging, and informative content concerning events of interest to them.
- This document presents the components of a content ranking algorithm and describes the various scoring contributions that it utilizes.
- the ranking algorithm dynamically combines the various scoring contributions in different ways over time, as the importance of those respective contributions (see section 2.3) will vary over time.
- the overall scoring method for a content item and an event are both defined in section 2.4.
- An example of an example scoring history for a content item in the system can be seen in FIGS. 5A-5D .
- the most basic element of scoring a content item is its relevance to a given event in the information retrieval sense, i.e., some characterization of similarity to a query or set of queries, or to a base or “seed” content item or set of items, associated with the event. This scoring contribution provides a baseline characterization of how topically related the content item is to that event.
- the model of similarity is fundamentally text overlap, for example, using the standard term frequency-inverse document frequency (TF-IDF) model, overlap of named entities, or similar measures.
- TF-IDF standard term frequency-inverse document frequency
- Two additional scoring contributions also address the topical relevance of a content item to an event: temporal and geographic proximity of times and locations mentioned in the content item, or attached as metadata to the content item, to times and locations associated with the event.
- the ranking algorithm also assesses the emotional impact of content items, in terms of valence (i.e., positive or negative and with what intensity), as part of the process of ranking those items. This scoring contribution allows the system to identify and prioritize potentially high-impact content.
- the ranking algorithm also employs user ratings as one of its scoring contributions.
- the user rating score (U(t), see section 2.3.1) is defined by both the number and value of user votes (s(t)) with user reputation (h m (t)) taken into account. Basically, a few votes by highly reputable users will have the same effect as a large number of votes from less reputable users. This behavior is defined by the equations (3) and (4) described below.
- the overall score of a content item will be primarily defined by the relevance score (R(t), see section 2.3.2) and the emotional score (E(t), see section 2.3.3).
- R(t) the relevance score
- E(t) the emotional score
- the effect of the user votes increases.
- the algorithm is forgiving of isolated “mistakes” by users, such as submission of low quality content or less frequent visits to the system.
- long term abusive behavior will result in much slower growth of a user's reputation.
- the exact definition of the user reputation, h m (t), is found in section 2.2.
- An event score is defined by a weighted sum of the user rating score of the event itself and the (possibly weighted) average total score of the content items associated with that event. It can easily be extended to be valid for non-linear weights.
- the user interaction score measures the interactivity of a user with the system.
- Some general characteristics of the user interaction score include:
- FIG. 2 An example of a user interaction history, u s (t), is shown in FIG. 2 .
- User reputation is defined as a weighted sum of the long term user interaction history (defined by the model parameter t 1 ) as well as the more recent user interaction history (defined by the model parameter t 0 ) inside the system. (See FIG. 2 for the effect of variations of these model parameters.)
- the user reputation, h m (t), of a user m is then given by a weighted sums of the different partial user reputations, h i,m ,
- h m ⁇ ( t ) ⁇ ⁇ 1 ⁇ h - 1 , m for ⁇ ⁇ t ⁇ t 0 ⁇ 1 ⁇ h - 1 , m + ⁇ 0 ⁇ h 0 , m for ⁇ ⁇ t 0 ⁇ t ⁇ t 1 ⁇ 1 ⁇ h 1 , m + ⁇ 0 ⁇ h 0 , m for ⁇ ⁇ t ⁇ t 1 ( 1 )
- the integral can be computed using simple numerical integration.
- a too high ratio of t 1 /t 0 will damp fluctuations of the user reputation over time.
- t 1 1000 days (almost 3 years)
- h m (t) the overall growth in the user reputation
- This artifact is purely an effect of the formulation of equation (1) and can be eliminated by introducing time-dependent weights ⁇ 0 (t) and ⁇ 1 (t). Large values of ⁇ 0 will drastically damp high-frequency fluctuations in the user reputation in time. Basically, choosing large values of ⁇ 0 results in low-frequency fluctuations in the user reputation, h m (t), in time.
- FIGS. 3A-3C The scoring distributions are shown in FIGS. 3A-3C .
- FIGS. 3A , 3 B, and 3 C are graphs illustrating the distribution of the different scoring contributions in one embodiment.
- the scores are normalized to make it easier to compare the different scoring contributions, and the normalized frequency refers to relatively how many documents obtain a particular score.
- each user vote, s m j should be weighted depending on their user reputation, h m .
- the user rating contribution can easily be extended to vary in different stages of the event detection process.
- the system will automatically detect such interactions, such attempts will result in a lower user interaction score and consequently a lower effective user rating score.
- the system allows the user interaction behavior to be dynamic over time, and the formulation of the user reputation ensures that the system automatically responds to such attempts and recalculates the total score of a content item in response.
- s j e,n is the effective user rating score.
- the weighting function below has been specifically designed to solve the issues previously mentioned. It gives low weight for users with low reputation and for the intermediate interval of the user reputation has an only slightly varying constant weight (Part 1) and controls the obtained weight for users with higher user reputation (Part 2). Also as the user reputation varies with time, the hierarchy in the system is time-dependent and thus also the effective vote, s j e,m , by a user m on the content item j.
- the weighting function, ⁇ (h m ), is therefore defined as,
- ⁇ ⁇ ( h m ) 1 + ( h m - h m 1 h m 1 ) h m 2 ⁇ Part ⁇ ⁇ 1 ⁇ exp ⁇ ( - h m h m 3 ) ⁇ Part ⁇ ⁇ 2 ( 4 )
- the relevance score, R(t), measures how relevant an item of content is to a particular event.
- An event is a collection of stories, blogs, user contributions, videos and images and can thus be considered as a container of content items j. (Typical events are, for instance, “The Conflict in Lebanon”, “The tragic death of Josh Hancock” or “Tom Cruise and Katie Holmes wedding”).
- the relevance score of an item will depend on:
- An important element of any story, video or image is the emotional element, which determines the emotional impact a particular item of content has.
- the emotional score, E(t) provides a quantitive measure on the sentiment, attitude and/or opinion of piece of content—whether it is positive or negative, and how much. Sentiment of a content item in this sense can be assessed using a variety of statistical or other methods, for the most part derived from corpora of labeled data used to determine features or feature combinations indicative of particular sentiment values.
- a content item can be a document, an image, a video or any other type of content in the system.
- the score of a content item j is denoted, g j , and is defined as a weighted sum of its scoring contributions. Let the vector of the scoring contributions be defined as
- v i j (t) denotes the i th element of the vector v j (t)
- the score, g j then defined by,
- N i is the normalization constant of contribution i
- ⁇ i j are the weighting function for the vector element v i j (t).
- the scores need to be normalized as the different scores result from different sorts of computation, have different magnitudes, and are not directly comparable.
- the weighting functions used in the scoring model are,
- ⁇ i 0 is the model parameter for weight of scoring contribution i.
- the weighting function, f(U j (t)), is defined as,
- FIGS. 5A-5D Some examples and basic sensitivity analysis of the score are presented in FIGS. 5A-5D .
- the user rating score for the event k itself be U 0 k , the average score for the associated content to event k, g ac k , and ⁇ k a weight factor.
- the total event score can then be defined as,
- g k( t ) ⁇ k ⁇ U 0 k ( t )+(1 ⁇ k ) ⁇ g ac k ( t )
- the system provides users with the ability to access and contribute content about events of interest to them, defining these events, tracking the evolution of the events over time, and organizing the events and the associated content to ensure users receive the most interesting and engaging experience.
- An event in its basic form is a collection of content items that refer to the same situation or circumstance. The scope of each event will be dynamic and vary over time.
- a content item may be a story, an image and/or a video. Both the content items and the events will be stored in databases and files, including XML-files.
- the ranking algorithm is distributed throughout the system, and consists of both on-demand and batch processes, including steps of both a serial and parallel nature.
- Step I Defining the scores and/or updating the user interaction record:
- a content item is registered at the system: The content item is matched against existing seeds and events: If the content item matches the contextual scope of an event: Update user interaction history for the particuiar user and/or source submitting the content item If no matching event or seed is found to an item: • Update user interaction record for the particuiar user and/or source submitting the content item • Create a new event instance (seed) in the database Calculate scoring contributions: Calculate the emotional score of the content item Update the contextual state of the event Calculate the relevance scores for content items associated with the event A user interacts with the system: Switch depending on the types of interaction with the system: A user vote is registered: • Update the user interaction record for 1.
- the ranking algorithm provides an appropriate prioritization of the content related to an event, automatically and without relying on human intervention, and is therefore scalable.
- the algorithm utilizes a combination of different scoring contributions -user rating, relevance or similarity, geography, time, and emotional impact.
- the user ratings are weighted by user reputation to get the necessary slight community hierarchy structure, as it is well-known that a too-flat authority hierarchy internally can destroy groups as much as one that is too steep.
- the scoring algorithm solves some of the common problems with user ratings by varying its contribution vis a vis other scoring contributions in a dynamic manner.
- the community help decide what is important, intelligent information retrieval technology help to decide what is relevant, and emotional analysis help to decide what is impactful.
- the ranking algorithm can assess and prioritize content along all these three dimensions—community, information retrieval, emotional analysis -jointly.
- the ranking algorithm together with the event detection model, also enables the swift detection of breaking events, which any model based on human editing or community assessment alone will be unable to match.
- the simplicity of the ranking algorithm makes it easy to implement but also makes the dynamics of the score predictable.
- the algorithm can also be easily extended to include additional scoring contributions, and it can be optimized via only a few model parameters.
- the formulation of the ranking algorithm makes it possible to introduce more complex weighting functions for different types of content, different users, and different types of events, varying over time as information about those events evolves.
Abstract
Description
- This application is a continuation of, and claims a benefit of priority under 35 U.S.C. 120 of the filing date of U.S. patent application Ser. No. 12/215,684, by inventors Tareen et al., entitled “RANKING CONTENT ITEMS RELATED TO AN EVENT,” filed on Jun. 27, 2008, which in turn claims the benefit of priority under 35 U.S.C. §119 to U.S. Provisional Patent Application No. 60/937,685, entitled “SYSTEM AND METHODS FOR RETRIEVING, RANKING, AND PRESENTING CONTENT RELATED TO AN EVENT,” filed Jun. 29, 2007, which are both incorporated herein by reference for all purposes.
- When displaying content items to a user, the content items are typically ranked so that a desired set of content items is displayed first. One problem is that, when a user rating is used to rank items, a content item with only a few ratings that are very high can be displayed within the desired set. However, this content item may not be appropriately ranked because the few very high ratings may not represent the content item's rating after more rating are aggregated.
- Various embodiments of the invention are disclosed in the following detailed description and the accompanying drawings.
-
FIG. 1 is a graph illustrating a simulated history of the user interaction score, us(t), in one embodiment. -
FIG. 2 is a graph illustrating an example of user reputation, hm(t): t0=21 days, t1=365 days, ω0=0.99, ω0=0.01 in some embodiments. -
FIG. 3A , 3B, and 3C are graphs illustrating embodiments of the distribution of the different scoring contributions. The scores are normalized to make it easier to compare the different scoring contributions, and the normalized frequency refers to relatively how many documents obtain a particular score. -
FIG. 4A and 4B are graphs illustrating an example of a user rating score, U(t), and the corresponding weighting function, f(U), for U1=20000 as a typical history of user votes, s, in one embodiments. -
FIG. 5A , 5B, 5C, and 5D are graphs illustrating an example of a total Masala score for a content piece for a user score history for U1=20000, ω1 0=0.2, ω2 0=0.4, and ω3 0=0.4. - The invention can be implemented in numerous ways, including as a process; an apparatus; a system; a composition of matter; a computer program product embodied on a computer readable storage medium; and/or a processor, such as a processor configured to execute instructions stored on and/or provided by a memory coupled to the processor. In this specification, these implementations, or any other form that the invention may take, may be referred to as techniques. In general, the order of the steps of disclosed processes may be altered within the scope of the invention. Unless stated otherwise, a component such as a processor or a memory described as being configured to perform a task may be implemented as a general component that is temporarily configured to perform the task at a given time or a specific component that is manufactured to perform the task. As used herein, the term ‘processor’ refers to one or more devices, circuits, and/or processing cores configured to process data, such as computer program instructions.
- A detailed description of one or more embodiments of the invention is provided below along with accompanying figures that illustrate the principles of the invention. The invention is described in connection with such embodiments, but the invention is not limited to any embodiment. The scope of the invention is limited only by the claims and the invention encompasses numerous alternatives, modifications and equivalents. Numerous specific details are set forth in the following description in order to provide a thorough understanding of the invention. These details are provided for the purpose of example and the invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, technical material that is known in the technical fields related to the invention has not been described in detail so that the invention is not unnecessarily obscured.
- Ranking content items is disclosed. A user input is received from each of one or more users indicating an opinion of the user with respect to a content item included in a plurality of content items. Based at least in part on a number of users from whom user input has been received, a degree is determined to which a ranking of the content item relative to one or more other content items in the plurality of content items is determined by user input. In some embodiments, the content item is associated with an event. In some embodiments, the user input is weighted by a user reputation. In some embodiments, the degree of contribution of user ratings to the overall ranking varies depending on how many user ratings there are; if there are very few, they are not counted much (e.g., to discount being a non-representative sample); if there are more, they are counted more; at some point though the contribution levels out with regard to the number of user ratings.
- A relevance of a content item to an event is determined. A user rating of the content item is received. A ranking of the content item is determined with respect to the event, relative to one or more other content items associated with the event based at least in part on the relevance and the user rating. In some embodiments, the ranking is determined based at least in part on a sentiment score associated with the content item.
- In some embodiments, the ranking algorithm disclosed provides a novel method for retrieving, scoring, ranking, and presenting content related to an event based on a combination of: Geography, Time, Similarity, Sentiment, User rating and User preferences. It is aimed at providing users with the most relevant, engaging, and informative content concerning events of interest to them.
- 2 Description
- This document presents the components of a content ranking algorithm and describes the various scoring contributions that it utilizes.
- 2.1 Conceptual Description
- The ranking algorithm dynamically combines the various scoring contributions in different ways over time, as the importance of those respective contributions (see section 2.3) will vary over time. The overall scoring method for a content item and an event are both defined in section 2.4. An example of an example scoring history for a content item in the system can be seen in
FIGS. 5A-5D . - The most basic element of scoring a content item is its relevance to a given event in the information retrieval sense, i.e., some characterization of similarity to a query or set of queries, or to a base or “seed” content item or set of items, associated with the event. This scoring contribution provides a baseline characterization of how topically related the content item is to that event. The model of similarity is fundamentally text overlap, for example, using the standard term frequency-inverse document frequency (TF-IDF) model, overlap of named entities, or similar measures.
- Two additional scoring contributions also address the topical relevance of a content item to an event: temporal and geographic proximity of times and locations mentioned in the content item, or attached as metadata to the content item, to times and locations associated with the event.
- The ranking algorithm also assesses the emotional impact of content items, in terms of valence (i.e., positive or negative and with what intensity), as part of the process of ranking those items. This scoring contribution allows the system to identify and prioritize potentially high-impact content.
- Finally, the ranking algorithm also employs user ratings as one of its scoring contributions. The user rating score (U(t), see section 2.3.1) is defined by both the number and value of user votes (s(t)) with user reputation (hm(t)) taken into account. Basically, a few votes by highly reputable users will have the same effect as a large number of votes from less reputable users. This behavior is defined by the equations (3) and (4) described below.
- For small numbers of user votes, the overall score of a content item will be primarily defined by the relevance score (R(t), see section 2.3.2) and the emotional score (E(t), see section 2.3.3). As the number of user votes grows, i.e., becomes statistically reliable, the effect of the user votes increases.
- Finally, as the number of user votes increases further, the importance of additional user votes decreases. This behavior is modeled by the weighting function defined in equation (9) described below.
- User reputation is based on both long term and short term interaction with the system.
- By defining the user reputation as a function of both long term and short term interactions in this way, the algorithm is forgiving of isolated “mistakes” by users, such as submission of low quality content or less frequent visits to the system. On the other hand, long term abusive behavior will result in much slower growth of a user's reputation. The exact definition of the user reputation, hm(t), is found in section 2.2.
- An event score is defined by a weighted sum of the user rating score of the event itself and the (possibly weighted) average total score of the content items associated with that event. It can easily be extended to be valid for non-linear weights.
- 2.2 Definition of the User Reputation, hm(t)
- The user interaction score, us(t), measures the interactivity of a user with the system. Some general characteristics of the user interaction score include:
- 1 The larger the number of visits, the higher user interaction score.
- 2 The higher frequency of revisits, the higher user interaction score.
- 3 Attempts to cram the system, as well as abusive behavior or language, will lower the user's interaction score.
- 4 The quality of a user's submitted content is also an important factor in his or her interaction score.
- An example of a user interaction history, us(t), is shown in
FIG. 2 . - User reputation is defined as a weighted sum of the long term user interaction history (defined by the model parameter t1) as well as the more recent user interaction history (defined by the model parameter t0) inside the system. (See
FIG. 2 for the effect of variations of these model parameters.) The user reputation, hm(t), of a user m is then given by a weighted sums of the different partial user reputations, hi,m, -
- where ω1 and ω0 are weights and ω0+ω1=1. The respective partial user reputation h−1,m h0,m and h1,m . . . is defined as,
-
- where t−1=t1, τ−1,0=0, τ0,0=t−t0 and τ1,0=t−t1 and us(τ) is the user interaction score at time τ. Note that τ−t≦0 for all t≧0. The integral can be computed using simple numerical integration.
- The model parameters—t0, t1 and ω0—enable control of the relative weight between recent and long term actions. A too high ratio of t1/t0 will damp fluctuations of the user reputation over time. For t1=1000 days (almost 3 years), there is a visual artifact that one can obtain an overall growth in the user reputation, hm(t), for higher t1. This artifact is purely an effect of the formulation of equation (1) and can be eliminated by introducing time-dependent weights ω0(t) and ω1(t). Large values of ω0 will drastically damp high-frequency fluctuations in the user reputation in time. Basically, choosing large values of ω0 results in low-frequency fluctuations in the user reputation, hm(t), in time.
- 2.3 Definition of the Scoring Contributions
- The scoring distributions are shown in
FIGS. 3A-3C .FIGS. 3A , 3B, and 3C are graphs illustrating the distribution of the different scoring contributions in one embodiment. The scores are normalized to make it easier to compare the different scoring contributions, and the normalized frequency refers to relatively how many documents obtain a particular score. - 2.3.1 User Rating Score, U(t)
- The importance of user rating contribution should increase over time, but should also be bounded for large t. Furthermore, to handle different abusive behaviors in online communities such as creation of multiple accounts to vote multiple times, other cramming attempts, and/or to fight online trolls, each user vote, sm j, should be weighted depending on their user reputation, hm. The user rating contribution can easily be extended to vary in different stages of the event detection process.
- As the system will automatically detect such interactions, such attempts will result in a lower user interaction score and consequently a lower effective user rating score. The system allows the user interaction behavior to be dynamic over time, and the formulation of the user reputation ensures that the system automatically responds to such attempts and recalculates the total score of a content item in response.
- Let s be the user vote by user m on the content piece j, hm(t), the reputation of the user m and ω(u) be the weighting function depending of the user reputation h(t). (Note that ω(u) is a different weight than the previously mentioned ω0 and ω1.) Furthermore, let Sj=Sj(t), be the set of users who have voted on a content piece j. The total user rating score, Uj=Uj(t) for a content piece j, will then be defined as,
-
- where sj e,n is the effective user rating score. The weighting function below has been specifically designed to solve the issues previously mentioned. It gives low weight for users with low reputation and for the intermediate interval of the user reputation has an only slightly varying constant weight (Part 1) and controls the obtained weight for users with higher user reputation (Part 2). Also as the user reputation varies with time, the hierarchy in the system is time-dependent and thus also the effective vote, sj e,m, by a user m on the content item j.
- The weighting function, ω(hm), is therefore defined as,
-
- where hi m are are model parameters.
- 2.3.2 Relevance Score, R(t)
- The relevance score, R(t), measures how relevant an item of content is to a particular event. An event is a collection of stories, blogs, user contributions, videos and images and can thus be considered as a container of content items j. (Typical events are, for instance, “The Conflict in Lebanon”, “The tragic death of Josh Hancock” or “Tom Cruise and Katie Holmes wedding”). The relevance score of an item will depend on:
-
- Geographical overlap between the content item and the event
- Temporal overlap between the content item and the event
- Similarity between the content item and the event as described earlier.
- 2.3.3 Emotional Score, E(t)
- An important element of any story, video or image is the emotional element, which determines the emotional impact a particular item of content has. The emotional score, E(t), provides a quantitive measure on the sentiment, attitude and/or opinion of piece of content—whether it is positive or negative, and how much. Sentiment of a content item in this sense can be assessed using a variety of statistical or other methods, for the most part derived from corpora of labeled data used to determine features or feature combinations indicative of particular sentiment values.
- 2.4 Definition of the Scores
- 2.4.1 The Score of a Content Item
- A content item can be a document, an image, a video or any other type of content in the system. The score of a content item j is denoted, gj, and is defined as a weighted sum of its scoring contributions. Let the vector of the scoring contributions be defined as
-
vj(t)=[Uj(t) Rj(t) Ej(t)], - where vi j(t) denotes the ith element of the vector vj(t) The score, gj, then defined by,
-
- where Ni is the normalization constant of contribution i and ωi j are the weighting function for the vector element vi j(t). The scores need to be normalized as the different scores result from different sorts of computation, have different magnitudes, and are not directly comparable. The weighting functions used in the scoring model are,
-
ω1 j=ω1 0·ƒ(U j(t)) (user rating score weight) (6) -
ω2 j=ω2 0 (relevance score weight) (7) -
ω3 j=ω3 0 (emotional score weight) (8) - where ωi 0 is the model parameter for weight of scoring contribution i. The weighting function, f(Uj(t)), is defined as,
-
-
FIGS. 4A and 4B are graphs illustrating an example of a user rating score, U(t), and the corresponding weighting function, f(U), for U1=20000 as a typical history of user votes, s, in one embodiment. - Some examples and basic sensitivity analysis of the score are presented in
FIGS. 5A-5D .FIG. 5A , 5B, 5C, and 5D are graphs illustrating an example of a total score for a content piece for a user in one embodiment for U1=20000, w0 1=0.2, w0 2=0.4, and w0 3=0.4. - 2.4.2 The Event Score
- Let the user rating score for the event k itself be U0 k, the average score for the associated content to event k,
g ac k , and ωk a weight factor. The total event score can then be defined as, -
gk(t)=ωk ·U 0 k(t)+(1−ωk)·g ac k(t) - Note that the total event score is then only given by one model function 107 k, and the respective scores Uk 0(t) and g-ac k(t) can be defined via one SQL-query.
- 3 Description of the Preferred Embodiment
- The system provides users with the ability to access and contribute content about events of interest to them, defining these events, tracking the evolution of the events over time, and organizing the events and the associated content to ensure users receive the most interesting and engaging experience. An event in its basic form is a collection of content items that refer to the same situation or circumstance. The scope of each event will be dynamic and vary over time. A content item may be a story, an image and/or a video. Both the content items and the events will be stored in databases and files, including XML-files.
- The ranking algorithm is distributed throughout the system, and consists of both on-demand and batch processes, including steps of both a serial and parallel nature.
-
Step I: Defining the scores and/or updating the user interaction record: A content item is registered at the system: The content item is matched against existing seeds and events: If the content item matches the contextual scope of an event: Update user interaction history for the particuiar user and/or source submitting the content item If no matching event or seed is found to an item: • Update user interaction record for the particuiar user and/or source submitting the content item • Create a new event instance (seed) in the database Calculate scoring contributions: Calculate the emotional score of the content item Update the contextual state of the event Calculate the relevance scores for content items associated with the event A user interacts with the system: Switch depending on the types of interaction with the system: A user vote is registered: • Update the user interaction record for 1. the user initially submitting/creating the particular content item and/or event 2. the user voting on the particular content item and/or event • update the user votes A user abuse report is registered: • Update the user interaction record for 1. the user initially submitting on the content item or event 2. the user filing the abuse report on the content item or event All other user interactions with the system: Update the user interaction records for the users affected by the particular interaction Redefine user reputations of affected users and calculate the user rating score Recalculate the user reputations for the users affected Calculate user rating score using the defined user reputations Step II: Calculate the final scores: Calculate total score for the content item Update total event score Step III: Update the state of the system Reprioritize content items for each event and the events Update the content storage - 4 Advantages
- The ranking algorithm provides an appropriate prioritization of the content related to an event, automatically and without relying on human intervention, and is therefore scalable. The algorithm utilizes a combination of different scoring contributions -user rating, relevance or similarity, geography, time, and emotional impact. The user ratings are weighted by user reputation to get the necessary slight community hierarchy structure, as it is well-known that a too-flat authority hierarchy internally can destroy groups as much as one that is too steep. The scoring algorithm solves some of the common problems with user ratings by varying its contribution vis a vis other scoring contributions in a dynamic manner. The community help decide what is important, intelligent information retrieval technology help to decide what is relevant, and emotional analysis help to decide what is impactful. Therefore the ranking algorithm can assess and prioritize content along all these three dimensions—community, information retrieval, emotional analysis -jointly. The ranking algorithm, together with the event detection model, also enables the swift detection of breaking events, which any model based on human editing or community assessment alone will be unable to match.
- The simplicity of the ranking algorithm makes it easy to implement but also makes the dynamics of the score predictable. The algorithm can also be easily extended to include additional scoring contributions, and it can be optimized via only a few model parameters. The formulation of the ranking algorithm makes it possible to introduce more complex weighting functions for different types of content, different users, and different types of events, varying over time as information about those events evolves.
- Although the foregoing embodiments have been described in some detail for purposes of clarity of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and not restrictive.
Claims (20)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/036,610 US20140025690A1 (en) | 2007-06-29 | 2013-09-25 | Content ranking system and method |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US93768507P | 2007-06-29 | 2007-06-29 | |
US12/215,684 US8548996B2 (en) | 2007-06-29 | 2008-06-28 | Ranking content items related to an event |
US14/036,610 US20140025690A1 (en) | 2007-06-29 | 2013-09-25 | Content ranking system and method |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/215,684 Continuation US8548996B2 (en) | 2007-06-29 | 2008-06-28 | Ranking content items related to an event |
Publications (1)
Publication Number | Publication Date |
---|---|
US20140025690A1 true US20140025690A1 (en) | 2014-01-23 |
Family
ID=40363778
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/215,684 Active 2029-06-07 US8548996B2 (en) | 2007-06-29 | 2008-06-28 | Ranking content items related to an event |
US14/036,610 Abandoned US20140025690A1 (en) | 2007-06-29 | 2013-09-25 | Content ranking system and method |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/215,684 Active 2029-06-07 US8548996B2 (en) | 2007-06-29 | 2008-06-28 | Ranking content items related to an event |
Country Status (1)
Country | Link |
---|---|
US (2) | US8548996B2 (en) |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120180138A1 (en) * | 2007-11-15 | 2012-07-12 | Yahoo! Inc. | Trust based moderation |
US20130031107A1 (en) * | 2011-07-29 | 2013-01-31 | Jen-Yi Pan | Personalized ranking method of video and audio data on internet |
US20130332468A1 (en) * | 2012-06-07 | 2013-12-12 | Spigit, Inc. | User Reputation in Social Network and eCommerce Rating Systems |
US9159056B2 (en) | 2012-07-10 | 2015-10-13 | Spigit, Inc. | System and method for determining the value of a crowd network |
US9542616B1 (en) | 2015-06-29 | 2017-01-10 | International Business Machines Corporation | Determining user preferences for data visualizations |
US10497032B2 (en) * | 2010-11-18 | 2019-12-03 | Ebay Inc. | Image quality assessment to merchandise an item |
US10545938B2 (en) | 2013-09-30 | 2020-01-28 | Spigit, Inc. | Scoring members of a set dependent on eliciting preference data amongst subsets selected according to a height-balanced tree |
US10599979B2 (en) | 2015-09-23 | 2020-03-24 | International Business Machines Corporation | Candidate visualization techniques for use with genetic algorithms |
US10685035B2 (en) | 2016-06-30 | 2020-06-16 | International Business Machines Corporation | Determining a collection of data visualizations |
Families Citing this family (64)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9535911B2 (en) * | 2007-06-29 | 2017-01-03 | Pulsepoint, Inc. | Processing a content item with regard to an event |
US8583633B2 (en) * | 2007-11-30 | 2013-11-12 | Ebay Inc. | Using reputation measures to improve search relevance |
US8819564B1 (en) * | 2008-02-22 | 2014-08-26 | Google Inc. | Distributed discussion collaboration |
US8468153B2 (en) * | 2009-01-21 | 2013-06-18 | Recorded Future, Inc. | Information service for facts extracted from differing sources on a wide area network |
US20100325205A1 (en) * | 2009-06-17 | 2010-12-23 | Microsoft Corporation | Event recommendation service |
US10353967B2 (en) | 2009-06-22 | 2019-07-16 | Microsoft Technology Licensing, Llc | Assigning relevance weights based on temporal dynamics |
JP5533868B2 (en) * | 2009-07-14 | 2014-06-25 | ソニー株式会社 | Content recommendation system, content recommendation method, content recommendation device, and information storage medium |
US8688434B1 (en) | 2010-05-13 | 2014-04-01 | Narrative Science Inc. | System and method for using data to automatically generate a narrative story |
US8355903B1 (en) | 2010-05-13 | 2013-01-15 | Northwestern University | System and method for using data and angles to automatically generate a narrative story |
US9208147B1 (en) | 2011-01-07 | 2015-12-08 | Narrative Science Inc. | Method and apparatus for triggering the automatic generation of narratives |
US8374848B1 (en) | 2010-05-13 | 2013-02-12 | Northwestern University | System and method for using data and derived features to automatically generate a narrative story |
US9009065B2 (en) * | 2010-12-17 | 2015-04-14 | Google Inc. | Promoting content from an activity stream |
US9697178B1 (en) | 2011-01-07 | 2017-07-04 | Narrative Science Inc. | Use of tools and abstraction in a configurable and portable system for generating narratives |
US8886520B1 (en) | 2011-01-07 | 2014-11-11 | Narrative Science Inc. | Method and apparatus for triggering the automatic generation of narratives |
US10185477B1 (en) | 2013-03-15 | 2019-01-22 | Narrative Science Inc. | Method and system for configuring automatic generation of narratives from data |
US9720899B1 (en) | 2011-01-07 | 2017-08-01 | Narrative Science, Inc. | Automatic generation of narratives from data using communication goals and narrative analytics |
US8630844B1 (en) | 2011-01-07 | 2014-01-14 | Narrative Science Inc. | Configurable and portable method, apparatus, and computer program product for generating narratives using content blocks, angels and blueprints sets |
US9576009B1 (en) | 2011-01-07 | 2017-02-21 | Narrative Science Inc. | Automatic generation of narratives from data using communication goals and narrative analytics |
US8775161B1 (en) | 2011-01-07 | 2014-07-08 | Narrative Science Inc. | Method and apparatus for triggering the automatic generation of narratives |
US10657201B1 (en) | 2011-01-07 | 2020-05-19 | Narrative Science Inc. | Configurable and portable system for generating narratives |
US9697197B1 (en) | 2011-01-07 | 2017-07-04 | Narrative Science Inc. | Automatic generation of narratives from data using communication goals and narrative analytics |
US8892417B1 (en) | 2011-01-07 | 2014-11-18 | Narrative Science, Inc. | Method and apparatus for triggering the automatic generation of narratives |
US9246957B2 (en) * | 2011-03-04 | 2016-01-26 | Viafoura | Systems and methods for interactive content generation |
US9519682B1 (en) * | 2011-05-26 | 2016-12-13 | Yahoo! Inc. | User trustworthiness |
US8650252B2 (en) * | 2011-09-28 | 2014-02-11 | Facebook, Inc. | Instantaneous recommendation of social interactions in a social networking system |
US9378287B2 (en) | 2011-12-14 | 2016-06-28 | Patrick Frey | Enhanced search system and method based on entity ranking |
KR101434624B1 (en) * | 2012-03-30 | 2014-08-27 | 단국대학교 산학협력단 | System for analyzing story of cinematographic work based on estimating tension of user and method for thereof |
US9396502B2 (en) | 2013-01-23 | 2016-07-19 | Facebook, Inc. | Enabling delayed interactions with content items presented by a social networking system |
US20140214570A1 (en) * | 2013-01-25 | 2014-07-31 | Bazaarvoice, Inc. | Method and system for content submission continuation |
US10902004B2 (en) * | 2013-10-16 | 2021-01-26 | Salesforce.Com, Inc. | Processing user-submitted updates based on user reliability scores |
US11238056B2 (en) | 2013-10-28 | 2022-02-01 | Microsoft Technology Licensing, Llc | Enhancing search results with social labels |
US9971756B2 (en) | 2014-01-03 | 2018-05-15 | Oath Inc. | Systems and methods for delivering task-oriented content |
US10503357B2 (en) | 2014-04-03 | 2019-12-10 | Oath Inc. | Systems and methods for delivering task-oriented content using a desktop widget |
US9558180B2 (en) * | 2014-01-03 | 2017-01-31 | Yahoo! Inc. | Systems and methods for quote extraction |
US11645289B2 (en) | 2014-02-04 | 2023-05-09 | Microsoft Technology Licensing, Llc | Ranking enterprise graph queries |
US9870432B2 (en) | 2014-02-24 | 2018-01-16 | Microsoft Technology Licensing, Llc | Persisted enterprise graph queries |
US11657060B2 (en) * | 2014-02-27 | 2023-05-23 | Microsoft Technology Licensing, Llc | Utilizing interactivity signals to generate relationships and promote content |
US10757201B2 (en) | 2014-03-01 | 2020-08-25 | Microsoft Technology Licensing, Llc | Document and content feed |
US10255563B2 (en) | 2014-03-03 | 2019-04-09 | Microsoft Technology Licensing, Llc | Aggregating enterprise graph content around user-generated topics |
US10061826B2 (en) | 2014-09-05 | 2018-08-28 | Microsoft Technology Licensing, Llc. | Distant content discovery |
US10747823B1 (en) | 2014-10-22 | 2020-08-18 | Narrative Science Inc. | Interactive and conversational data exploration |
US11922344B2 (en) | 2014-10-22 | 2024-03-05 | Narrative Science Llc | Automatic generation of narratives from data using communication goals and narrative analytics |
US11475076B2 (en) | 2014-10-22 | 2022-10-18 | Narrative Science Inc. | Interactive and conversational data exploration |
US11238090B1 (en) | 2015-11-02 | 2022-02-01 | Narrative Science Inc. | Applied artificial intelligence technology for using narrative analytics to automatically generate narratives from visualization data |
US10430421B2 (en) * | 2014-12-29 | 2019-10-01 | Facebook, Inc. | Recommending content items in a social network using delayed interaction |
US10373181B2 (en) * | 2015-01-08 | 2019-08-06 | International Business Machines Corporation | Evaluating and displaying feedback for an item distributed to a group of users at a collaborative event |
US10673965B2 (en) * | 2015-08-28 | 2020-06-02 | Microsoft Technology Licensing, Llc | Adjusting heavy users' affinity for heavy user entity-pairs in a social network |
US11232268B1 (en) | 2015-11-02 | 2022-01-25 | Narrative Science Inc. | Applied artificial intelligence technology for using narrative analytics to automatically generate narratives from line charts |
US11170038B1 (en) | 2015-11-02 | 2021-11-09 | Narrative Science Inc. | Applied artificial intelligence technology for using narrative analytics to automatically generate narratives from multiple visualizations |
US11222184B1 (en) | 2015-11-02 | 2022-01-11 | Narrative Science Inc. | Applied artificial intelligence technology for using narrative analytics to automatically generate narratives from bar charts |
US10853583B1 (en) | 2016-08-31 | 2020-12-01 | Narrative Science Inc. | Applied artificial intelligence technology for selective control over narrative generation from visualizations of data |
US10943069B1 (en) | 2017-02-17 | 2021-03-09 | Narrative Science Inc. | Applied artificial intelligence technology for narrative generation based on a conditional outcome framework |
US11068661B1 (en) | 2017-02-17 | 2021-07-20 | Narrative Science Inc. | Applied artificial intelligence technology for narrative generation based on smart attributes |
US10699079B1 (en) | 2017-02-17 | 2020-06-30 | Narrative Science Inc. | Applied artificial intelligence technology for narrative generation based on analysis communication goals |
US10585983B1 (en) | 2017-02-17 | 2020-03-10 | Narrative Science Inc. | Applied artificial intelligence technology for determining and mapping data requirements for narrative stories to support natural language generation (NLG) using composable communication goals |
US11568148B1 (en) | 2017-02-17 | 2023-01-31 | Narrative Science Inc. | Applied artificial intelligence technology for narrative generation based on explanation communication goals |
US11042708B1 (en) | 2018-01-02 | 2021-06-22 | Narrative Science Inc. | Context saliency-based deictic parser for natural language generation |
US11003866B1 (en) | 2018-01-17 | 2021-05-11 | Narrative Science Inc. | Applied artificial intelligence technology for narrative generation using an invocable analysis service and data re-organization |
US11126798B1 (en) | 2018-02-19 | 2021-09-21 | Narrative Science Inc. | Applied artificial intelligence technology for conversational inferencing and interactive natural language generation |
US11334726B1 (en) | 2018-06-28 | 2022-05-17 | Narrative Science Inc. | Applied artificial intelligence technology for using natural language processing to train a natural language generation system with respect to date and number textual features |
US11314408B2 (en) | 2018-08-25 | 2022-04-26 | Microsoft Technology Licensing, Llc | Computationally efficient human-computer interface for collaborative modification of content |
US10990767B1 (en) | 2019-01-28 | 2021-04-27 | Narrative Science Inc. | Applied artificial intelligence technology for adaptive natural language understanding |
CN111078859B (en) * | 2019-11-22 | 2021-02-09 | 北京市科学技术情报研究所 | Author recommendation method based on reference times |
WO2021119119A1 (en) * | 2019-12-09 | 2021-06-17 | Miso Technologies Inc. | System and method for a personalized search and discovery engine |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050125216A1 (en) * | 2003-12-05 | 2005-06-09 | Chitrapura Krishna P. | Extracting and grouping opinions from text documents |
US20070124432A1 (en) * | 2000-10-11 | 2007-05-31 | David Holtzman | System and method for scoring electronic messages |
US20070294281A1 (en) * | 2006-05-05 | 2007-12-20 | Miles Ward | Systems and methods for consumer-generated media reputation management |
US20080126303A1 (en) * | 2006-09-07 | 2008-05-29 | Seung-Taek Park | System and method for identifying media content items and related media content items |
US20080183700A1 (en) * | 2007-01-31 | 2008-07-31 | Gabriel Raefer | Identifying and changing personal information |
US20090157667A1 (en) * | 2007-12-12 | 2009-06-18 | Brougher William C | Reputation of an Author of Online Content |
US20100257184A1 (en) * | 2006-12-20 | 2010-10-07 | Victor David Uy | Method and apparatus for scoring electronic documents |
US8554601B1 (en) * | 2003-08-22 | 2013-10-08 | Amazon Technologies, Inc. | Managing content based on reputation |
Family Cites Families (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5758257A (en) * | 1994-11-29 | 1998-05-26 | Herz; Frederick | System and method for scheduling broadcast of and access to video programs and other data using customer profiles |
US7143089B2 (en) * | 2000-02-10 | 2006-11-28 | Involve Technology, Inc. | System for creating and maintaining a database of information utilizing user opinions |
US6895385B1 (en) * | 2000-06-02 | 2005-05-17 | Open Ratings | Method and system for ascribing a reputation to an entity as a rater of other entities |
US20020062368A1 (en) * | 2000-10-11 | 2002-05-23 | David Holtzman | System and method for establishing and evaluating cross community identities in electronic forums |
US7617160B1 (en) * | 2003-02-05 | 2009-11-10 | Michael I. Grove | Choice-based relationship system (CRS) |
US7536315B2 (en) * | 2003-02-13 | 2009-05-19 | Sap Aktiengesellschaft | Self-balancing of idea ratings |
US7822631B1 (en) * | 2003-08-22 | 2010-10-26 | Amazon Technologies, Inc. | Assessing content based on assessed trust in users |
US8200477B2 (en) * | 2003-10-22 | 2012-06-12 | International Business Machines Corporation | Method and system for extracting opinions from text documents |
US7558769B2 (en) * | 2005-09-30 | 2009-07-07 | Google Inc. | Identifying clusters of similar reviews and displaying representative reviews from multiple clusters |
US20070143300A1 (en) * | 2005-12-20 | 2007-06-21 | Ask Jeeves, Inc. | System and method for monitoring evolution over time of temporal content |
US8374973B2 (en) * | 2006-02-16 | 2013-02-12 | Microsoft Corporation | Reputation system |
US7904448B2 (en) * | 2006-03-29 | 2011-03-08 | Yahoo! Inc. | Incremental update of long-term and short-term user profile scores in a behavioral targeting system |
US20080109245A1 (en) * | 2006-11-03 | 2008-05-08 | Sezwho Inc. | Method and system for managing domain specific and viewer specific reputation on online communities |
US20080243906A1 (en) * | 2007-03-31 | 2008-10-02 | Keith Peters | Online system and method for providing geographic presentations of localities that are pertinent to a text item |
-
2008
- 2008-06-28 US US12/215,684 patent/US8548996B2/en active Active
-
2013
- 2013-09-25 US US14/036,610 patent/US20140025690A1/en not_active Abandoned
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070124432A1 (en) * | 2000-10-11 | 2007-05-31 | David Holtzman | System and method for scoring electronic messages |
US8554601B1 (en) * | 2003-08-22 | 2013-10-08 | Amazon Technologies, Inc. | Managing content based on reputation |
US20050125216A1 (en) * | 2003-12-05 | 2005-06-09 | Chitrapura Krishna P. | Extracting and grouping opinions from text documents |
US20070294281A1 (en) * | 2006-05-05 | 2007-12-20 | Miles Ward | Systems and methods for consumer-generated media reputation management |
US20080126303A1 (en) * | 2006-09-07 | 2008-05-29 | Seung-Taek Park | System and method for identifying media content items and related media content items |
US20100257184A1 (en) * | 2006-12-20 | 2010-10-07 | Victor David Uy | Method and apparatus for scoring electronic documents |
US20080183700A1 (en) * | 2007-01-31 | 2008-07-31 | Gabriel Raefer | Identifying and changing personal information |
US20090157667A1 (en) * | 2007-12-12 | 2009-06-18 | Brougher William C | Reputation of an Author of Online Content |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120180138A1 (en) * | 2007-11-15 | 2012-07-12 | Yahoo! Inc. | Trust based moderation |
US9576253B2 (en) * | 2007-11-15 | 2017-02-21 | Yahoo! Inc. | Trust based moderation |
US10497032B2 (en) * | 2010-11-18 | 2019-12-03 | Ebay Inc. | Image quality assessment to merchandise an item |
US11282116B2 (en) | 2010-11-18 | 2022-03-22 | Ebay Inc. | Image quality assessment to merchandise an item |
US20130031107A1 (en) * | 2011-07-29 | 2013-01-31 | Jen-Yi Pan | Personalized ranking method of video and audio data on internet |
US20130332468A1 (en) * | 2012-06-07 | 2013-12-12 | Spigit, Inc. | User Reputation in Social Network and eCommerce Rating Systems |
US9159056B2 (en) | 2012-07-10 | 2015-10-13 | Spigit, Inc. | System and method for determining the value of a crowd network |
US10545938B2 (en) | 2013-09-30 | 2020-01-28 | Spigit, Inc. | Scoring members of a set dependent on eliciting preference data amongst subsets selected according to a height-balanced tree |
US11580083B2 (en) | 2013-09-30 | 2023-02-14 | Spigit, Inc. | Scoring members of a set dependent on eliciting preference data amongst subsets selected according to a height-balanced tree |
US9542616B1 (en) | 2015-06-29 | 2017-01-10 | International Business Machines Corporation | Determining user preferences for data visualizations |
US10599979B2 (en) | 2015-09-23 | 2020-03-24 | International Business Machines Corporation | Candidate visualization techniques for use with genetic algorithms |
US10607139B2 (en) | 2015-09-23 | 2020-03-31 | International Business Machines Corporation | Candidate visualization techniques for use with genetic algorithms |
US11651233B2 (en) | 2015-09-23 | 2023-05-16 | International Business Machines Corporation | Candidate visualization techniques for use with genetic algorithms |
US10685035B2 (en) | 2016-06-30 | 2020-06-16 | International Business Machines Corporation | Determining a collection of data visualizations |
US10949444B2 (en) | 2016-06-30 | 2021-03-16 | International Business Machines Corporation | Determining a collection of data visualizations |
Also Published As
Publication number | Publication date |
---|---|
US8548996B2 (en) | 2013-10-01 |
US20090049041A1 (en) | 2009-02-19 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8548996B2 (en) | Ranking content items related to an event | |
US9703877B2 (en) | Computer-based evaluation tool for selecting personalized content for users | |
US8140541B2 (en) | Time-weighted scoring system and method | |
CN107851097B (en) | Data analysis system, data analysis method, data analysis program, and storage medium | |
US8234311B2 (en) | Information processing device, importance calculation method, and program | |
US20160179958A1 (en) | Related entities | |
US8935263B1 (en) | Generating rankings of reputation scores in reputation systems | |
US9075882B1 (en) | Recommending content items | |
US10354308B2 (en) | Distinguishing accessories from products for ranking search results | |
US20050210025A1 (en) | System and method for predicting the ranking of items | |
US20150262081A1 (en) | Estimating reputation scores in reputation systems | |
CN105917364B (en) | Ranking discussion topics in question-and-answer forums | |
AU2017250467B2 (en) | Query optimizer for combined structured and unstructured data records | |
US8489604B1 (en) | Automated resource selection process evaluation | |
US9330125B2 (en) | Querying of reputation scores in reputation systems | |
US9336330B2 (en) | Associating entities based on resource associations | |
US20160085868A1 (en) | Digital search filter that presents results focused only on the most relevant solutions | |
Jenson | Peer reviewed: psychosocial and behavioral risk profiles of cigarette smokers and E-Cigarette users among adolescents in Minnesota: the 2016 Minnesota student survey | |
CN108009194A (en) | A kind of books method for pushing, electronic equipment, storage medium and device | |
Sato et al. | Exploring an optimal online model for new job recommendation: Solution for recsys challenge 2017 | |
JP6414192B2 (en) | Information processing apparatus, information processing method, and information processing program | |
Kamihata et al. | A quantitative contents diversity analysis on a consumer generated media site | |
JP4639388B2 (en) | Important word extraction method, important word extraction apparatus, computer program, and program storage medium in document database | |
WO2008032037A1 (en) | Method and system for filtering and searching data using word frequencies | |
JP6065061B2 (en) | Information processing device, terminal device, information processing method, display method, information processing program, and display program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: ALLVOICES, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TAREEN, AMRA Q.;SUNDELOF, ERIK;BIRNBAUM, LAWRENCE A.;AND OTHERS;SIGNING DATES FROM 20080929 TO 20081001;REEL/FRAME:031349/0685 |
|
AS | Assignment |
Owner name: DATRAN MEDIA CORP., NEW YORK Free format text: OFFICER'S CERTIFICATE;ASSIGNOR:ALLVOICES, INC.;REEL/FRAME:031352/0574 Effective date: 20110428 |
|
AS | Assignment |
Owner name: PULSEPOINT, INC., NEW YORK Free format text: MERGER;ASSIGNOR:DATRAN MEDIA CORP.;REEL/FRAME:031359/0581 Effective date: 20110921 |
|
AS | Assignment |
Owner name: PULSEPOINT, INC., NEW YORK Free format text: CORRECTION OF COVER SHEET ERROR FOR REEL/FRAME 031359/0581 OF ASSIGNEE ADDRESS. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNOR(S) INTEREST;ASSIGNOR:DATRAN MEDIA CORP.;REEL/FRAME:031550/0875 Effective date: 20110921 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |