Patent US7917514 - Visual and multi-dimensional search

IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIW

US007917514B2

(12) United States Patent ao) Patent No.: Us 7,917,514 B2

Lawler et al. (45) Date of Patent: Mar. 29,2011

(73) Assignee: Microsoft Corporation, Redmond, WA (US)

( * ) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 U.S.C. 154(b) by 221 days.

(21) Appl.No.: 11/427,305

(22) Filed: Jun. 28, 2006

(65) Prior Publication Data

US 2008/0005105 Al Jan. 3, 2008

(51) Int. CI.

G06F 7/00 (2006.01)
G06F17/30 (2006.01)

(52) U.S. CI 707/739; 707/711

(58) Field of Classification Search 707/1 10,

707/711, 739 See application file for complete search history.

(56) References Cited

U.S. PATENT DOCUMENTS

A system that can analyze a multi-dimensional input thereafter establishing a search query based upon extracted features from the input. In a particular example, an image can be used as an input to a search mechanism. Pattern recognition and image analysis can be applied to the image thereafter establishing a search query that corresponds to features extracted from the image input. The system can also facilitate indexing multi-dimensional searchable items thereby making them available to be retrieved as results to a search query. More particularly, the system can employ text analysis, pattern and/or speech recognition mechanisms to extract features from searchable items. These extracted features can be employed to index the searchable items.

20 Claims, 14 Drawing Sheets

Page 2

U.S. PATENT DOCUMENTS

2001/0040590 Al 11/2001 Abbott etal.

2001/0040591 Al 11/2001 Abbott etal.

2001/0043231 Al 11/2001 Abbott etal.

2001/0043232 Al 11/2001 Abbott etal.

2001/0046332 Al * 11/2001 Chang 382/305

2002/0032689 Al 3/2002 Abbott, III et al.

2002/0044152 Al 4/2002 Abbott, III et al.

2002/0052930 Al 5/2002 Abbott etal.

2002/0052963 Al 5/2002 Abbott etal.

2002/0054130 Al 5/2002 Abbott, III et al.

2002/0054174 Al 5/2002 Abbott etal.

2002/0078204 Al 6/2002 Newell et al.

2002/0080155 Al 6/2002 Abbott etal.

2002/0080156 Al 6/2002 Abbott etal.

2002/0083025 Al 6/2002 Robartsetal.

2002/0083158 Al 6/2002 Abbott etal.

2002/0087525 Al 7/2002 Abbott etal.

2002/0095412 Al * 7/2002 Lee et al 707/7

2002/0099817 Al 7/2002 Abbott etal.

2003/0046401 Al 3/2003 Abbott etal.

2003/0123721 Al * 7/2003 Megiddoetal 382/159

2003/0154476 Al 8/2003 Abbott, III et al.

2003/0177111 Al* 9/2003 Egendorfetal 707/3

2003/0195883 Al * 10/2003 Moj silovic et al 707/6

2004/0024790 Al 2/2004 Everett

2005/0021512 Al * 1/2005 Koenig 707/3

2005/0034078 Al 2/2005 Abbott etal.

2005/0084154 Al * 4/2005 Li etal 382/190

2005/0108001 Al* 5/2005 Aarskog 704/10

2005/0289448 Al * 12/2005 Megiddoetal 715/501.1

2006/0012677 Al 1/2006 Nevenetal.

2006/0015492 Al * 1/2006 Keating etal 707/4

2006/0115108 Al* 6/2006 Rodriguez et al 382/100

2006/0116994 Al * 6/2006 Jonkeretal 707/3

2006/0143176 Al * 6/2006 Moj silovic et al 707/6

2006/0221357 Al * 10/2006 Uzawa 358/1.1

2006/0227992 Al * 10/2006 Rathusetal 382/100

2007/0033170 Al * 2/2007 Sulletal 707/3

2008/0065964 Al * 3/2008 Zarrinkoub et al 714/776

OTHER PUBLICATIONS

Guanling Chen, et al., A Survey of Context-Aware Mobile Computing Research, Dartmouth Computer Science Technical Report, 2000, 16 pages.

William Noah Schilt, A System Architecture for Context-Aware
Mobile Computing, Columbia University, 1995, 153 pages.
Mike Spreitzer, et al., Providing Location Information in a Ubiqui-
tous Computing Environment, SIGOPS '93, 1993, pp. 270-283.
Marvin Theimer, et al., Operating System Issues for PDAs, In Fourth
Workshop on Workstation Operating Systems, 1993, 7 pages.
Roy Want, Active Badges and Personal Interactive Computing
Objects, IEEE Transactions on Consumer Electronics, 1992, 11
pages, vol. 38-No. 1.

Bill N. Schilit, et al., The ParcTab Mobile Computing System, IEEE
WWOS-IV, 1993, 4 pages.

Bill Schilit, et al., Context-Aware Computing Applications, In Proceedings of the Workshop on Mobile Computing Systems and Applications, Dec. 1994. pp. 85-90.

BillN. Schilit, etal., Customizing Mobile Applications, Proceedings USENIX Symposium on Mobile and Location Independent Computing, Aug. 1993, 9 pages.

Mike Spreitzer, et al., Architectural Considerations for Scalable, Secure, Mobile Computing with Location Information, In The 14th International Conference on Distributed Computing Systems, Jun. 1994, pp. 29-38.

Mike Spreitzer et al., Scalable, Secure, Mobile Computing with Location Information, Communications of the ACM, Jul. 1993, 1 page, vol. 36-No. 7.

Roy Want, et al., The Active Badge Location System, ACM Transactions on Information Systems, Jan. 1992,pp.91-102,vol. 10-No. 1. Mark Weiser, Some Computer Science Issues in Ubiquitous Computing, Communications of the ACM, Jul. 1993, pp. 75-84, vol. 36-No. 7.

M. Billinghurst, et al., An Evaluation of Wearable Information Spaces, Proceedings of the Virtual Reality Annual International Symposium, 1998, 8 pages.

Bradley J. Rhodes, Remembrance Agent: A continuously running automated information retrieval system, The Proceedings of the First International Conference on the Practical Application of Intelligent Agents and Multi Agent Technology, 1996, pp. 487-495. Eric Horvitz, et al., In Pursuit of Effective Handsfree Decision Support: Coupling Bayesian Inference, Speech Understanding, and User Models, 1995, 8 pages.

Bradley J. Rhodes, The Wearable Remembrance Agent: A System for
Augmented Theory, The Proceedings of the First International Sym-
posium on Wearable Computers, Oct. 1997, pp. 123-128.
Eric Horvitz, et al., Attention-Sensitive Alerting in Computing Sys-
tems, Microsoft Research, Aug. 1999.

Bill N. Schilit, et al., Disseminationg Active Map Information to
Mobile Hosts, IEEE Network, 1994, pp. 22-32, vol. 8-No. 5.
Mark Billinghurst, et al., Wearable Devices: New Ways to Manage
Information, IEEE Computer Society, Jan. 1999, pp. 57-64.
Thad Eugene Starner, Wearable Computing and Contextual Aware-
ness, Massachusetts Institute of Technology, Jun. 1999, 248 pages.
Bradley J. Rhodes, The Wearable Remembrance Agent: A System for
Augmented Memory, Personal Technologies Journal Special Issue
on Wearable Computing, 1997, 12 pages.
Workshop on Wearable Computing Systems, Aug. 19-21, 1996.
Mark Billinghurst, Research Directions in Wearable Computing,
University of Washington, May 1998, 48 pages.
Mark Weiser, The Computer for the 21st Century, Scientific Ameri-
can, Sep. 1991, 8 pages.

T. Joachims, Text categorization with supportvectormachines: learning with many relevant features, Machine Learning, European Conference on Machine Learning, Apr. 21, 1998, pp. 137-142. International Search Report dated Sep. 29, 2003 for PCT Application Serial No. 00/20685, 3 pages.

Robert M. Losee, Jr., Minimizing information overload: the ranking of electronic messages, Journal of Information Science 15, Elsevier Science Publishers B.V., 1989, pp. 179-189.

Funkhouser, et al., "A Search Engine for 3D Models", ACM Transactions on Graphics, Jan. 2003.

Image Understanding, Stuart Shapiro, http://www.aaai.org/AITopics/pmwiki/pmwiki.php/AITopics/ImageUnderstanding, 1996, pp. 1-13.

International Search Report dated Nov. 30,2007 for PCT Application Serial No. PCT/US2007/014990, 7 pages.

Spoken dialogue technology: enabling the conversational user interface, Michael McTear, ACM, Mar. 2002, pp. 1-80. Wactar, et al., Complimentary Video and Audio Analysis for Broadcast News Archives, Communications of the ACM vol. 43, No. 2, Feb. 2000.

Yeh, et al., "Searching the Web with Mobile Images for Location Recognition". Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVRP'04). Yeh, et al., "A Picture is Worth a Thousand Keywords: Image-Based Object Search on a Mobile Platform", In CHI 2005, Conference on Human Factors in Computing Systems, Portland, OR, Apr. 2005.

* cited by examiner

[merged small][table][merged small][merged small][merged small][merged small][merged small][merged small][merged small]

« ZurückWeiter »

Patente

Visual and multi-dimensional search