WO2012092224A3 - Filtering queried data on data stores - Google Patents

Filtering queried data on data stores Download PDF

Info

Publication number
WO2012092224A3
WO2012092224A3 PCT/US2011/067307 US2011067307W WO2012092224A3 WO 2012092224 A3 WO2012092224 A3 WO 2012092224A3 US 2011067307 W US2011067307 W US 2011067307W WO 2012092224 A3 WO2012092224 A3 WO 2012092224A3
Authority
WO
WIPO (PCT)
Prior art keywords
data
query
processing
node
stores
Prior art date
Application number
PCT/US2011/067307
Other languages
French (fr)
Other versions
WO2012092224A2 (en
Inventor
Nir Nice
Daniel Sitton
Dror Kremer
Michael Feldman
Original Assignee
Microsoft Corporation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Corporation filed Critical Microsoft Corporation
Priority to CA2822900A priority Critical patent/CA2822900C/en
Priority to JP2013547600A priority patent/JP5990192B2/en
Priority to EP11854242.2A priority patent/EP2659403A4/en
Publication of WO2012092224A2 publication Critical patent/WO2012092224A2/en
Publication of WO2012092224A3 publication Critical patent/WO2012092224A3/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/903Querying
    • G06F16/90335Query processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2453Query optimisation
    • G06F16/24532Query optimisation of parallel queries

Abstract

A data set may be distributed over many data stores, and a query may be distributively evaluated by several data stores with the results combined to form a query result (e.g., utilizing a MapReduce framework). However, such architectures may violate security principles by performing sophisticated processing, including the execution of arbitrary code, on the same machines that store the data. Instead of processing queries, a data store may be configured only to receive requests specifying one or more filtering criteria, and to provide the data items satisfying the filtering criteria. A compute node may apply a query by generating a request including one o more filter criteria, providing the request to a data node, and applying the remainder of the query (including sophisticated processing, and potentially the execution of arbitrary code) to the data items provided by the data node, thereby improving the security and efficiency of query processing.
PCT/US2011/067307 2010-12-28 2011-12-24 Filtering queried data on data stores WO2012092224A2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CA2822900A CA2822900C (en) 2010-12-28 2011-12-24 Filtering queried data on data stores
JP2013547600A JP5990192B2 (en) 2010-12-28 2011-12-24 Filtering query data in the data store
EP11854242.2A EP2659403A4 (en) 2010-12-28 2011-12-24 Filtering queried data on data stores

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US12/979,467 US10311105B2 (en) 2010-12-28 2010-12-28 Filtering queried data on data stores
US12/979,467 2010-12-28

Publications (2)

Publication Number Publication Date
WO2012092224A2 WO2012092224A2 (en) 2012-07-05
WO2012092224A3 true WO2012092224A3 (en) 2012-10-11

Family

ID=46318296

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2011/067307 WO2012092224A2 (en) 2010-12-28 2011-12-24 Filtering queried data on data stores

Country Status (7)

Country Link
US (2) US10311105B2 (en)
EP (1) EP2659403A4 (en)
JP (1) JP5990192B2 (en)
CN (1) CN102682052B (en)
CA (1) CA2822900C (en)
HK (1) HK1174111A1 (en)
WO (1) WO2012092224A2 (en)

Families Citing this family (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8918388B1 (en) * 2010-02-26 2014-12-23 Turn Inc. Custom data warehouse on top of mapreduce
US20130091266A1 (en) 2011-10-05 2013-04-11 Ajit Bhave System for organizing and fast searching of massive amounts of data
US10534606B2 (en) 2011-12-08 2020-01-14 Oracle International Corporation Run-length encoding decompression
US9792117B2 (en) 2011-12-08 2017-10-17 Oracle International Corporation Loading values from a value vector into subregisters of a single instruction multiple data register
US9697174B2 (en) 2011-12-08 2017-07-04 Oracle International Corporation Efficient hardware instructions for processing bit vectors for single instruction multiple data processors
US9600522B2 (en) * 2012-08-20 2017-03-21 Oracle International Corporation Hardware implementation of the aggregation/group by operation: filter method
US9563658B2 (en) 2012-08-20 2017-02-07 Oracle International Corporation Hardware implementation of the aggregation/group by operation: hash-table method
US9727606B2 (en) 2012-08-20 2017-08-08 Oracle International Corporation Hardware implementation of the filter/project operations
CN103034678A (en) * 2012-11-01 2013-04-10 沈阳建筑大学 RkNN (reverse k nearest neighbor) inquiring method based on Voronoi diagram
US9152671B2 (en) * 2012-12-17 2015-10-06 General Electric Company System for storage, querying, and analysis of time series data
CN103257987A (en) * 2012-12-30 2013-08-21 北京讯鸟软件有限公司 Rule-based distributed log service implementation method
US8694503B1 (en) * 2013-07-31 2014-04-08 Linkedin Corporation Real-time indexing of data for analytics
US11113054B2 (en) 2013-09-10 2021-09-07 Oracle International Corporation Efficient hardware instructions for single instruction multiple data processors: fast fixed-length value compression
US9606921B2 (en) 2013-09-21 2017-03-28 Oracle International Corporation Granular creation and refresh of columnar data
CN106062739B (en) * 2013-11-12 2020-02-28 皮沃塔尔软件公司 Dynamic stream computation topology
KR102103543B1 (en) * 2013-11-28 2020-05-29 삼성전자 주식회사 All-in-one data storage device having internal hardware filter, method thereof, and system having the data storage device
US9886310B2 (en) * 2014-02-10 2018-02-06 International Business Machines Corporation Dynamic resource allocation in MapReduce
US10007692B2 (en) * 2014-03-27 2018-06-26 Microsoft Technology Licensing, Llc Partition filtering using smart index in memory
US10409835B2 (en) * 2014-11-28 2019-09-10 Microsoft Technology Licensing, Llc Efficient data manipulation support
US9892164B2 (en) 2015-01-30 2018-02-13 International Business Machines Corporation Reducing a large amount of data to a size available for interactive analysis
US10025823B2 (en) 2015-05-29 2018-07-17 Oracle International Corporation Techniques for evaluating query predicates during in-memory table scans
US10067954B2 (en) 2015-07-22 2018-09-04 Oracle International Corporation Use of dynamic dictionary encoding with an associated hash table to support many-to-many joins and aggregations
US10666574B2 (en) * 2015-09-28 2020-05-26 Amazon Technologies, Inc. Distributed stream-based database triggers
US10565209B2 (en) * 2015-12-01 2020-02-18 International Business Machines Corporation Injecting outlier values
US10061832B2 (en) 2016-11-28 2018-08-28 Oracle International Corporation Database tuple-encoding-aware data partitioning in a direct memory access engine
US10055358B2 (en) 2016-03-18 2018-08-21 Oracle International Corporation Run length encoding aware direct memory access filtering engine for scratchpad enabled multicore processors
US10402425B2 (en) 2016-03-18 2019-09-03 Oracle International Corporation Tuple encoding aware direct memory access engine for scratchpad enabled multi-core processors
US10061714B2 (en) 2016-03-18 2018-08-28 Oracle International Corporation Tuple encoding aware direct memory access engine for scratchpad enabled multicore processors
US11188542B2 (en) * 2016-05-03 2021-11-30 Salesforce.Com, Inc. Conditional processing based on data-driven filtering of records
US10599488B2 (en) 2016-06-29 2020-03-24 Oracle International Corporation Multi-purpose events for notification and sequence control in multi-core processor systems
US10380058B2 (en) 2016-09-06 2019-08-13 Oracle International Corporation Processor core to coprocessor interface with FIFO semantics
US10783102B2 (en) 2016-10-11 2020-09-22 Oracle International Corporation Dynamically configurable high performance database-aware hash engine
US10176114B2 (en) 2016-11-28 2019-01-08 Oracle International Corporation Row identification number generation in database direct memory access engine
US10459859B2 (en) 2016-11-28 2019-10-29 Oracle International Corporation Multicast copy ring for database direct memory access filtering engine
US10725947B2 (en) 2016-11-29 2020-07-28 Oracle International Corporation Bit vector gather row count calculation and handling in direct memory access engine
US10725799B2 (en) * 2017-02-22 2020-07-28 Microsoft Technology Licensing, Llc Big data pipeline management within spreadsheet applications
US11157690B2 (en) 2017-02-22 2021-10-26 Microsoft Technology Licensing, Llc Techniques for asynchronous execution of computationally expensive local spreadsheet tasks
CN107451204B (en) * 2017-07-10 2021-01-05 创新先进技术有限公司 Data query method, device and equipment
CN108920602B (en) * 2018-06-28 2021-12-14 北京京东尚科信息技术有限公司 Method and apparatus for outputting information
US10997177B1 (en) 2018-07-27 2021-05-04 Workday, Inc. Distributed real-time partitioned MapReduce for a data fabric
CN111163056B (en) * 2019-12-06 2021-08-31 西安电子科技大学 Data confidentiality method and system aiming at MapReduce calculation

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6931389B1 (en) * 1997-10-14 2005-08-16 International Business Machines Corporation System and method for filtering query statements from multiple plans and packages according to user-defined filters of query explain data
US20080201303A1 (en) * 2007-02-20 2008-08-21 International Business Machines Corporation Method and system for a wizard based complex filter with realtime feedback
US20080288524A1 (en) * 2007-05-18 2008-11-20 Microsoft Corporation Filtering of multi attribute data via on-demand indexing
US20090125482A1 (en) * 2007-11-12 2009-05-14 Peregrine Vladimir Gluzman System and method for filtering rules for manipulating search results in a hierarchical search and navigation system

Family Cites Families (60)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5421008A (en) * 1991-11-08 1995-05-30 International Business Machines Corporation System for interactive graphical construction of a data base query and storing of the query object links as an object
US5560007A (en) * 1993-06-30 1996-09-24 Borland International, Inc. B-tree key-range bit map index optimization of database queries
US5584024A (en) * 1994-03-24 1996-12-10 Software Ag Interactive database query system and method for prohibiting the selection of semantically incorrect query parameters
US5787411A (en) * 1996-03-20 1998-07-28 Microsoft Corporation Method and apparatus for database filter generation by display selection
US6493700B2 (en) 1997-10-14 2002-12-10 International Business Machines Corporation System and method for specifying custom qualifiers for explain tables
EP1350184B1 (en) * 2000-05-26 2014-11-19 CA, Inc. System and method for automatically generating database queries
US6925608B1 (en) * 2000-07-05 2005-08-02 Kendyl A. Roman Graphical user interface for building Boolean queries and viewing search results
US6785668B1 (en) * 2000-11-28 2004-08-31 Sas Institute Inc. System and method for data flow analysis of complex data filters
US20020174295A1 (en) 2001-01-29 2002-11-21 Ulrich Thomas R. Enhanced file system failure tolerance
US6931390B1 (en) * 2001-02-27 2005-08-16 Oracle International Corporation Method and mechanism for database partitioning
US6778977B1 (en) * 2001-04-19 2004-08-17 Microsoft Corporation Method and system for creating a database table index using multiple processors
US6789071B1 (en) * 2001-04-20 2004-09-07 Microsoft Corporation Method for efficient query execution using dynamic queries in database environments
US20020194166A1 (en) * 2001-05-01 2002-12-19 Fowler Abraham Michael Mechanism to sift through search results using keywords from the results
US7315894B2 (en) 2001-07-17 2008-01-01 Mcafee, Inc. Network data retrieval and filter systems and methods
AU2002322835A1 (en) 2001-08-01 2003-02-17 Kim Updike Methods and apparatus for fairly placing players in bet positions
US7024414B2 (en) * 2001-08-06 2006-04-04 Sensage, Inc. Storage of row-column data
US8868544B2 (en) * 2002-04-26 2014-10-21 Oracle International Corporation Using relational structures to create and support a cube within a relational database system
US7136850B2 (en) * 2002-12-20 2006-11-14 International Business Machines Corporation Self tuning database retrieval optimization using regression functions
US7584264B2 (en) 2003-08-19 2009-09-01 Google Inc. Data storage and retrieval systems and related methods of storing and retrieving data
US20070078826A1 (en) * 2005-10-03 2007-04-05 Tolga Bozkaya Analytic enhancements to model clause in structured query language (SQL)
US20050131893A1 (en) * 2003-12-15 2005-06-16 Sap Aktiengesellschaft Database early parallelism method and system
US20050192942A1 (en) * 2004-02-27 2005-09-01 Stefan Biedenstein Accelerated query refinement by instant estimation of results
US7299220B2 (en) * 2004-03-31 2007-11-20 Microsoft Corporation Constructing database object workload summaries
US7650331B1 (en) 2004-06-18 2010-01-19 Google Inc. System and method for efficient large-scale data processing
US7756881B2 (en) * 2006-03-09 2010-07-13 Microsoft Corporation Partitioning of data mining training set
JP4696011B2 (en) * 2006-03-24 2011-06-08 パナソニック株式会社 Gravure coating equipment
US7461050B2 (en) * 2006-03-30 2008-12-02 International Business Machines Corporation Methods of cost estimation using partially applied predicates
US20070250470A1 (en) * 2006-04-24 2007-10-25 Microsoft Corporation Parallelization of language-integrated collection operations
US8555288B2 (en) * 2006-05-17 2013-10-08 Teradata Us, Inc. Managing database utilities to improve throughput and concurrency
US7624118B2 (en) * 2006-07-26 2009-11-24 Microsoft Corporation Data processing over very large databases
US7962442B2 (en) * 2006-08-31 2011-06-14 International Business Machines Corporation Managing execution of a query against selected data partitions of a partitioned database
US8700579B2 (en) * 2006-09-18 2014-04-15 Infobright Inc. Method and system for data compression in a relational database
GB0625641D0 (en) 2006-12-21 2007-01-31 Symbian Software Ltd Dynamic filtering for partially trusted servers
JP2008181243A (en) * 2007-01-23 2008-08-07 Hitachi Ltd Database management system for controlling setting of cache partition region of storage system
US8156107B2 (en) * 2007-02-02 2012-04-10 Teradata Us, Inc. System and method for join-partitioning for local computability of query over shared-nothing clusters
US20090006347A1 (en) * 2007-06-29 2009-01-01 Lucent Technologies Inc. Method and apparatus for conditional search operators
US7895151B2 (en) * 2008-06-23 2011-02-22 Teradata Us, Inc. Fast bulk loading and incremental loading of data into a database
US8250088B2 (en) * 2007-10-05 2012-08-21 Imation Corp. Methods for controlling remote archiving systems
US8386508B2 (en) * 2008-04-28 2013-02-26 Infosys Technologies Limited System and method for parallel query evaluation
JP4207096B2 (en) 2008-06-12 2009-01-14 株式会社日立製作所 Database management method
US8364751B2 (en) * 2008-06-25 2013-01-29 Microsoft Corporation Automated client/server operation partitioning
US8041714B2 (en) 2008-09-15 2011-10-18 Palantir Technologies, Inc. Filter chains with associated views for exploring large data sets
US8145806B2 (en) * 2008-09-19 2012-03-27 Oracle International Corporation Storage-side storage request management
EP2169563A1 (en) * 2008-09-26 2010-03-31 Siemens Aktiengesellschaft Method for performing a database query in a relational database
US8478775B2 (en) 2008-10-05 2013-07-02 Microsoft Corporation Efficient large-scale filtering and/or sorting for querying of column based data encoded structures
US7917463B2 (en) * 2008-10-10 2011-03-29 Business.Com, Inc. System and method for data warehousing and analytics on a distributed file system
US20100121869A1 (en) * 2008-11-07 2010-05-13 Yann Le Biannic Normalizing a filter condition of a database query
US10831724B2 (en) * 2008-12-19 2020-11-10 Bmc Software, Inc. Method of reconciling resources in the metadata hierarchy
US8533181B2 (en) * 2009-04-29 2013-09-10 Oracle International Corporation Partition pruning via query rewrite
US8103638B2 (en) * 2009-05-07 2012-01-24 Microsoft Corporation Partitioning of contended synchronization objects
CN101963965B (en) * 2009-07-23 2013-03-20 阿里巴巴集团控股有限公司 Document indexing method, data query method and server based on search engine
US9135299B2 (en) * 2009-09-01 2015-09-15 Teradata Us, Inc. System, method, and computer-readable medium for automatic index creation to improve the performance of frequently executed queries in a database system
US8874600B2 (en) * 2010-01-30 2014-10-28 International Business Machines Corporation System and method for building a cloud aware massive data analytics solution background
JP5593792B2 (en) * 2010-03-31 2014-09-24 富士通株式会社 RAID device, storage control method, and storage control program
US8606756B2 (en) * 2010-04-09 2013-12-10 Ca, Inc. Distributed system having a shared central database
US8386471B2 (en) * 2010-05-27 2013-02-26 Salesforce.Com, Inc. Optimizing queries in a multi-tenant database system environment
US8370316B2 (en) * 2010-07-12 2013-02-05 Sap Ag Hash-join in parallel computation environments
US8392399B2 (en) * 2010-09-16 2013-03-05 Microsoft Corporation Query processing algorithm for vertically partitioned federated database systems
US8413145B2 (en) * 2010-09-30 2013-04-02 Avaya Inc. Method and apparatus for efficient memory replication for high availability (HA) protection of a virtual machine (VM)
US8818989B2 (en) * 2010-11-30 2014-08-26 International Business Machines Corporation Memory usage query governor

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6931389B1 (en) * 1997-10-14 2005-08-16 International Business Machines Corporation System and method for filtering query statements from multiple plans and packages according to user-defined filters of query explain data
US20080201303A1 (en) * 2007-02-20 2008-08-21 International Business Machines Corporation Method and system for a wizard based complex filter with realtime feedback
US20080288524A1 (en) * 2007-05-18 2008-11-20 Microsoft Corporation Filtering of multi attribute data via on-demand indexing
US20090125482A1 (en) * 2007-11-12 2009-05-14 Peregrine Vladimir Gluzman System and method for filtering rules for manipulating search results in a hierarchical search and navigation system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP2659403A4 *

Also Published As

Publication number Publication date
CN102682052B (en) 2015-08-19
HK1174111A1 (en) 2013-05-31
JP2014502762A (en) 2014-02-03
EP2659403A4 (en) 2017-06-07
US10311105B2 (en) 2019-06-04
JP5990192B2 (en) 2016-09-07
WO2012092224A2 (en) 2012-07-05
CA2822900A1 (en) 2012-07-05
CN102682052A (en) 2012-09-19
US20120166447A1 (en) 2012-06-28
EP2659403A2 (en) 2013-11-06
US20190266195A1 (en) 2019-08-29
CA2822900C (en) 2020-01-07

Similar Documents

Publication Publication Date Title
WO2012092224A3 (en) Filtering queried data on data stores
WO2014145218A3 (en) Networked management of dispensables
WO2013041852A3 (en) Scalable distributed transaction processing system
WO2016018472A3 (en) Content-based association of device to user
MX2015008570A (en) Modifying structured search queries on online social networks.
WO2014052677A3 (en) Mechanism to chain continuous queries
MX347812B (en) Using inverse operators for queries on online social networks.
WO2014145884A3 (en) Syntactic tagging in a domain-specific context
WO2014093248A3 (en) Query and index over documents
WO2014201238A3 (en) Multi-tenant enabling a single-tenant computer program product
WO2012122410A3 (en) Platform for rapid development of applications
GB2526231A (en) Flexible store fulfillment
WO2016109672A3 (en) Feed data storage and query
CA2879417A1 (en) Structured search queries based on social-graph information
JP2011076709A5 (en)
WO2016099902A3 (en) Efficiently providing virtual machine reference points
EP3614267A3 (en) Recoverable stream processing
WO2009042911A3 (en) Search based data management
WO2017052811A3 (en) Secure modular exponentiation processors, methods, systems, and instructions
WO2014055764A3 (en) Generating snippets for prominent users for information retrieval queries
PH12017550081A1 (en) Conditionally controlled styling
MX2015008571A (en) Ambiguous structured search queries on online social networks.
EP3267388A4 (en) Commerce system, management server, and program
WO2015008144A3 (en) Interactive or code management system
WO2013167052A3 (en) Internet information management method and system, and server

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11854242

Country of ref document: EP

Kind code of ref document: A2

ENP Entry into the national phase

Ref document number: 2822900

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 2011854242

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2013547600

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE